IJAHP Essay: Mu/Reporting public multicriteria decision-making applications: a journal editor’s perspective International Journal of the Analytic Hierarchy Process 1 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1025 Reporting Public Multicriteria Decision-Making Applications: A Journal Editor’s Perspective The following reflections refer to the development and reporting of public decision- making studies. These observations are provided with the hope of improving the quality and validity of multicriteria decision-making (MCDM) studies in the public sector. While my thoughts here are not intended to be a comprehensive study of how to report public MCDM studies, my intention is to highlight some key elements that are often not properly addressed by researchers. These observations are particularly relevant for AHP/ANP applications due to their widespread use in public decision-making. The Analytic Hierarchy/Network Process (AHP/ANP) is commonly used in practical applications in many different areas of decision-making. As a matter of fact, the AHP is one of, if not the most widely used multicriteria decision-making analysis methodologies worldwide. While the AHP calculations are rather simple to perform using a spreadsheet (Mu & Pereyra-Rojas, 2018), there are currently many AHP software packages to facilitate the analysis (Goepel, 2018; Solutions, 2022; SuperDecisions, 2020). In situations where either AHP axioms (e.g., criteria independence) do not hold or the situation of mutual interaction (e.g., among the alternatives) arises, a generalization, the Analytic Network Process, is used (Saaty, 2004). To date, the only package, to my knowledge, that can help perform ANP analysis is SuperDecisions (2020) developed by the Creative Decisions Foundation (2022). This software also facilitates traditional AHP calculations. The success of AHP/ANP in addressing societal issues is due mainly to its intuitive simplicity which allows public participation through the incorporation of multiple stakeholders, who may not need to be trained in the intricacies of MCDA, to express their opinions in a way that can be quantitatively incorporated into the MCDA process. Public participation in the decision-making process requires a systematic “engagement of all the societal actors (research, industry, policy-makers, and civil society) and their joint participation in the research and innovation process” (Strand et al., 2015). At the very least, and when public participation is not directly possible, a group of experts is conveyed to represent the various stakeholders’ interests. Many published AHP/ANP studies use one of these two approaches (Gonzalez-Urango, 2018; Mu & Stern, 2014). As an example, let’s consider a public decision among different urban developments. This decision will most likely require decision-making consultation and broader participation from the public. For this purpose, stakeholders will need to be identified and prioritized to ensure that the consideration of their perspectives and interests, as well as their participation, is proportional to their relative importance. There are several ways to engage stakeholders, and selecting the preferred method will be dictated by the specific context of the decision; however, the most important aspect, in this case, is that there is a proper systematic engagement of the stakeholders and that this is reported in the article. Unfortunately, many analysts either do not perform a systematic stakeholder analysis or do not report it when submitting their manuscripts for publication (Gonzalez-Urango, Mu, & García-Melón, 2021). This lack of precision in the research report casts a shadow on the validity of the overall study. IJAHP Essay: Mu/Reporting public multicriteria decision-making applications: a journal editor’s perspective International Journal of the Analytic Hierarchy Process 2 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1025 In cases where broad participatory decision making is pursued, it is necessary to consult stakeholders using either a survey methodology, or a face-to-face, virtual, or hybrid group decision making approach. In the case of using a survey analysis, proper survey reporting is needed (e.g., response rate, sample of questions). Survey research has a long tradition in science, and what needs to be reported and how it is reported is highly standardized. A description of survey development and reliability tests are part of what an editor would expect as reasonable elements to report for this approach (Burns & Kho, 2015). For this reason, analysts should pay careful attention to follow established practices when reporting MCDA survey use. Sometimes, during the MCDA stage, handling stakeholders directly may be too cumbersome and an MCDA team that represents the various stakeholders’ perspectives is used instead. This is what constitutes a group decision-making approach (Saaty & Peniwati, 2008). In this situation, the analysts are required to report group-decision methodological aspects such as how the group membership was decided (to ensure that specific members represent the needs of specific stakeholders); the group process followed (e.g., consensus, voting); and how the various perspectives were aggregated. In the AHP/ANP, the aggregation of perspectives can take place during the individual comparison judgments or at the final priorities. Many articles do not provide enough information to assess the group decision-making protocol, again causing doubt about the overall validity of the MCDA study (Mu & Cooper, 2022). In conclusion, while authors must report their MCDA analysis according to the established practices in the field, they must also be mindful of the methodological context (e.g., stakeholder engagement, group decision-making) and report according to best practices in the corresponding methodological approaches that complement their central MCDA methodology (e.g., AHP/ANP). Not doing so casts a shadow over the validity of the MCDM study and jeopardizes the opportunities for publication. IJAHP Essay: Mu/Reporting public multicriteria decision-making applications: a journal editor’s perspective International Journal of the Analytic Hierarchy Process 3 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1025 REFERENCES Burns, K. E. A., & Kho, M. E. (2015). How to assess a survey report: a guide for readers and peer reviewers. CMAJ, 187(6), E198-E205. Doi:10.1503/cmaj.140545 Creative_Decisions_Foundation. (2022). Changing the way we measure things. Goepel, K. D. (2018). Implementation of an online software tool for the Analytic Hierarchy Process (AHP-OS). International Journal of the Analytic Hierarchy Process, 10(3), 469-487. Doi:https://doi.org/10.13033/ijahp.v10i3.590 Gonzalez-Urango, H. (2018). A combined social network analysis-Analytic Network Process approach to evaluate sustainable tourist strategies. Paper presented at the Latin American Studies Association, Barcelona, Spain. Gonzalez-Urango, H., Mu, E., & García-Melón, M. (2021). Stakeholder engagement and ANP best research practices in sustainable territorrial and urban strategic planning. In M. Doumpos, F. Ferreira, & C. Zopounidis (Eds.), Multiple Criteria Decision Making for sustainable development. Switzerland: Springer Nature. Mu, E., & Cooper, O. (2022). A contingency approach to multi-criteria decision-making: A search for validity through rigor and relevance. In I. Topcu, S. O. Ekici, O. Kabak, E. Aktas, & O. Ozaydin (Eds.), New perspectives in operations research and management science. London: Springer. Mu, E., & Pereyra-Rojas, M. (2018). Practical decision making using Super Decisions v3: Springer. Mu, E., & Stern, H. (2014). The City of Pittsburgh goes to the cloud: a case study of cloud strategic selection and deployment. Journal of Information Technology Teaching Cases, 4, 70-85. Saaty, T. L. (2004). Fundamentals of the Analytic Network Process - dependence and feedback in decision making with a single network. Journal of Systems Science and Systems Engineering, 13(2), 129-157. Doi: https://doi.org/10.1007/s11518-006-0158-y Saaty, T. L., & Peniwati, K. (2008). Group decision making: Drawing out and reconciling differences. Pittsburgh, PA: RWS Publications. Solutions, T. (2022). Decision mentor. Retrieved from https://www.decisionmentor.app/ Strand, R., Spaapen, J., Bauer, M. W., Hogan, E., Revuelta, G., Stagl, S., . . . Guimarães Pereira, A. (2015). Indicators for promoting and monitoring responsible research and innovation - Report from the expert group on policy indicators for responsible research and innovation. SuperDecisions. (2020). Super Decisions. Retrieved from https://www.superdecisions.com/ https://doi.org/10.13033/ijahp.v10i3.590 https://www.decisionmentor.app/ https://www.superdecisions.com/