IJAHP Article: Moradi/Performance evaluation of University faculty by combining BSC, AHP and TOPSIS: From the students’ perspective International Journal of the Analytic Hierarchy Process 1 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 PERFORMANCE EVALUATION OF UNIVERSITY FACULTY BY COMBINING BSC, AHP AND TOPSIS: FROM THE STUDENTS’ PERSPECTIVE Nima Moradi 1 Faculty of Engineering and Natural Sciences, Sabanci University, Istanbul, Turkey nimamoradi@sabanciuniv.edu ABSTRACT Today, the university plays an important role in establishing a relationship between industry and academia by training a specialized workforce. Due to the important role of the university in the development of a country, evaluating the performance of the faculty or research centers of universities is one of the vital issues in the quality management of universities. In this paper, a performance evaluation method is presented for three faculty of a university located in Istanbul, Turkey (the name of university is kept in confidential due to the request of the university’s expert). The proposed method is based on the combination of Balanced Scorecard (BSC), Analytic Hierarchy Process (AHP), and Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS). First, BSC and AHP are integrated, and then the strategies and measures are introduced for each perspective of BSC. Then, by implementing the TOPSIS method, a comprehensive performance evaluation approach was proposed and discussed with the university management. The proposed methodology was validated by a real case study based on the judgments of students and verified by sensitivity analysis. Finally, several managerial insights, conclusions, and suggestions for future studies are presented. Keywords: university performance evaluation; BSC; AHP; TOPSIS 1. Introduction Currently, the role of the university or higher education institution as one of the main pillars of development and progress in any country is well known. The university plays an important role in establishing a relationship between industry and academia by training a specialized workforce for the industry. Also, in addition to training specialists, the university has an important role in promoting community culture. Due to the Acknowledgements The authors of this project would like to acknowledge the anonymous students of the Engineering department (4 students), Social Sciences department (3 students) and Business school (3 students), which presented their judgments for our discretion. mailto:nimamoradi@sabanciuniv.edu IJAHP Article: Moradi/Performance evaluation of University faculty by combining BSC, AHP and TOPSIS: From the students’ perspective International Journal of the Analytic Hierarchy Process 2 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 important role of the university in the development of a country, evaluating and analyzing the performance of the faculty or research centers of the university will be one of the vital issues in their quality management. Some of the current questions that arise are as follows: what is an effective way to evaluate the performance of the faculty of a university?, what measures and factors should be considered to evaluate the performance of the faculty of a university? and what method can comprehensively and appropriately evaluate the performance of the university when considering the domestics factors? These are the questions this paper will address. Moreover, the main motivation of this paper is to study the strengths and weaknesses of each faculty at a university located at Istanbul, Turkey (the name of university is kept in confidential due to a request from the university’s expert), which has not been done before by multi-criteria decision-making methods such as the Analytic Hierarchy Process (AHP) and Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS). Also, the results are all obtained from the students’ perspective which shows their opinions about the quality of each faculty; thus, this can be a useful method for a university to check its performance. Therefore, the main purpose of this paper is to introduce a performance evaluation method for faculty of a university located in Istanbul, Turkey based on the combination of Balanced Scorecard (BSC), AHP, and TOPSIS. First, BSC is integrated with AHP. Then, by modifying the TOPSIS method, the AHP is integrated with TOPSIS, which makes the proposed approach a comprehensive performance evaluation tool. Finally, the methodology is validated by implementing it on a real case study. As a result, not only is the newly introduced method able to theoretically evaluate university performance, but it is also practical. To summarize, the contributions of the present work can be presented as follows:  Proposing a performance evaluation method for three main faculty of the University including Faculty of Engineering and Natural Sciences (FENS), Faculty of Art and Social Sciences (FASS), and the Business school based on the BSC, AHP and TOPSIS.  Defining the strategies and measures using the four perspectives of BSC for three main faculty of the University, although the proposed strategies and measures can be used for a performance evaluation of any other university.  Calculating the weights of the measures and strategies and ranks of each faculty based on the judgments of students from the FENS, FASS and business school. In the next section, the literature is reviewed briefly. In the third section, BSC, AHP and TOPSIS are explained, and the methodology of this work is presented in detail. In the fourth section, the results of the methodology are given. In the fifth section, managerial insights are provided with analysis and discussion of the results. Finally, in the sixth section, the conclusion and suggestions for future studies are presented. IJAHP Article: Moradi/Performance evaluation of University faculty by combining BSC, AHP and TOPSIS: From the students’ perspective International Journal of the Analytic Hierarchy Process 3 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 2. Literature review Historically, many methods for performance evaluation have been introduced, from traditional methods such as expert opinions and meetings, which are generally qualitative methods, to new methods based on real data and statistics and various quantitative measurements. In addition to choosing a performance evaluation method, it is more important to create a hybrid performance evaluation system based on experts’ opinions. In this way, if university administrators only pay attention to one method of performance evaluation, they will no longer be able to identify the strengths and weaknesses of the faculties of university. 2.1 Balanced scorecard (BSC) Before introducing BSC, only financial aspects were important to managers who practically ignored other aspects of the organization or enterprise. Because these financial-specific methods were not extensive, they were not effective in evaluating the performance of the organization from an overall perspective. In the 1980s, a novel four- dimensional model for performance management known as BSC was introduced by Kaplan and Norton (2001). BSC is a performance management tool that helps organizations practically reach their goals, vision, and strategies (Kaplan & Norton, 2001). The BSC approach has four main perspectives as follows (Kaplan & Norton, 1996):  Growth and learning: In this perspective, an organization tries to find the strategies that lead to improvement and long-term growth. In addition, an enterprise or company tries to work toward value creation and innovation and doing innovative activities to create new services or ideas.  Internal business processes: In this perspective, an organization tries to define the critical internal processes which are important for an organization’s success. By implementing the right internal processes, an organization can find ways to satisfy the customers’ expectations and financial objectives.  Customer: In this perspective, the organization tries to satisfy the customer’s expectations. Moreover, identifying new customers and customer retention are considered as critical factors.  Financial: In this perspective, the organization tries to reach the profitability and financial objectives. In other words, the organization emphasizes the financial performance such as profit, income, cost, etc. 2.2 Analytic Hierarchy Process (AHP) & the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) The AHP method was first introduced by Thomas L. Saaty in the 1980s. This method is one of the most-used multi-criteria decision-making (MCDM) methods. The AHP is based on the hierarchical structure and considers both quantitative and qualitative criteria in the model. Moreover, it finds the consistency and inconsistency of the comparison between alternatives. As an important point, inputs of the AHP are pairwise comparison matrices, which are filled by the judgments of experts. For interested readers, steps of the AHP method are explained in Saaty (2008). The AHP consists of the following steps: IJAHP Article: Moradi/Performance evaluation of University faculty by combining BSC, AHP and TOPSIS: From the students’ perspective International Journal of the Analytic Hierarchy Process 4 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915  Hierarchical tree formation: Including objectives, criteria, sub-criteria and alternatives hierarchically, so that the relationship of each level with its upper and lower levels is known.  Formation of pairwise comparison matrices: For each criterion or sub-criterion, the alternatives in the problem are compared in a matrix.  Calculating the weights for criteria and sub-criteria.  Calculating the final weight of each alternative by adding the multiplication of each weight related to each of the sub-criteria. It is noteworthy to say, in this paper, among MCDM methods, multi-attribute decision making (MADM) methods are proper for our case study since comparisons between different alternatives are made according to the different criteria. Also, among MADM methods, TOPSIS is chosen since "compensatory methods such as TOPSIS allow trade- offs between criteria, where a poor result in one criterion can be negated by a good result in another criterion" (Greene, et al., 2011). TOPSIS was first introduced by Ching-Lai Hwang and Yoon in 1981 and like AHP, it is one of the most popular MCDM methods. In TOPSIS, “the best alternative should have the least distance from the positive ideal solution and the greatest distance from the negative ideal solution” (Hwang, Lai, & Liu, 1993). For interested readers, steps of TOPSIS are explained briefly in Hwang & Yoon (1981). 2.3 Combination of BSC, AHP and TOPSIS The BSC is known as one of the most extensive strategic management tools. The AHP and TOPSIS both have their own strengths and weaknesses. For example, TOPSIS did not consider any weights or preferences between the criteria, so the AHP could support TOPSIS for finding the weights of the criteria in comparison to each other with a quantitative analysis. A combination of BSC, AHP and TOPSIS can lead to finding a comprehensive method which has the strengths of all three tools in one place. A BSC- AHP-TOPSIS approach has been studied by several researchers in recent years (Table 1). According to Table 1, there are only two papers that used BSC, AHP and TOPSIS to evaluate the performance of faculty of engineering education and determine a strategic plan for higher education; as a result, there is no similar work in the literature to the present work. IJAHP Article: Moradi/Performance evaluation of University faculty by combining BSC, AHP and TOPSIS: From the students’ perspective International Journal of the Analytic Hierarchy Process 5 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 Table 1 Recent articles on BSC-AHP-TOPSIS approach Ref. Year Case Methodology BSC AHP TOPSIS (Ertuğrul & Karakaşoğlu, 2008) 2008 Facility location selection    (Lee, Chen, & Chang, 2008) 2008 Evaluating performance of IT department in the manufacturing industry in Taiwan    (Seçme, Bayrakdaroğlu, & Kahraman, 2009) 2009 Performance evaluation in Turkish banking sector    (Ertuğrul & Karakaşoğlu, 2009) 2009 Performance evaluation of Turkish cement firms    (Gumus, 2009) 2009 Evaluation of hazardous waste transportation firms    (Azar, Olfat, Khosravani, & Jalali, 2011) 2011 Supplier selection strategy    (Manian, Fathi, Zarchi, & Omidian, 2011) 2011 Performance Evaluating of IT department    (Bentes, Carneiro, da Silva, & Kimura, 2012) 2012 Multidimensional assessment of organizational performance    (Shojaee & Fallah, 2012) 2012 Strategic planning    (Bhutia & Phipon, 2012) 2012 Supplier selection problem    (Önder, Taş, & Hepsen, 2013) 2013 Performance evaluation of Turkish banks    (Sundharam, Sharma, & Stephan Thangaiah, 2013) 2013 Sustainable growth of manufacturing industries    (Fallah Shams Lialestanei, Raji, & Khajeh Poor, 2013) 2013 Evaluate the performance of organization branches in Tehran    (Vinodh, Prasanna, & Prakash, 2014) 2014 Selecting the best plastic recycling method    (Aly, Attia, & Mohammed, 2014) 2014 Prioritizing faculty of engineering education Performance    (Graham, Freeman, & Chen, 2015) 2015 Green supplier selection    (Sehhat, Taheri, & Sadeh, 2015) 2015 Ranking of insurance companies in Iran    (Yudatama & Sarno, 2016) 2016 Priority determination for higher education strategic    IJAHP Article: Moradi/Performance evaluation of University faculty by combining BSC, AHP and TOPSIS: From the students’ perspective International Journal of the Analytic Hierarchy Process 6 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 Ref. Year Case Methodology BSC AHP TOPSIS planning (Pramanik, Haldar, Mondal, Naskar, & Ray, 2017) 2017 Resilient supplier selection    (Hájek, Stříteská, & Prokop, 2018) 2018 Innovation performance evaluation    (Moradi, Malekmohammad, & Jamalzadeh, 2018) 2018 Performance evaluation of digital game industry    (Yılmaz & Nuri İne, 2018) 2018 Assessment of sustainability performances of banks    (Chou, Yen, Dang, & Sun, 2019) 2019 Assessing the human resource in science and technology for Asian countries    (Chatterjee & Stević, 2019) 2019 Supplier evaluation in manufacturing environment    (Guru & Mahalik, 2019) 2019 Performance measurement of Indian public sector banks    (Ban, Ban, Bogdan, Popa, & Tuse, 2020) 2020 Performance evaluation model of Romanian manufacturing listed companies    (Yildiz, Ayyildiz, Taskin Gumus, & Ozkan, 2020) 2020 ATM site selection problem    (Yucesan & Gul, 2020) 2020 Hospital service quality evaluation    (Moradi & Moradi, 2021) 2020 Performance evaluation of a project-based growth and entrepreneurship organization in Iran    Present Work 2021 Performance evaluation of faculties at the University    2.4 University performance evaluation According to the literature, there are several works which have studied the performance evaluation of a university or a higher education institution. Chen et al. (2006) used BSC as a performance evaluation tool for the Taiwanese higher education sector. By implementing the proposed method on a real case study, they constructed five major strategic themes such as an adequate financial structure, an accord with customer expectations, an excellent learning environment, organizational learning and management, and high-quality staff. Farid et al. (2008) used BSC as a strategic management and powerful measurement tool in universities and higher education institutes. Finally, the performance measures have been introduced for the real case study to validate the proposed BSC. Taylor and Baines (2012) implemented BSC in UK IJAHP Article: Moradi/Performance evaluation of University faculty by combining BSC, AHP and TOPSIS: From the students’ perspective International Journal of the Analytic Hierarchy Process 7 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 universities to evaluate their performance. The real case study included four UK universities and interviews with senior managers. Also, the results provided insight into the application of new management tools within higher education in UK universities. Al-Zwyalif (2012) used BSC to evaluate the performance of Jordanian private universities. To reach the goals of the study, data were collected from the Jordanian private universities through a questionnaire for faculty deans, deputy deans, heads of scientific departments, financial managers, and administrative managers. The results showed that “the Jordanian private universities are aware of the importance of implementing the BSC in performance evaluation”. Cugini and Michelon (2007) proposed and developed a performance evaluation approach which is suitable for the specific features of an academic department. Their case study is the University of Padua, Italy, where data were collected. Wu and Li (2009) extracted the performance measure indicators (PMIs) for higher education based on BSC. In addition to BSC, they used DRF (data reduction factor) and DEA (data envelopment analysis) tools to complete the evaluation performance process. In their case study, 15 Science and Technology universities of the MOE (Ministry of Education) were selected. Özdemir and Tüysüz (2017) proposed a fuzzy decision making based BSC model for performance evaluation of universities. Their decision-making approach includes a fuzzy Decision-Making Trial and Evaluation Laboratory (DEMATEL) and fuzzy Analytic Network Process (ANP) methods. The fuzzy DEMATEL method is used for showing the relationship among the perspectives and the strategies of the BSC. Finally, by applying fuzzy ANP, the weights of perspectives and strategies are obtained. Ramasamy et al. (2016) proposed a performance evaluation tool based on BSC. They also used the AHP to prioritize the performance measures of higher-level academic institutions over BSC perspectives. Using the AHP, the weights of evaluation indexes were obtained and a real case study, a university in South India, was studied to validate the proposed method. Yousif and Shaout (2018) presented a fuzzy logic computational model based on a survey to measure and classify the performance of Sudanese universities and academic staff. Also, they used AHP and TOPSIS to determine the criteria weights and overall evaluation of Sudanese universities and academic staff. In recent works, Mu and Nicola (2019) developed a model for rank and tenure (R&T) decisions using AHP. They used a case method approach for the development of the model and the demonstration of its use. They concluded that the proposed model rendered objectivity, transparency, and customization for R&T committee decisions in higher-education institutions (Mu & Nicola, 2019). Moreover, there are several papers which have studied performance evaluation at the university or in higher education by presenting various methodologies such as Big Data Analytics (Job, 2018), DEA (Majidi, Fallah Lajimi, & Safaei Ghadikolaei, 2021; Navas et al., 2020; Soummakie & Wegener; Villegas, Castañeda, & Castañeda-Gómez, 2020), BSC (Anuforo, Ayoup, Mustapha, & Abubakar, 2019; Doh, 2015; Gamal & Soemantri, 2017; Ilyasin, 2017; Nazari-Shirkouhi et al., 2020; Peris-Ortiz, García-Hurtado, & Devece, 2019; Ruggiero, 2004; H.-Y. Wu, Lin, & Chang, 2011; Zolfani & Ghadikolaei, 2013), and review of BSC (Al-Hosaini & Sofian, 2015). Mu and Pereya-Rojas (2017) is a nice work on AHP and its applications. As a result, the proposed methodology is a unique IJAHP Article: Moradi/Performance evaluation of University faculty by combining BSC, AHP and TOPSIS: From the students’ perspective International Journal of the Analytic Hierarchy Process 8 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 topic in the literature and the aim is to fill this research gap which has both theoretical and practical implications. 3. Research methodology The steps used for research methodology in this paper are as follows (Mu, Cooper, & Peasley, 2020):  Introducing strategies for each perspective of the BSC based on the University's mission and goals (based on literature, judgments of experts and the website).  Introducing measures for each strategy of the BSC perspectives (based on the literature and experts’ opinions).  Calculating the weights of the measures and perspectives of BSC using the AHP (based on the scores given by students on a questionnaire).  Calculating the weights and ranks of the faculty including the FENS, FASS, and Business school using TOPSIS (based on the scores given by students on a questionnaire). 3.1 Data collection tools As mentioned in the first step of methodology, the contents of the literature and available references have been consulted. In addition to these resources, the questionnaire has been used to gather the judgments of students of the various faculty of the University (see Appendix). These faculty are located at the University and the number of students is given in the acknowledgments. Also, all of the calculations related to the AHP and TOPSIS were done in Excel Microsoft Office. 3.2 Strategies for the perspectives of BSC First, the strategies for each perspective of BSC were extracted using the papers in the literature for BSC perspectives (Beard, 2009; Chen et al., 2006; Kaplan & Norton, 2015), strategies (Alani, Khan, & Manuel, 2018; Aslam, 2011; Cugini & Michelon, 2007; Farid et al., 2008) and Turkish higher education (Mizikaci, 2003; Özdemir & Tüysüz, 2017; Soummakie & Wegener) and confirmed by experts at the University who have more than 10 years teaching and research experience and are presented in Table 2. IJAHP Article: Moradi/Performance evaluation of University faculty by combining BSC, AHP and TOPSIS: From the students’ perspective International Journal of the Analytic Hierarchy Process 9 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 Table 2 Strategies for each perspective of BSC Perspective Strategy 1. Growth and learning 1.1 Student literacy development 1.2 Faculty development 1.3 Increase the motivation of students 1.4 Improve work environment-faculty and staff 1.5 Development of organizational culture and civilization 1.6 Increase the competence and ability of staff 1.7 Improve research quality 1.8 Promote online learning applications 2. Internal business processes 2.1 Transfer of learning 2.2 Curriculum excellence 2.3 Information technology development 2.4 Establish high quality service process 2.5 Complete teaching facility 2.6 Provide excellent teaching quality 2.7 Establish coordination among all parts of the university 3. Customer 3.1 Customer satisfaction-students, faculty and staff 3.2 Community satisfaction 3.3 Consistent with customer's expectations 4. Financial 4.1 Sufficient generation of funds 4.2 Increase asset usage rate 4.3 Reduce redundant costs 4.4 Investment in Research and Development (R&D) 4.5 Budget management 3.3 Measures for the strategies and perspectives of BSC In the next step, the measures for each strategy are extracted using the papers and other useful resources (Al-Zwyalif, 2012; Beard, 2009; Chen et al., 2006; Cugini & Michelon, 2007; Doh, 2015; Navas et al., 2020; Nazari-Shirkouhi et al., 2020; Özdemir & Tüysüz, 2017; Ruggiero, 2004; Taylor & Baines, 2012; Y. Wu & Li, 2009; Zolfani & Ghadikolaei, 2013), in which the proposed measures were verified. In Table 3, the measures for the strategies of BSC perspectives are provided. There are 23 extensive strategies and 56 measures for BSC perspectives which were used to evaluate the performance of each faculty comprehensively. IJAHP Article: Moradi/Performance evaluation of University faculty by combining BSC, AHP and TOPSIS: From the students’ perspective International Journal of the Analytic Hierarchy Process 10 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 Table 3 Measures for the strategies of BSC perspectives Strategy Measures 1.1 Student literacy development 1.1.1 Number of licenses owned by students 1.1.2 Number of seminars held for students 1.1.3 Number of graduate students with GPA higher than 3.5 1.2 Faculty development 1.2.1 Number of licenses owned by faculty members 1.2.2 Number of conferences held for faculty members 1.2.3 Ratio of the citations for each faculty member 1.3 Increase the motivation of students 1.3.1 Ratio of the graduated students to total number of students 1.3.2 Number of students who continue study for their PhD 1.3.3 Number of students participating in conferences and seminars 1.4 Improve work environment-faculty and staff 1.4.1 Modernization of equipment/facilities 1.4.2 Upgrading of teaching methodology 1.5 Development of organizational culture and civilization 1.5.1 Ratio of the number of hours of seminars for strengthening communication skills to total number of staff 1.6 Increase the competence and ability of staff 1.6.1 Organization active rate 1.6.2 Internal promotion rate 1.7 Improve research quality 1.7.1 Number of papers published 1.7.2 National science conference rate 1.7.3 Faculty obtaining qualification and patent rate 1.7.4 Faculty writing teaching materials or books ratio 1.7.5 Number of TÜBİTAK projects 1.8 Promote online learning applications 1.8.1 Number of distant teaching applications 1.8.2 Familiarity of staff using computers 2.1 Transfer of learning 2.1.1 Number of reports about learning experiences during each year 2.2 Curriculum excellence 2.2.1 Number of non-conflict courses with each other 2.2.2 Number of new courses presented during each semester 2.2.3 Adequate budget on course development 2.2.4 Automated process on updating courses 2.3 Information technology development 2.3.1 Ratio of administration computerized Training 2.3.2 Customer satisfaction level of administration computerized 2.3.3 Teaching facility use rate 2.3.4 Ratio of administration computerized 2.4 Establish high quality 2.4.1 Student/staff ratio IJAHP Article: Moradi/Performance evaluation of University faculty by combining BSC, AHP and TOPSIS: From the students’ perspective International Journal of the Analytic Hierarchy Process 11 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 Strategy Measures service process 2.4.2 Full-time staff rate 2.5 Complete teaching facilities 2.5.1 Teaching facility renewal rate 2.6 Provide excellent quality education 2.6.1 Library availability and facility ratio 2.6.2 Number of areas available to everyone for use 2.6.3 International scholar academic exchange rate 2.7 Establish coordination among all parts of the university 2.7.1 Library availability and facility ratio 3.1 Customer satisfaction 3.1.1 Staff satisfaction level 3.1.2 Student satisfaction level 3.1.3 Faculty member satisfaction level 3.2 Community satisfaction 3.2.1 Satisfaction level of external partners 3.3 Consistent with customers' expectations 3.3.1 Number of modifications made due to customers' expectations 3.3.2 Numbers of customer complaints 3.3.3 Numbers participating in public charity activities 4.1 Sufficient funds generation 4.1.1 Tuition income 4.1.2 Education promotion rewards 4.1.3 Amount of cooperation between education and business 4.1.4 Business donation 4.1.5 Ministerial grants and research grants 4.1.6 Allowance amount 4.2 Increase asset usage rate 4.2.1 Assets and facilities recycle rate 4.2.2 Assets and facilities return rate 4.2.3 Library resources and facilities usage rate 4.3 Reduce redundant costs 4.3.1 Human resources expense rate 4.3.2 Elimination rate of unsuitable staff 4.4 Investment in Research and Development (R&D) 4.4.1 R&D expense rate 4.5 Budget management 4.5.1 Gross profit 3.4 Combination of BSC and AHP To combine the BSC with the AHP, the measures and BSC perspectives are considered as the alternatives and criteria in the AHP, respectively. Therefore, the measures in the previous section are compared with each other in accordance with the related BSC perspective. For example, there are 21 measures in the growth & learning perspective, which are given in a pairwise comparison matrix and assigned a score by students according to their importance in the growth & learning perspective. This process is repeated for the other measures of the other three perspectives. However, due to the huge pairwise comparison matrices, the strategies of each perspective are compared with each other in the pairwise comparison matrices and then the weight of each strategy is divided evenly among its measures to obtain the weight of each measure. IJAHP Article: Moradi/Performance evaluation of University faculty by combining BSC, AHP and TOPSIS: From the students’ perspective International Journal of the Analytic Hierarchy Process 12 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 3.5 Calculation of the weight of the three main faculty with TOPSIS In this section, we implement TOPSIS in the proposed model. As described before, the AHP is based on a pairwise comparison matrix, while TOPSIS is based on the decision matrix (comparison between alternatives according to the different criteria). With TOPSIS, in the decision matrix, we use three groups of faculty (FENS, FASS and Business school) and measures as the alternatives and criteria, respectively (Table 4). Thus, the weights and ranks of each faculty can be calculated based on the judgments of the students according to the following algorithm: - Start: consider matrix 𝑅𝑛,𝑚 as the decision matrix in TOPSIS (input). - For all 𝑖, 𝑗 do: { 𝑛𝑖𝑗 = 𝑟𝑖𝑗 ∑ 𝑟𝑖𝑗 𝑛 𝑖=1 ⁄ (Normalization step) } - For all 𝑖, 𝑗 do: { 𝑉𝑛,𝑚 = 𝑊𝑛,𝑛 × 𝑁𝑛,𝑚 (𝑊𝑛,𝑛 is the diagonal matrix with the weights of the measures in its main diagonal and 𝑁𝑛,𝑚 is the normalized matrix) } - For all 𝑖 do: { 𝑑𝑖 + = √∑ (𝑣𝑖𝑗 − 𝑣𝑗 +) 2𝑚 𝑗=1 and 𝑑𝑖 − = √∑ (𝑣𝑖𝑗 − 𝑣𝑗 −) 2𝑚 𝑗=1 (𝑣𝑗 +, positive ideal solution (PIS), and 𝑣𝑗 −, negative ideal solution (NIS), are the maximum value of the j-th column and the minimum value of the j-th column of the matrix 𝑉𝑛,𝑚, respectively) } - For all 𝑖 do: { 𝐶𝐿𝑖 ∗ = 𝑑𝑖 − (𝑑𝑖 − + 𝑑𝑖 +) ⁄ (𝐶𝐿∗is the closeness coefficient of each alternative) } - Rank the alternatives (faculty) in descending order according to their 𝐶𝐿∗. - End Table 4 Decision matrix in the proposed AHP-TOPSIS model Decision matrix in TOPSIS Criteria Measure 1.1.1 … Measure 4.5.1 Alternatives FENS 𝑟1,1 … 𝑟1,56 FASS 𝑟2,1 … 𝑟2,56 Business school 𝑟3,1 … 𝑟3,56 IJAHP Article: Moradi/Performance evaluation of University faculty by combining BSC, AHP and TOPSIS: From the students’ perspective International Journal of the Analytic Hierarchy Process 13 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 4. Results 4.1 Case study To verify our methodology, a questionnaire survey was used to obtain the weights of the measures and faculty. The questionnaires were distributed to 10 students; 4 students from FENS, 3 students from FASS and 3 students from the Business school, located at the University to aggregate their judgments. The aggregation took place through a designed Google Form and placing the form on social networks such as Telegram and WhatsApp groups. The profile of the individual student is anonymous since we promised that these judgments would remain confidential, but the statistics of all participants are given in Table 5. We continued the questionnaire survey until the inconsistency of the AHP matrices became acceptable; in other words, since the pairwise comparison matrices of the AHP were inconsistent due to the inconsistency rate in the first round of gathering the scores given by the students, we continued gathering the judgments of the students until the inconsistency rate of the AHP matrices became an acceptable inconsistency rate (less than 0.1) (Saaty, 2008). In addition, these faculty have not been evaluated by the students before, so these judgments will be helpful for our model verification. Table 5 Statistics of all participants in the aggregation process Participant Age Education level Faculty Department 1 30 PhD FENS Industrial Engineering 2 29 PhD FENS Industrial Engineering 3 33 M.Sc. FENS Industrial Engineering 4 34 PhD FENS Industrial Engineering 5 30 PhD FASS Economics 6 27 M.Sc. FASS Turkish studies 7 26 M.Sc. FASS Turkish studies 8 31 PhD Business General business 9 28 M.Sc. Business MBA 10 27 M.Sc. Business MBA 4.2 Pairwise comparison matrices (AHP input) In this section, as the inputs of the AHP, the preferences of each of the four perspectives of the BSC and strategies of each perspective were determined by the questionnaire survey; here, the average of the scores was rounded to the nearest integer number (Tables 6-10). In these pairwise comparison matrices, the inconsistency of each is less than 0.1, so these matrices can be used as inputs for calculating the weights of the measures. Also, Tables 7-10 show the weight of each strategy. IJAHP Article: Moradi/Performance evaluation of University faculty by combining BSC, AHP and TOPSIS: From the students’ perspective International Journal of the Analytic Hierarchy Process 14 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 Table 6 Pairwise comparison matrix for the perspectives of the BSC BSC 1 2 3 4 Weights 1 1.00 2.00 3.00 4.00 0.45 2 0.70 1.00 2.00 3.00 0.29 3 0.40 0.30 1.00 2.00 0.15 4 0.10 0.60 0.40 1.00 0.09 Inconsistency rate 0.02 Table 7 Pairwise comparison matrix for the strategies of the growth & learning perspective Growth & learning 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 Weights 1.1 1.00 1.00 0.50 0.70 0.30 1.00 0.10 0.90 0.06 1.2 0.90 1.00 2.00 1.00 2.00 2.00 1.00 3.00 0.16 1.3 2.00 0.50 1.00 2.00 2.00 1.00 0.30 0.90 0.10 1.4 3.00 2.00 0.50 1.00 3.00 1.00 0.20 1.00 0.12 1.5 2.00 0.30 0.40 0.50 1.00 0.50 0.10 0.40 0.05 1.6 1.00 0.50 1.00 1.00 3.00 1.00 0.50 1.00 0.10 1.7 7.00 1.00 4.00 5.00 5.00 2.00 1.00 3.00 0.29 1.8 0.50 1.00 2.00 0.50 1.00 1.00 0.30 1.00 0.09 Inconsistency rate 0.09 Table 8 Pairwise comparison matrix for the strategies of the business internal processes Business internal processes 2.1 2.2 2.3 2.4 2.5 2.6 2.7 Weights 2.1 1.00 2.00 2.00 3.00 2.00 1.00 3.00 0.23 2.2 1.00 1.00 2.00 2.00 1.00 2.00 3.00 0.20 2.3 0.50 1.00 1.00 0.70 0.40 0.50 1.00 0.09 2.4 0.30 1.00 1.00 1.00 2.00 1.00 2.00 0.13 2.5 0.50 1.00 2.00 1.00 1.00 0.50 2.00 0.12 2.6 0.90 0.40 0.30 0.50 3.00 1.00 3.00 0.14 2.7 0.50 0.40 0.50 0.60 1.00 0.30 1.00 0.07 Inconsistency rate 0.09 Table 9 Pairwise comparison matrix for the strategies of the customer perspective Customer 3.1 3.2 3.3 Weights 3.1 1.00 2.00 1.00 0.42 3.2 0.30 1.00 2.00 0.30 3.3 1.00 0.40 1.00 0.26 Inconsistency rate 0.08 IJAHP Article: Moradi/Performance evaluation of University faculty by combining BSC, AHP and TOPSIS: From the students’ perspective International Journal of the Analytic Hierarchy Process 15 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 Table 10 Pairwise comparison matrix for the strategies of the financial perspective Financial 4.1 4.2 4.3 4.4 4.5 Weights 4.1 1.00 2.00 2.00 3.00 1.00 0.31 4.2 0.40 1.00 1.00 2.00 1.00 0.18 4.3 0.50 1.00 1.00 2.00 2.00 0.22 4.4 0.50 0.30 0.20 1.00 1.00 0.10 4.5 1.00 1.00 0.50 1.00 1.00 0.17 Inconsistency rate 0.01 4.3 Weights of the measures (AHP output/TOPSIS input) After the AHP calculations, the weights of the measures and perspectives of the BSC are presented in Table 11. Here, the weight of each measure is calculated by dividing the weight of each strategy by the number of its measures. These weights are now the output of the AHP, which can be considered as the inputs of the TOPSIS in the next step. Table 11 Weights of the measures and perspectives of the BSC (M: Measure, W: Weight) M 1.1.1 1.1.2 1.1.3 1.2.1 1.2.2 1.2.3 1.3.1 1.3.2 1.3.3 1.4.1 1.4.2 1.5.1 W 0.010 0.010 0.010 0.025 0.025 0.025 0.015 0.015 0.015 0.028 0.028 0.023 M 1.6.1 1.6.2 1.7.1 1.7.2 1.7.3 1.7.4 1.7.5 1.8.1 1.8.2 2.1.1 2.2.1 2.2.2 W 0.023 0.023 0.027 0.027 0.027 0.027 0.027 0.020 0.020 0.068 0.015 0.015 M 2.2.3 2.2.4 2.3.1 2.3.2 2.3.3 2.4.1 2.4.2 2.5.1 2.6.1 2.6.2 2.6.3 2.7.1 W 0.015 0.015 0.006 0.006 0.006 0.006 0.019 0.019 0.038 0.020 0.020 0.021 M 3.1.1 3.1.2 3.1.3 3.2.1 3.3.1 3.3.2 3.3.3 4.1.1 4.1.2 4.1.3 4.1.4 4.1.5 W 0.021 0.021 0.021 0.047 0.013 0.013 0.013 0.004 0.004 0.004 0.004 0.004 M 4.1.6 4.2.1 4.2.2 4.2.3 4.3.1 4.3.2 4.4.1 4.5.1 1 2 3 4 W 0.004 0.005 0.005 0.005 0.009 0.009 0.009 0.015 0.459 0.296 0.153 0.090 4.4 Weights of the faculty (TOPSIS output) In this section, the weight of each faculty is calculated which gives their rank. The initial scores given by students for the decision matrix of TOPSIS are not given in this paper due to its large dimension. Hence, by the given scores and TOPSIS calculations, the distance of each project phase from PIS and NIS, the closeness coefficient (weight), and ranking of each faculty (FENS, FASS, Business school) are presented in Table 12. IJAHP Article: Moradi/Performance evaluation of University faculty by combining BSC, AHP and TOPSIS: From the students’ perspective International Journal of the Analytic Hierarchy Process 16 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 Table 12 Distance from PIS and NIS, the closeness coefficient (weight) and rank of each faculty (TOPSIS output) Faculty FENS FASS Business school 𝑑+ 0.007135 0.009788 0.006501 𝑑− 0.007049 0.00498 0.008966 𝐶𝐿∗ 0.496938 0.337234 0.579686 Rank 2 nd 3 rd 1 st 5. Discussion of results This section includes two sub-sections. First, the sensitivity analysis with 9 scenarios was performed to verify the robustness of TOPSIS method. After the sensitivity analysis, we provided the managerial insights according to the obtained results where precise analysis was done over the computational results. 5.1 Sensitivity analysis 5.1.1 Scenario 1 Here, we examined the following scenario: What if we do not use the AHP to weigh the measures and instead use the same weights for each measure (or apply solo TOPSIS)? This scenario is equal to removing the AHP from the methodology and just applying TOPSIS with the same weights for the measures. The results of this scenario are given in Table 13. As seen in Table 13, changing weights, and considering them as the same value did not impact the final ranking although adding weights by AHP gives us better and more precise results. Table 13 Results for scenario 1 Faculty FENS FASS Business school 𝐶𝐿∗ (With AHP) 0.496938 0.337234 0.579686 𝐶𝐿∗ (Without AHP) 0.472355 0.280902 0.641319 Rank (With AHP) 2 nd 3 rd 1 st Rank (Without AHP) 2 nd 3 rd 1 st 5.1.2 Scenario 2 Here, we examined the following scenario: What if we replace the highest weight among the measures with the lowest weight among the measures (measure 2.1.1 with measure 4.1.1)? The results of this scenario are given in Table 14. As seen in Table 14, replacing the highest weight among the measures with the lowest weight among the measures did not impact the final ranking. IJAHP Article: Moradi/Performance evaluation of University faculty by combining BSC, AHP and TOPSIS: From the students’ perspective International Journal of the Analytic Hierarchy Process 17 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 Table 14 Results for scenario 2 Faculty FENS FASS Business school 𝐶𝐿∗ (Old) 0.496938 0.337234 0.579686 𝐶𝐿∗ (New) 0.517615 0.327338 0.592475 Rank (Old) 2 nd 3 rd 1 st Rank (New) 2 nd 3 rd 1 st 5.1.3 Scenario 3 Here, we examined the following scenario: What if we replace the two highest weights among the measures with the two lowest weights among the measures (measures 2.1.1 and 3.2.1 with measures 4.1.1 and 4.1.2, respectively)? The results of this scenario are given in Table 15. As seen by Table 15, replacing the two highest weights among the measures with the two lowest weights among the measures did not impact the final ranking although the weights of the FENS and Business school are very close. Table 15 Results for scenario 3 Faculty FENS FASS Business school 𝐶𝐿∗ (Old) 0.496938 0.337234 0.579686 𝐶𝐿∗ (New) 0.531702 0.374632 0.546032 Rank (Old) 2 nd 3 rd 1 st Rank (New) 2 nd 3 rd 1 st 5.1.4 Scenario 4 Here, we examined the following scenario: What if we use the same weight for the highest and lowest weight among the measures (the same weight is their average weight)? The results of this scenario are given in Table 16. As seen in Table 16, using the same weight for the highest and lowest weight among the measures did not impact the final ranking. Table 16 Results for scenario 4 Faculty FENS FASS Business school 𝐶𝐿∗ (Old) 0.496938 0.337234 0.579686 𝐶𝐿∗ (New) 0.503139 0.334331 0.583451 Rank (Old) 2 nd 3 rd 1 st Rank (New) 2 nd 3 rd 1 st 5.1.5 Scenario 5 Here, we examined the following scenario: What if we use the same weight for the two highest and two lowest weights among the measures (the same weight is their average IJAHP Article: Moradi/Performance evaluation of University faculty by combining BSC, AHP and TOPSIS: From the students’ perspective International Journal of the Analytic Hierarchy Process 18 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 weight)? The results of this scenario are given in Table 17. As seen in Table 17, using the same weight for the two highest and two lowest weights among the measures did not impact the final ranking. Table 17 Results for scenario 5 Faculty FENS FASS Business school 𝐶𝐿∗ (Old) 0.496938 0.337234 0.579686 𝐶𝐿∗ (New) 0.507613 0.359702 0.557367 Rank (Old) 2 nd 3 rd 1 st Rank (New) 2 nd 3 rd 1 st 5.1.6 Scenario 6 Here, we examined the following scenario: What if we replace the three highest weights among the measures with the three lowest weights among the measures (measures 2.1.1, 3.2.1 and 2.5.1 with measures 4.1.1, 4.1.2 and 4.1.3, respectively)? The results of this scenario are given in Table 18, in which replacing the three highest weights among the measures with the three lowest weights among the measures did not impact the final ranking, although there is no change for the weight of the FENS. Table 18 Results for scenario 6 Faculty FENS FASS Business school 𝐶𝐿∗ (Old) 0.496938 0.337234 0.579686 𝐶𝐿∗ (New) 0.497359 0.262501 0.625372 Rank (Old) 2 nd 3 rd 1 st Rank (New) 2 nd 3 rd 1 st 5.1.7 Scenario 7 Here, we examined the following scenario: What if we replace the four highest weights among the measures with the four lowest weights among the measures (measures 2.1.1, 3.2.1, 2.5.1 and 1.4.1 with measures 4.1.1, 4.1.2, 4.1.3 and 4.1.4, respectively)? The results of this scenario are given in Table 19. According to Table 19, replacing the four highest weights among the measures with the four lowest weights among the measures did not impact the final ranking, although there is no significant change for the weight of the FENS. IJAHP Article: Moradi/Performance evaluation of University faculty by combining BSC, AHP and TOPSIS: From the students’ perspective International Journal of the Analytic Hierarchy Process 19 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 Table 19 Results for scenario 7 Faculty FENS FASS Business school 𝐶𝐿∗ (Old) 0.496938 0.337234 0.579686 𝐶𝐿∗ (New) 0.495644 0.263232 0.65448 Rank (Old) 2 nd 3 rd 1 st Rank (New) 2 nd 3 rd 1 st 5.1.8 Scenario 8 Here, we examined the following scenario: What if we replace the five highest weights among the measures with the five lowest weights among the measures (measures 2.1.1, 3.2.1, 2.5.1, 1.4.1 and 1.4.2 with measures 4.1.1, 4.1.2, 4.1.3, 4.1.4 and 4.1.5, respectively)? The results of this scenario are given in Table 20. As seen in Table 20, replacing the five highest weights among the measures with the five lowest weights among the measures did not impact the final ranking. Table 20 Results for scenario 8 Faculty FENS FASS Business school 𝐶𝐿∗ (Old) 0.496938 0.337234 0.579686 𝐶𝐿∗ (New) 0.500713 0.265453 0.651497 Rank (Old) 2 nd 3 rd 1 st Rank (New) 2 nd 3 rd 1 st 5.1.9 Scenario 9 Here, we examined the following scenario: What if we remove FASS from the alternatives to see the competition between the FENS and Business school in the absence of the FASS? The results of this scenario are given in Table 21. As seen in Table 21, after removing the FASS from the alternatives, the Business school is still better than the FENS, so TOPSIS is robust in this scenario. Table 21 Results for scenario 9 Faculty FENS FASS Business school 𝐶𝐿∗ (Old) 0.496938 0.337234 0.579686 𝐶𝐿∗ (New) 0.868203 - 0.909646 Rank (Old) 2 nd 3 rd 1 st Rank (New) 2 nd - 1 st After examining different scenarios, we can see that after sensitivity analyses, the TOPSIS method is robust and changing the weights using the AHP has not had a significant impact on TOPSIS. Therefore, TOPSIS is independent of the AHP, and works IJAHP Article: Moradi/Performance evaluation of University faculty by combining BSC, AHP and TOPSIS: From the students’ perspective International Journal of the Analytic Hierarchy Process 20 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 with the scores given to the decision matrix. In other words, changing the weights of the measures has not had a major impact on the final ranking according to the sensitivity analysis; as a result, TOPSIS is robust. 5.2 Managerial insights and implications According to the results and sensitivity analysis, some managerial insights can be elicited as follows:  Inconsistency rates of all AHP pairwise comparison matrices are lower than 0.1, so AHP is verified (According to Tables 6-10).  Changing weights by AHP and removing the FASS from the alternatives has not had a significant impact on the final ranking by TOPSIS, so the TOPSIS method is robust and verified (According to Tables 13-21).  The Business school has a higher rank in comparison with the FENS and FASS from the students’ perspective; this shows that students at the Business school at the University view the performance of their faculty more satisfactorily (see Table 12).  Among all of the measures, measures 1.4.1 modernization of equipment/facilities, 1.4.2 upgrading teaching methodology, 2.1.1 number of reports about learning experiences during each year, 2.6.1 everyone could use library and facilities ratio, 3.2.1 satisfaction level of external partners have higher weights which shows the high importance of satisfaction level and teaching technology in the students’ opinion (see Table 11).  From the students’ perspective, among BSC perspectives, growth & learning and business internal processes have higher weights in comparison to customer and financial factors; this shows that students emphasize learning and business internal processes and they are more satisfied if these two sections are improved (see Table 6).  In the growth & learning perspective, among its strategies, strategy 1.7 improve research quality has the highest weight in comparison to other strategies of the growth & learning section; this shows that research quality such as number of articles or conference participation is important to students (see Table 7).  In the business internal processes perspective, among its strategies, strategy 2.1 transfer of learning has the highest weight in comparison to other strategies of the business processes section; this shows that an exchange program or sending students to the other universities as an additional activity is important for students of the FENS, FASS and business although there may be some biases according to the opinions of only ten students (see Table 8).  In the customer perspective, among its strategies, strategy 3.1 customer satisfaction has the highest weight in comparison to other strategies of the customer section; this shows that satisfaction level including students and faculty members’ satisfaction is one of the most important factors among the other factors (see Table 9).  In the financial perspective, among its strategies, strategy 4.1 sufficient funds generation has the highest weight in comparison to other strategies; this shows that funds generation is important for students since they want to support their education costs (see Table 10). IJAHP Article: Moradi/Performance evaluation of University faculty by combining BSC, AHP and TOPSIS: From the students’ perspective International Journal of the Analytic Hierarchy Process 21 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 6. Conclusions and suggestions for future studies In this paper, a performance evaluation method is presented for faculty of a university located in Istanbul, Turkey. The proposed method is based on the combination of Balanced Scorecard (BSC), AHP, and TOPSIS. First, we integrated BSC with AHP. Then, by modifying the TOPSIS method, we integrated the AHP with TOPSIS, which makes our approach a comprehensive performance evaluation tool. Finally, we validated our methodology by implementing it for a real case study and based on the judgments of students. Also, this method can use the opinions of students at the university to extract strategies and performance measures and to obtain the weights of each strategy and measure. Finally, we conclude that since the inconsistency rates of all AHP pairwise comparison matrices are lower than 0.1, the AHP is verified. Also, changing weights by AHP and removing the FASS from the alternatives did not significantly impact the final ranking by TOPSIS; therefore, the TOPSIS method is robust and verified. In addition, the Business school has a higher rank in comparison to the FENS and FASS from the students’ perspective; this shows that students at the Business school at the University are more satisfied with the performance of their faculty. For future studies, we suggest that the opinions of faculty members and staff can be added to the model to obtain more precise results. IJAHP Article: Moradi/Performance evaluation of University faculty by combining BSC, AHP and TOPSIS: From the students’ perspective International Journal of the Analytic Hierarchy Process 22 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 REFERENCES Al-Hosaini, F. F., & Sofian, S. (2015). A review of balanced scorecard framework in higher education institution (HEIs). International Review of Management and Marketing, 5(1), 26. Al-Zwyalif, I. M. (2012). The possibility of implementing balanced scorecard in Jordanian private universities. International Business Research, 5(11), 113. Doi: https://doi.org/10.5539/ibr.v5n11p113 Alani, F. S., Khan, M. F. R., & Manuel, D. F. (2018). University performance evaluation and strategic mapping using balanced scorecard (BSC): Case study–Sohar University, Oman. International Journal of Educational Management. Doi: https://doi.org/10.1108/ijem-05-2017-0107 Aly, M., Attia, H., & Mohammed, A. M. (2014). Prioritizing faculty of engineering education performance by using AHP-TOPSIS and Balanced Scorecard Approach. International Journal of Engineering Science and Innovative Technology, 3(1), 11-23. Anuforo, P. U., Ayoup, H., Mustapha, U. A., & Abubakar, A. H. (2019). The Implementation of Balance Scorecard and its impact on performance: Case of Universiti Utara Malaysia. International Journal of Accounting & Finance Review, 4(1), 1-16. Doi: https://doi.org/10.46281/ijafr.v4i1.226 Aslam, H. D. (2011). Performance evaluation of teachers in universities: Contemporary issues and challenges. Journal of Educational and Social Research, 1(2), 11-11. Azar, A., Olfat, L., Khosravani, F., & Jalali, R. (2011). A BSC method for supplier selection strategy using TOPSIS and VIKOR: A case study of part maker industry. Management Science Letters, 1(4), 559-568. Doi: https://doi.org/10.5267/j.msl.2011.05.005 Ban, A. I., Ban, O. I., Bogdan, V., Popa, D. C. S., & Tuse, D. (2020). Performance evaluation model of Romanian manufacturing listed companies by fuzzy AHP and TOPSIS. Technological and Economic Development of Economy, 1-29. Doi: https://doi.org/10.3846/tede.2020.12367 Beard, D. F. (2009). Successful applications of the balanced scorecard in higher education. Journal of Education for Business, 84(5), 275-282. Doi: https://doi.org/10.3200/joeb.84.5.275-282 Bentes, A. V., Carneiro, J., da Silva, J. F., & Kimura, H. (2012). Multidimensional assessment of organizational performance: Integrating BSC and AHP. Journal of Business Research, 65(12), 1790-1799. Doi: https://doi.org/10.1016/j.jbusres.2011.10.039 IJAHP Article: Moradi/Performance evaluation of University faculty by combining BSC, AHP and TOPSIS: From the students’ perspective International Journal of the Analytic Hierarchy Process 23 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 Bhutia, P. W., & Phipon, R. (2012). Application of AHP and TOPSIS method for supplier selection problem. IOSR Journal of Engineering, 2(10), 43-50. Doi: https://doi.org/10.9790/3021-021034350 Chatterjee, P., & Stević, Ž. (2019). A two-phase fuzzy AHP-fuzzy TOPSIS model for supplier evaluation in manufacturing environment. Operational Research in Engineering Sciences: Theory and Applications, 2(1), 72-90. Chen, S. H., Yang, C. C., & Shiau, J. Y. (2006). The application of balanced scorecard in the performance evaluation of higher education. The TQM magazine. Doi: https://doi.org/10.1108/09544780610647892 Chou, Y.-C., Yen, H.-Y., Dang, V. T., & Sun, C.-C. (2019). Assessing the human resource in science and technology for Asian countries: Application of fuzzy AHP and fuzzy TOPSIS. Symmetry, 11(2), 251. Doi: https://doi.org/10.3390/sym11020251 Cugini, A., & Michelon, G. (2007). Performance evaluation in research departments: from the Balanced Scorecard to the Strategy Map. Paper presented at the 4th conference on performance measurement and management control, Measuring and rewarding performance. Nice, France. Doh, B. T. S. (2015). Evaluating the strategic objectives of Cameroonian higher education: An application of the Balanced Scorecard. Doctoral dissertation. Research Unit for the Sociology of Education. Ertuğrul, İ., & Karakaşoğlu, N. (2008). Comparison of fuzzy AHP and fuzzy TOPSIS methods for facility location selection. The International Journal of Advanced Manufacturing Technology, 39(7-8), 783-795. Doi: https://doi.org/10.1007/s00170-007- 1249-8 Ertuğrul, İ., & Karakaşoğlu, N. (2009). Performance evaluation of Turkish cement firms with fuzzy analytic hierarchy process and TOPSIS methods. Expert Systems with Applications, 36(1), 702-715. Doi: https://doi.org/10.1016/j.eswa.2007.10.014 Fallah Shams Lialestanei, M., Raji, R., & Khajeh Poor, K. P. (2013). Performance evaluation by using hybrid method: BSC, TOPSIS and AHP. Industrial Management Journal, 5(1), 81-100. Farid, D., Nejati, M., & Mirfakhredini, H. (2008). Balanced Scorecard application inuniversities and higher education institutes: Implemntation guide in an Iranian context. Annals of the University of Bucarest, the Economic & Administrative Series, 2. Gamal, A., & Soemantri, A. I. (2017). The effect of balanced scorecard on the private college performance (Case study at the University of WR Supratman Surabaya). Archives of Business Research, 5(5). Doi: https://doi.org/10.14738/abr.55.3093 IJAHP Article: Moradi/Performance evaluation of University faculty by combining BSC, AHP and TOPSIS: From the students’ perspective International Journal of the Analytic Hierarchy Process 24 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 Graham, G., Freeman, J., & Chen, T. (2015). Green supplier selection using an AHP- Entropy-TOPSIS framework. Supply Chain Management: An International Journal, 20(3), 327-340. Doi: https://doi.org/10.1108/scm-04-2014-0142 Gumus, A. T. (2009). Evaluation of hazardous waste transportation firms by using a two step fuzzy-AHP and TOPSIS methodology. Expert Systems with Applications, 36(2), 4067-4074. Doi: https://doi.org/10.1016/j.eswa.2008.03.013 Guru, S., & Mahalik, D. (2019). A comparative study on performance measurement of Indian public sector banks using AHP-TOPSIS and AHP-grey relational analysis. OPSEARCH, 56(4), 1213-1239. Doi: https://doi.org/10.1007/s12597-019-00411-1 Hájek, P., Stříteská, M., & Prokop, V. (2018). Integrating balanced scorecard and fuzzy TOPSIS for innovation performance evaluation. PACIS 2018 Proceedings. Hwang, C.-L., Lai, Y.-J., & Liu, T.-Y. (1993). A new approach for multiple objective decision making. Computers & operations research, 20(8), 889-899. Hwang, C.-L., & Yoon, K. (1981). Methods for multiple attribute decision making. In Multiple attribute decision making (pp. 58-191): Springer. Doi: https://doi.org/10.1016/0305-0548(93)90109-v Ilyasin, M. (2017). Balanced Scorecard: A strategy for the quality improvement of Islamic higher education. Dinamika Ilmu, 17(2), 223-236. Doi: https://doi.org/10.21093/di.v17i2.703 Job, M. A. (2018). An efficient way of applying big data analytics in higher education sector for performance evaluation. International Journal of Computer Applications, 180(23), 25-32. Doi: https://doi.org/10.5120/ijca2018916434 Kaplan, R. S., & Norton, D. P. (1996). Linking the balanced scorecard to strategy. California Management Review, 39(1), 53-79. Doi: https://doi.org/10.2307/41165876 Kaplan, R. S., & Norton, D. P. (2001). Transforming the balanced scorecard from performance measurement to strategic management: Part 1. Accounting Horizons, 15(1), 87-104. Doi: https://doi.org/10.2308/acch.2001.15.1.87 Kaplan, R. S., & Norton, D. P. (2015). Balanced Scorecard Success: The Kaplan-Norton Collection (4 Books): Harvard Business Review Press. Lee, A. H., Chen, W.-C., & Chang, C.-J. (2008). A fuzzy AHP and BSC approach for evaluating performance of IT department in the manufacturing industry in Taiwan. Expert Systems with Applications, 34(1), 96-107. Doi: https://doi.org/10.1016/j.eswa.2006.08.022 Majidi, S., Fallah Lajimi, H., & Safaei Ghadikolaei, A. (2021). The application of Data Envelopment Analysis in evaluating the performance of universities and higher education IJAHP Article: Moradi/Performance evaluation of University faculty by combining BSC, AHP and TOPSIS: From the students’ perspective International Journal of the Analytic Hierarchy Process 25 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 institutions: A systematic review of the literature. Journal of Industrial Management Perspective, 11, 53-80. Manian, A., Fathi, M. R., Zarchi, M. K., & Omidian, A. (2011). Performance evaluating of IT department using a modified Fuzzy TOPSIS and BSC methodology (Case study: Tehran Province Gas Company). Journal of Management Research, 3(2), 1. Doi: https://doi.org/10.5296/jmr.v3i2.640 Mizikaci, F. (2003). Quality systems and accredition in higher education: An overview of Turkish higher education. Quality in Higher Education, 9(1), 95-106.Doi: https://doi.org/10.1080/13538320308160 Moradi, N., Malekmohammad, H., & Jamalzadeh, S. (2018). A model for performance evaluation of digital game industry using integrated AHP and BSC. Journal of Applied Research on Industrial Engineering, 5(2), 97-109. Moradi, N., & Moradi, S. (2021). A method for project performance evaluation by combining the project golden triangle, BSC, AHP, and TOPSIS. International Journal of Supply and Operations Management, 8(1), 81-95. Mu, E., Cooper, O., & Peasley, M. (2020). Best practices in analytic network process studies. Expert Systems with Applications, 159, 113536. Doi: https://doi.org/10.1016/j.eswa.2020.113536 Mu, E., & Nicola, C. B. (2019). Managing university rank and tenure decisions using a multi-criteria decision-making approach. International Journal of Business and Systems Research, 13(3), 297-320. Doi: https://doi.org/10.1504/ijbsr.2019.100374 Mu, E., & Pereyra-Rojas, M. (2017). Understanding the analytic hierarchy process. In Practical Decision Making (pp. 7-22): Springer.Doi: https://doi.org/10.1007/978-3-319- 33861-3_2 Navas, L. P., Montes, F., Abolghasem, S., Salas, R. J., Toloo, M., & Zarama, R. (2020). Colombian higher education institutions evaluation. Socio-Economic Planning Sciences, 71, 100801. Doi: https://doi.org/10.1016/j.seps.2020.100801 Nazari-Shirkouhi, S., Mousakhani, S., Tavakoli, M., Dalvand, M. R., Šaparauskas, J., & Antuchevičienė, J. (2020). Importance-performance analysis based balanced scorecard for performance evaluation in higher education institutions: an integrated fuzzy approach. Journal of Business Economics and Management, 21(3), 647-678. Doi: https://doi.org/10.3846/jbem.2020.11940 Önder, E., Taş, N., & Hepsen, A. (2013). Performance evaluation of Turkish banks using analytical hierarchy process and TOPSIS methods. Journal of International Scientific Publication: Economy & Business, 7(Part 1), 470-503. IJAHP Article: Moradi/Performance evaluation of University faculty by combining BSC, AHP and TOPSIS: From the students’ perspective International Journal of the Analytic Hierarchy Process 26 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 Özdemir, A., & Tüysüz, F. (2017). An Integrated Fuzzy DEMATEL and Fuzzy ANP based Balanced Scorecard approach: Application in Turkish higher education Institutions. Journal of Multiple-Valued Logic & Soft Computing, 28. Peris-Ortiz, M., García-Hurtado, D., & Devece, C. (2019). Influence of the balanced scorecard on the science and innovation performance of Latin American universities. Knowledge Management Research & Practice, 17(4), 373-383. Doi: https://doi.org/10.1080/14778238.2019.1569488 Pramanik, D., Haldar, A., Mondal, S. C., Naskar, S. K., & Ray, A. (2017). Resilient supplier selection using AHP-TOPSIS-QFD under a fuzzy environment. International Journal of Management Science and Engineering Management, 12(1), 45-54. Doi: https://doi.org/10.1080/17509653.2015.1101719 Ramasamy, N., Rajesh, R., Pugazhendhi, S., & Ganesh, K. (2016). Development of a hybrid BSC-AHP model for institutions in higher education. International Journal of Enterprise Network Management, 7(1), 13-26. Doi: https://doi.org/10.1504/ijenm.2016.075174 Ruggiero, J. (2004). Performance evaluation in education. In Handbook on data envelopment analysis (pp. 323-346): Springer. Saaty, T. L. (2008). Decision making with the analytic hierarchy process. International Journal of Services Sciences, 1(1), 83-98. Seçme, N. Y., Bayrakdaroğlu, A., & Kahraman, C. (2009). Fuzzy performance evaluation in Turkish banking sector using analytic hierarchy process and TOPSIS. Expert Systems with Applications, 36(9), 11699-11709. Doi: https://doi.org/10.1016/j.eswa.2009.03.013 Sehhat, S., Taheri, M., & Sadeh, D. H. (2015). Ranking of insurance companies in Iran using AHP and TOPSIS techniques. American Journal of Research Communication, 3(1), 51-60. Shojaee, M., & Fallah, M. (2012). A hybrid TOPSIS-BSC method for strategic planning. Management Science Letters, 2(8), 2845-2850. Doi: https://doi.org/10.5267/j.msl.2012.09.029 Soummakie, B., & Wegener, M. (2021). A two-stage research performance assessment of Turkish higher education institutions using Data Envelopment Analysis and Beta Regression. Preprint. Sundharam, V., Sharma, V., & Stephan Thangaiah, I. (2013). An integration of BSC and AHP for sustainable growth of manufacturing industries. International Journal of Business Excellence, 6(1), 77-92. Doi: https://doi.org/10.1504/ijbex.2013.050577 Taylor, J., & Baines, C. (2012). Performance management in UK universities: implementing the Balanced Scorecard. Journal of Higher Education Policy and Management, 34(2), 111-124. Doi: https://doi.org/10.1080/1360080x.2012.662737 IJAHP Article: Moradi/Performance evaluation of University faculty by combining BSC, AHP and TOPSIS: From the students’ perspective International Journal of the Analytic Hierarchy Process 27 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 Villegas, J. G., Castañeda, C., & Castañeda-Gómez, E. (2020). Planning and performance measurement in higher education: three case studies of operational research application. Revista Facultad de Ingeniería Universidad de Antioquia, 100, 97-112. Doi: https://doi.org/10.17533/udea.redin.20210526 Vinodh, S., Prasanna, M., & Prakash, N. H. (2014). Integrated Fuzzy AHP–TOPSIS for selecting the best plastic recycling method: A case study. Applied Mathematical Modelling, 38(19-20), 4662-4672. Doi: https://doi.org/10.1016/j.apm.2014.03.007 Wu, H.-Y., Lin, Y.-K., & Chang, C.-H. (2011). Performance evaluation of extension education centers in universities based on the balanced scorecard. Evaluation and Program Planning, 34(1), 37-50. Doi: https://doi.org/10.1016/j.evalprogplan.2010.06.001 Wu, Y., & Li, C. (2009). Research on performance evaluation of higher education based on the model of BSC-DRF-DEA. Paper presented at the 2009 16th International Conference on Industrial Engineering and Engineering Management. Doi: https://doi.org/10.1109/icieem.2009.5344267 Yildiz, A., Ayyildiz, E., Taskin Gumus, A., & Ozkan, C. (2020). A modified balanced scorecard based hybrid pythagorean fuzzy AHP-topsis methodology for ATM site selection problem. International Journal of Information Technology & Decision Making, 19(02), 365-384. Yılmaz, G., & Nuri İne, M. (2018). Assessment of sustainability performances of banks by TOPSIS method and balanced scorecard approach. International Journal of Business and Applied Social Science (IJBASS), 4(1). Yousif, M. K., & Shaout, A. (2018). Fuzzy logic computational model for performance evaluation of Sudanese Universities and academic staff. Journal of King Saud University- Computer and Information Sciences, 30(1), 80-119. Doi: https://doi.org/10.1016/j.jksuci.2016.08.002 Yucesan, M., & Gul, M. (2020). Hospital service quality evaluation: an integrated model based on Pythagorean fuzzy AHP and fuzzy TOPSIS. Soft Computing, 24(5), 3237- 3255.Doi: https://doi.org/10.1007/s00500-019-04084-2 Yudatama, U., & Sarno, R. (2016). Priority determination for higher education strategic planning using balanced scorecard, FAHP and TOPSIS (Case study: XYZ University). Paper presented at the IOP Conference Series: Materials Science and Engineering. Doi: https://doi.org/10.1088/1757-899x/105/1/012040 Zolfani, S. H., & Ghadikolaei, A. S. (2013). Performance evaluation of private universities based on balanced scorecard: empirical study based on Iran. Journal of Business Economics and Management, 14(4), 696-714. Doi: https://doi.org/10.3846/16111699.2012.665383 IJAHP Article: Moradi/Performance evaluation of University faculty by combining BSC, AHP and TOPSIS: From the students’ perspective International Journal of the Analytic Hierarchy Process 28 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 IJAHP Article: Moradi/Performance evaluation of University faculty by combining BSC, AHP and TOPSIS: From the students’ perspective International Journal of the Analytic Hierarchy Process 29 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 APPENDIX During the research process six questionnaires, including five related to the AHP and one related to TOPSIS, were distributed among 10 students. Since five AHP questionnaires were similar, the only difference being their rows and columns, only one of them (pairwise comparison matrices of BSC perspectives) is included in the Appendix as Table 22. Next, the TOPSIS questionnaire is given in Table 23. Table 22 Questionnaire of the pairwise comparison matrices of BSC perspectives Education level: Faculty: Department: Age: Instruction for filling out the questionnaire: The table (matrix) below reflects your opinions and preferences towards the perspectives of BSC, which are given as the rows and columns of the matrix. To express your preference in a correct way, you should obey the following rule: if you prefer element X of a row over element Y of a column, then use integer numbers ranging 2 to 9 at the blank place, the greater the number is, the stronger your preference is; if you prefer element Y of a column over element X of a row, then choose a number in the set {1/9,1/8,1/7,1/6,1/5,1/4,1/3,1/2}, the lower the number is, the stronger your preference is. Number 1 shows indifference! BSC 1 2 3 4 1 ■ 2 ■ ■ 3 ■ ■ ■ 4 ■ ■ ■ ■ Inconsistency rate ■ Table 23 Questionnaire of the Decision Matrix of TOPSIS Education level: Faculty: Department: Age: Instruction for filling out the questionnaire: The table (matrix) below reflects your opinions on the score of each measure for each faculty, which are given as the rows and columns of the matrix. To express your preference in a correct way, you should obey the following rule: choose an integer number ranging 1 to 9 to give a score for the performance of each faculty in each measure, the greater the number is, the stronger your score is. Decision matrix Measures 1.1.1 1.1.2 … 4.5.1 FENS … FASS … Business school …