IJAHP Article: Shamshol Bahri, Shariff, Yahya/ Comparative analysis on decision criteria for port personnel using hybrid Analytical Hierarchy Process (H-AHP) International Journal of the Analytic Hierarchy Process 1 Vol. 14 Issue 3 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 COMPARATIVE ANALYSIS ON DECISION CRITERIA FOR PORT PERSONNEL USING HYBRID ANALYTICAL HIERARCHY PROCESS (H-AHP) Muhamad Safuan Shamshol Bahri Malaysia Institute of Transport (MITRANS) Universiti Teknologi MARA, Shah Alam Malaysia msafuansham@gmail.com S.Sarifah Radiah Shariff 1* Malaysia Institute of Transport (MITRANS) Universiti Teknologi MARA, Shah Alam School of Mathematical Sciences, College of Computing, Informatic and Media Universiti Teknologi MARA, Shah Alam Malaysia shari990@uitm.edu.my Nazry Yahya Johor Port Skills Centre, Johor Port Berhad Malaysia nazry@johorport.com.my ABSTRACT The demand for talented labor to serve the maritime logistics, particularly in port operations, is growing as the industry expands globally. This requires professional competency in terms of manpower and skills that must be developed effectively. Furthermore, considering the harm done to the industry from the COVID-19 pandemic, capable professionals in port management are a crucial part to reviving the business and its long-term growth and viability. This study explores the criteria for the needed talent from the perspective of port logistics experts using the following multicriteria decision making (MCDM) approaches: Analytical Hierarchy Process (AHP), Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) and Preference Ranking Organization System Method for Enrichment Evaluation (PROMETHEE). The objective of this study is to identify the important criteria for personnel performance evaluation in the port marine logistics industry. In order to determine the performance evaluation framework for personnel performance evaluation, the study uses the AHP method to calculate the weightage of the criteria. The highest weightage is Work Attitude (0.560), followed by Job Performance (0.298) and Work Ability (0.120). Lastly, in order to identify the suitable hybrid MCDM approaches for personnel performance evaluation in the port marine logistics industry, three different MCDM approaches (AHP, TOPSIS and PROMETHEE) were used and the results show that the AHP is the best MCDM method to rank the 1 Acknowledgements: We would like to acknowledge Malaysia Institute of Transport (MITRANS) UiTM Shah Alam for funding this study and all participants who contributed to this study. *Corresponding author mailto:msafuansham@gmail.com mailto:shari990@uitm.edu.my IJAHP Article: Shamshol Bahri, Shariff, Yahya/ Comparative analysis on decision criteria for port personnel using hybrid Analytical Hierarchy Process (H-AHP) International Journal of the Analytic Hierarchy Process 2 Vol. 14 Issue 3 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 personnel in the port industry by obtaining the highest Kendallโ€™s Tau coefficient of 0.619. Keywords: Analytical Hierarchy Process; TOPSIS; PROMETHEE; hybrid; marine personnel 1. Introduction Logistics assists the flow of goods within and ahead of national borders and is deemed a significant component of the modern economy. It is a key enabler for economic sectors such as manufacturing, agriculture, and retail. Therefore, the logistics industry is important since it has the ability to improve the economic and social opportunities of a nation in delivering positive multiplier effects such as improved market accessibility and job availability. Since a good business performance measurement system is a very powerful tool to motivate and monitor employees, especially within the logistics related industry, a well-designed system has gathered interest among researchers. In the current competitive environment, a performance measurement system adopted by companies should include strategic success factors as part of the elements. At the same time, as the logistics industry has become a critical component in the commercial link by adding value, the growing business in sea trade has increased the value of services such as electronic tracking, warehousing, and resources distribution. In addition, 90% of global trade in Malaysia is conducted through marine transportation and logistics. The marine logistics industry is expected to grow globally, at a Compound Annual Growth Rate (CAGR) of 4.5% during the forecasted period of 2021 to 2030 (Industry Growth Insight, 2021). The growth of this market also drives the need for talented manpower to support the industry. Key players in the industry in many countries are focusing on producing talented manpower who are innovative and creative with technological advances and able to develop pioneering breakthroughs. However, there is a need to determine the traits of the right individuals for the industry. In Malaysia, in order to remain competitive, all industry players need to act fast in line with the development of technology produced through Industry 4.0. Necessary steps include strengthening the education system in the field of maritime technology and establishing a National Shipping and Port Council (NSPC) in order to ensure the continuity of the talent produced for the industry (AbdHamid, 2020). Also, in Malaysia the employment market is rigid in nature, meaning people who have been in a particular industry, namely marine logistics, for many years accumulating specific knowledge and skills are at a loss when they are retrenched because all their hard- earned knowledge might not be suitable in other industries (Benjamin, 2019; Cicek et al., 2019). In this study, several traits or criteria for talented professionals, especially for the marine logistics industry, were considered in an effort to develop a performance evaluation model. Since performance evaluation is expected to be comprehensive, it should also be reliable and consistent. The chosen indicators must be selected carefully as most of the time those indicators are weighted subjectively. Different levels of working positions as well as different job scopes produce different opinions and views about the indicators. Qu et al. (2015) proposed a hybrid model to measure the performance of professionals in the marine logistics industry for innovation and technology. Our study adopted the same three main criteria: job performance, work ability and work attitude, followed by several other sub-criteria. In a study by Othman et al. (2020), an IJAHP Article: Shamshol Bahri, Shariff, Yahya/ Comparative analysis on decision criteria for port personnel using hybrid Analytical Hierarchy Process (H-AHP) International Journal of the Analytic Hierarchy Process 3 Vol. 14 Issue 3 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 advanced Analytical Hierarchy Process (AHP) was used to identify the main factors that contribute to imbalanced cargo flows at large-scale minor ports in Malaysia. Extensive adaptation of the AHP can be seen in constructing the performance evaluation metrics within various organizations. In this study, in line with the growing need for talented manpower in maritime logistics, specifically focusing on port personnel (all roles in ports that are able to contribute to smooth operations), criteria for measuring their performance were explored using various combinations of the AHP. Therefore, the objective of this study is to adopt various combinations of MCDM approaches to the AHP to evaluate the performance of port personnel and compare the performance of each of the combinations. 2. Methodology Table 1 shows the proposed criteria to be considered as the indicators for performance. The ultimate goal of the performance of each professional in the industry is to measure their performance while considering three elements: job performance, work ability and work attitude. However, each of the elements is constituted by several criteria as described in Table 1. Table 1 Proposed elements for three level indicators Goal indicator Level indicator Secondary indicators Tertiary indicators To determine the criteria with the main priority Job performance Productivity(๐‘‰11) Job experience (๐‘‰111) Workplace (๐‘‰112) Organization Rule (๐‘‰113) Personnel behavior(๐‘‰12) Leadership(๐‘‰121) Work culture(๐‘‰122) Effective communication(๐‘‰123) Work ability Social influence(๐‘‰21) Reputation(๐‘‰211) Success rate(๐‘‰212) Referral center(๐‘‰213) Core competence(๐‘‰22) Keen insight and flexibility (๐‘‰221) Logical thinking(๐‘‰222) Innovation ability(๐‘‰223) Work attitude Teamwork(๐‘‰31) Sharing knowledge(๐‘‰311) Cooperation spirit(๐‘‰312) Team Diversity(๐‘‰313) Job satisfaction(๐‘‰32) Responsibility(๐‘‰321) Enthusiasm for work(๐‘‰322) Discipline(๐‘‰323) Source: (Bahri et al., 2020) IJAHP Article: Shamshol Bahri, Shariff, Yahya/ Comparative analysis on decision criteria for port personnel using hybrid Analytical Hierarchy Process (H-AHP) International Journal of the Analytic Hierarchy Process 4 Vol. 14 Issue 3 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 2.1 Data setting Close-structured questionnaires were distributed to port personnel at the managerial level in several ports in Malaysia to gain their feedback regarding the research. The purposive sampling technique was applied to focus on the selection of a group of qualified decision makers based on their experience and expertise within the scope of this study. In this study, there were two parts to the questionnaire (Part A and Part B). The objective of Part A was to determine the elements for the performance indicators of the port personnel and to ensure that they are in line with the expertsโ€™ opinions. The respondents were asked to rank the importance of each element and its sub-elements. The responses were tabulated and examined for consistency and validity. In Part B, the objective of the questionnaire was to determine the weightage of the indicator of the performance evaluation index. The respondents were asked to compare the importance of each sub-element based on Saatyโ€™s level of importance as seen in Table 2. Table 2 Saatyโ€™s level of importance Scale Meaning Explanation 1 Equally important Two criteria have equal importance 3 Weakly important Experience and judgement slightly favor one criterion over another 5 Strongly important Experience and judgement strongly favor one criterion over another 7 Very strongly important A criterion is favored very strongly over another 9 Extremely important The evidence favoring one criterion over another is of the highest possible affirmative 2,4,6,8 Intermediate value between adjacent scales When there is a compromise between the judgement Source: (Saaty, 1990) Considering the consistency and validity of the responses, only five (5) expertsโ€™ responses were used for the detailed analysis. Those experts are the top managers from different departments of major ports in Malaysia, had more than 10 years working experience and hold decision making positions. Table 3 summarizes the profile of experts. IJAHP Article: Shamshol Bahri, Shariff, Yahya/ Comparative analysis on decision criteria for port personnel using hybrid Analytical Hierarchy Process (H-AHP) International Journal of the Analytic Hierarchy Process 5 Vol. 14 Issue 3 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 Table 3 Selected experts Respondent Organization Department Position Years worked R1 Thought Partners Group Consulting - Group Managing Partner 30 R2 Johor Port Operations Group Managing Container 27 R3 Kuantan Port Operations Chief of Operation 24 R4 Port Tanjung Pelepas Quality Assistant Group Managing 19 R5 Johor Port Berhad Special Project Senior Manager 14 2.2 Hybrid Multi-Criteria Decision Making (MCDM) There has been a substantial effort in the use of combined MCDM methods or hybrid methods in order to improve decisions. Mesghouni et al. (1999), one of the earlier references to a hybrid method in decision-making (HMCDM), considered three approaches in their study. Genetic algorithms (GAs), constraints logic programming (CLP), and MCDM were combined to address a scheduling challenge. Shyur and Shih (2006) coined the name "HMCDM" to describe the MCDM approach that combined the Analytic Network Process (ANP) with TOPSIS. Figure 1 shows how MCDM methods can be combined with other methods to calculate the relative significance of criteria. Figure 1 Composition of hybrid MCDM Source: (Zavadskas, et al., 2016) IJAHP Article: Shamshol Bahri, Shariff, Yahya/ Comparative analysis on decision criteria for port personnel using hybrid Analytical Hierarchy Process (H-AHP) International Journal of the Analytic Hierarchy Process 6 Vol. 14 Issue 3 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 Several flaws in traditional MCDM methods can be addressed by employing the proposed hybrid methods, as follows: (1) Choosing the best strategy is a constant issue in any circumstance that necessitates a decision. Different MCDM methods can result in different alternative rankings. There is no single method, specific or general, that can be deemed the best for any situation (Saaty & Ergu, 2015). Therefore, it is recommended that more than one MCDM approach be employed and the results combined before making a final decision. (2) The importance of each criterion in the studied situation might have a substantial impact on the ranking order and final conclusion. There are studies available that do not use weighting and assign the same importance to all criteria (Dahooie et al., 2018). The hybrid strategy proposes doing two jobs at once, defining criteria weights and values and incorporating them into the multiattribute utility function value. Furthermore, incorporating criteria weights, which are derived using a variety of objective and subjective weighing methodologies, helps to more accurately reflect stakeholders' preferences. (3) The decision-making models should be as near to real-world challenges as possible. Fuzziness in the decision-making process sometimes arises from unclear management situations where ambiguities and challenges make it difficult to draw informed conclusions. As a result, it is preferable to combine MCDM with fuzzy sets or grey numbers. Fuzzy logic could aid in the resolution of uncertainty arising from human qualitative judgments and incomplete preference connections (Herrera-Viedma et al., 2020). (4) Other methods can be used to provide further rationale in the problem formulation. Because there are no universally accepted metrics for sustainability assessments (Ingwersen et al., 2014; Lima-Junior & Carpinetti, 2020), quantitative and qualitative methods can be used to generalize data, choose sustainability assessment indicators, and derive evaluation criteria for multiple criteria analysis. 2.3 Types of MCDM methods There are several MCDM approaches that are commonly used in the areas of operations research and management science. These two areas are focused on making better decisions based on logical approaches. Therefore, in order to ensure structured decisions are made, many attempts to consider all possible elements resulted in these approaches. These approaches each have their own strengths in producing structured decisions for any organization. Analytical Hierarchy Process (AHP) According to Yadav et al., (2015), the Analytical Hierarchy Process (AHP) was first established in 1977 by Thomas Saaty at the Wharton Business School. The AHP is primarily based on pairwise comparison matrices, which are used to establish preferences between alternatives for various criteria and rating systems with the help of a decision maker. The technique analyzes both qualitative and quantitative elements. It breaks down complex issues into tangible and intangible components, then organizes them into hierarchies of criteria and options, ranking them from most to least important (Korkmaz, 2019). IJAHP Article: Shamshol Bahri, Shariff, Yahya/ Comparative analysis on decision criteria for port personnel using hybrid Analytical Hierarchy Process (H-AHP) International Journal of the Analytic Hierarchy Process 7 Vol. 14 Issue 3 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 Many academics are interested in the AHP because of its solid mathematical approach and the ease in which the essential input data can be obtained. The AHP is a decision-making approach that can be used to solve difficult decisions involving many criteria. Objectives, criteria, subcriteria, and options are organized in a multi- level hierarchical structure. A set of paired comparisons is used to acquire relevant data. In terms of each individual decision criterion, this comparison is used to determine the importance of decision criteria and alternative performance indicators. There is also a way to improve consistency if the comparison is not entirely consistent (Pamucar et al., 2018; Mi et al., 2019) The AHP provides decision-makers with a variety of benefits and advantages. Making the selection process transparent, organizing decisions in a structured hierarchical style, offering a framework for reviewing and reconciling decision contradictions, and facilitating information synthesis and sensitivity analysis are all examples of the advantages of using the AHP (Sharma & Sehrawat, 2020). According to Kraujalienฤ—, (2019), the AHP's main feature is pairwise comparison, which is useful for comparing several options in the case of multiple variables and subjective preferences. To derive the priority scales, this method relies on the judgements of selected specialists or experts. Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) Kahraman et al. (2007) were among the first to design TOPSIS, the well-known traditional MCDM approach. TOPSIS finds the solutions that are simultaneously close to the ideal point and far from the anti-ideal point by positioning the options. As a result, the relevant options are chosen. TOPSIS is based on a simple and straightforward concept. It depicts the distances to both the positive and negative ideal solutions (PIS and NIS) in real time, and preferences are ordered based on their relative closeness, as well as a mixture of these two-distance metrics (Solangi et al., 2021). TOPSIS is a method for defining ideal and anti-ideal solutions that is based on the premise that the best alternatives should be the closest to the Positive Ideal Solution (PIS) and the furthest away from the Negative Ideal Solution (NIS). TOPSIS is a preferred method since it gives better results compared to other considered approaches in ranking the candidates or the alternatives (Korkmaz, 2019). The TOPSIS approach is simple to use and can efficiently manage both qualitative and quantitative data. However, it only analyzes a single value, whereas human judgments are frequently confusing and cannot be assessed using fixed numbers. The TOPSIS tool is based on the following rule according to Ighravwe et al. (2021). The optimal output should be the farthest point from the negative ideal solution point and the shortest line from the positive ideal solution point. Preference Ranking Organization System for Enrichment Evaluation (PROMETHEE) PROMETHEE (Preference Ranking Organization System for Enrichment Evaluation) is another preferred decision-making method due to its simple concept and application when compared to other methods. It is ideally suited to challenges with a limited number of possible actions to rate based on some, often contradicting, criteria (Turcksin et al., 2011). The PROMETHEE technique has six primary types of optional functions, allowing the decision maker to define flexible standards based on specific requirements. IJAHP Article: Shamshol Bahri, Shariff, Yahya/ Comparative analysis on decision criteria for port personnel using hybrid Analytical Hierarchy Process (H-AHP) International Journal of the Analytic Hierarchy Process 8 Vol. 14 Issue 3 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 The PROMETHEE method is an outranking approach that evaluates alternatives with respect to multiple criteria, which is a highly useful characteristic in this study because there are a few alternatives considered simultaneously. In addition, the PROMETHEE method is capable of approaching the methods of expression and synthesis of human mind choices in the face of competing perspectives. The PROMETHEE approach involves constructing an outranking relationship by comparing the assessed alternatives pairwise in each individual criterion (Kolios et al., 2016). 3. Results and discussion The objective of this study is to adopt various combinations of MCDM approaches with the AHP to evaluate the performance of port personnel and compare the performance of each of the combinations. Part A of the questionnaire asked the experts to rank the importance of all criteria and subcriteria considered in this study. Table 4 shows the results of Part A. Based on the ranking from 5 to 1 (Very Strongly Agree to Disagree), the average score from the experts was 3.88 with a low standard deviation of 0.61. However, the lowest score was 3.00 for code (๐‘‰212) which is Success Rate. This means that most of the experts gave a rank of 3 meaning they agree with considering this criterion in the evaluation of personnel. The highest standard deviation was 0.89 for codes (๐‘‰122), (๐‘‰123) and (๐‘‰221) which are Work Culture, Effective Communication and Keen Insight and Flexibility, respectively. Table 4 shows that the scores given in ranking the indicators were not largely dispersed or were nearly constant. This is due to the low value of the standard deviation (0.61). A low standard deviation means the values are consistent with each other. A low standard deviation indicates that the values tend to be close to the mean while a high standard deviation indicates that the data are more spread out and not consistent. Large standard deviation values show that an estimation procedure yields very different parameter estimates when facing equivalent data and its estimations are not precise (Bahri et al., 2020). Thus, the proposed criteria are acceptable to evaluate the personnel of port logistics. IJAHP Article: Shamshol Bahri, Shariff, Yahya/ Comparative analysis on decision criteria for port personnel using hybrid Analytical Hierarchy Process (H-AHP) International Journal of the Analytic Hierarchy Process 9 Vol. 14 Issue 3 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 Table 4 Summary of expertsโ€™ opinions on level of importance of evaluation criteria Criteria Ranking by Experts Code Average Std Dev (๐‘‰111) 4.40 0.55 (๐‘‰112) 3.60 0.55 (๐‘‰113) 3.80 0.45 (๐‘‰121) 4.60 0.55 (๐‘‰122) 3.60 0.89 (๐‘‰123) 4.60 0.89 (๐‘‰211) 3.20 0.45 (๐‘‰212) 3.00 0.00 (๐‘‰213) 3.20 0.45 (๐‘‰221) 3.60 0.89 (๐‘‰222) 4.00 0.71 (๐‘‰223) 4.00 0.71 (๐‘‰311) 3.80 0.84 (๐‘‰312) 4.20 0.84 (๐‘‰313) 3.40 0.55 (๐‘‰321) 4.00 0.71 (๐‘‰322) 4.20 0.45 (๐‘‰323) 4.60 0.55 Average 3.88 0.61 After the criteria and subcriteria were finalized, the weightage of the criteria and subcriteria were calculated using the Analytical Hierarchy Process (AHP). As mentioned earlier, a tri-level hierarchical structure of criteria, with three criteria at the first level, six sub-elements at the second level and 18 sub-elements in the third level, with three for each sub-elements of the second level criteria, was proposed. Each of the main criteria were compared against each other (pairwise) and the results are presented in the preference matrix in Table 5. In the AHP, when two criteria are compared the reciprocal relationship between them is assumed (Qu et al., 2015; Othman et al., 2020). In Table 5, when Job Performance (๐‘‰1) is compared to Work Ability (๐‘‰2), the value is 8/3 which means Job Performance (๐‘‰1) is very extremely important compared to Work Ability (๐‘‰2), and Work Ability (๐‘‰2) is only 3/8 = 0.375 or 37.5% important compared to Job Performance (๐‘‰1). Note that the value for comparing Work Attitude (๐‘‰3) to Work Ability (๐‘‰2) is not exactly its reciprocal of comparison between Work Ability (๐‘‰2) to Work Attitude (๐‘‰3). As there was more than one decision maker in this study, the preferences of the decision makers were averaged, and the value of ( ) was calculated using the following equation: IJAHP Article: Shamshol Bahri, Shariff, Yahya/ Comparative analysis on decision criteria for port personnel using hybrid Analytical Hierarchy Process (H-AHP) International Journal of the Analytic Hierarchy Process 10 Vol. 14 Issue 3 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 where K = total number of decision-makers (Othman et al., 2020). Table 5 Pairwise Comparison Matrix for main criteria (๐‘ฝ๐Ÿ) (๐‘ฝ๐Ÿ) (๐‘ฝ๐Ÿ‘) (๐‘‰1) 1 8/3 1/7 (๐‘‰2) 3/8 1 1/7 (๐‘‰3) 13/2 36/5 1 Table 6 shows the calculated weights of each criterion. The weightage for Job Performance (๐‘‰1) is 0.161, Work Ability (๐‘‰2) = 0.084 and Work Attitude (๐‘‰3) = 0.755. The weightage of Work Attitude (๐‘‰3) is the highest which means it is ranked as the top priority in measuring performance. Job Performance (๐‘‰1) is ranked second and Work Ability (๐‘‰2) is ranked third. Table 6 Calculated weightage (w) of main criteria (๐‘ฝ๐Ÿ) (๐‘ฝ๐Ÿ) (๐‘ฝ๐Ÿ‘) weightage (๐‘‰1) 0.13 0.25 0.11 0.161 (๐‘‰2) 0.05 0.09 0.11 0.084 (๐‘‰3) 0.83 0.66 0.78 0.755 3.1 Consistency check The consistency is calculated in order to ensure the judgements of the experts are consistent. The consistency check of the comparisons is determined by calculating the maximal eigenvalue according to the following equation: IJAHP Article: Shamshol Bahri, Shariff, Yahya/ Comparative analysis on decision criteria for port personnel using hybrid Analytical Hierarchy Process (H-AHP) International Journal of the Analytic Hierarchy Process 11 Vol. 14 Issue 3 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 The value in Table 5 is used to calculate the Consistency Index (CI) for the main criteria. For the main criteria, the CI is calculated as follows: 3.0834 0.0417 Lambda(max) is the matrixโ€™s largest eigenvalue. Then, the consistency ratio is calculated using the random consistency index (RI) where consistency ratio (CR) = CI/RI = 0.0417/0.52 = 0.0802 x 100% = 8.02 % The pairwise comparisons for the evaluation of the main criteria are consistent because the CR for the comparison matrix is less than 10%. As depicted in Figure 2, all the consistency ratios for each of the criteria are less than 10% which means that the judgements of the experts are acceptable. The highest CR is 9.55% for Social Influence Competence (๐‘‰21) which is close to 10%. This means that the judgement may be inconsistent but is still reliable and acceptable. The lowest CR is 0% for Job Performance, Work Ability and Work Attitude, which means that the judgements are perfectly consistent (Rahul et al., 2018; Meybodi, 2015). Table 7 shows the global weightage for the subcriteria. Figure 2 Summary of evaluation structure with its consistency ratio value IJAHP Article: Shamshol Bahri, Shariff, Yahya/ Comparative analysis on decision criteria for port personnel using hybrid Analytical Hierarchy Process (H-AHP) International Journal of the Analytic Hierarchy Process 12 Vol. 14 Issue 3 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 Table 7 Global weightage for subcriteria Elements Weightage Consistency Index ๐‘‰111 0.055 0.0728 ๐‘‰112 0.069 ๐‘‰113 0.116 ๐‘‰121 0.030 0.0955 ๐‘‰122 0.006 ๐‘‰123 0.023 ๐‘‰211 0.023 0.0754 ๐‘‰212 0.029 ๐‘‰213 0.024 ๐‘‰221 0.010 0.0176 ๐‘‰222 0.011 ๐‘‰223 0.044 ๐‘‰311 0.147 0.0147 ๐‘‰312 0.199 ๐‘‰313 0.118 ๐‘‰321 0.038 0.0190 ๐‘‰322 0.023 ๐‘‰323 0.035 3.2 Data preparation and application This study attempted to rank the port personnel based on the proposed criteria identified in section 3.1. The ranking was carried out using three multi-criteria decision-making methods, AHP, TOPSIS and PROMETHEE. The reason the MCDM methods were used was that they allow the personnel evaluation to be conducted in an easier and faster manner, considering input from experienced management within the organization. The weightage of the calculated criteria using the AHP (as summarized in Figure 2) is used in this section in order to rank the personnel. Figure 3 shows the flow of the analysis. In this study, the performance of seven personnel (Demerci & Kฤฑlฤฑรง, 2019) was used to test the applicability of the proposed approaches. The personnel scores on the seven criteria used to measure their performance are shown in Tables 8 and 9. The criteria are Education, Personality and Personal Skills, Experience, Technical Skills and Requirements, Foreign Language, Vocational Flexibility and Exam Results. IJAHP Article: Shamshol Bahri, Shariff, Yahya/ Comparative analysis on decision criteria for port personnel using hybrid Analytical Hierarchy Process (H-AHP) International Journal of the Analytic Hierarchy Process 13 Vol. 14 Issue 3 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 Figure 3 Flow of analysis IJAHP Article: Shamshol Bahri, Shariff, Yahya/ Comparative analysis on decision criteria for port personnel using hybrid Analytical Hierarchy Process (H-AHP) International Journal of the Analytic Hierarchy Process 14 Vol. 14 Issue 3 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 Table 8 Adapted criteria from literature No Criteria from Demerci and Kฤฑlฤฑรง (2019) Proposed criteria (in this study) 1 Education Quality of personnel 2 Personality Personnel behavior 3 Experience Social Influence Technical Skills 4 Exam result Core competence 5 Foreign language Team cooperation 6 Vocational Job satisfaction The scores of the seven criteria in Table 9 were then adapted into the new value to be in line with the new proposed criteria as shown in Table 10 using the Monte Carlo simulation method (Zhu, Tian, & Yan, 2020). Table 9 Scores of seven criteria from the literature Education Personality Experience Technical Skills Foreign Language Vocational Exam Results C 1 100 100 100 80 85 100 100 C 2 97 100 80 80 85 100 100 C 3 97 100 80 70 75 100 95 C 4 97 100 90 85 80 100 80 C 5 100 100 80 70 65 100 10 C 6 100 100 80 75 70 100 15 C 7 100 100 50 70 50 100 20 Each personnel has his/her own associated score of criteria. The score for each personnel was derived from the literature review (Demirci & Kฤฑlฤฑรง, 2019) and was modified by recalculating the probability distribution for scores in the form of the proposed structure of criteria depicted in Figure 2. Recall that the proposed structure of performance evaluation is based on three main criteria and 18 subcriteria (Figure 2). As mentioned earlier, the relevant scores were then generated using the Monte Carlo simulation method (Zhu, Tian, & Yan, 2020). For example, C1 from the literature scored 100 for Education (which is mapped onto Quality of Personnel, refer to Table 8), then using the Monte Carlo simulation method, the total random score for the subcriteria with respect to Quality of Personnel must also be 100. The first step is to randomize the number from 0-100; then, the initial value is chosen to be the value IJAHP Article: Shamshol Bahri, Shariff, Yahya/ Comparative analysis on decision criteria for port personnel using hybrid Analytical Hierarchy Process (H-AHP) International Journal of the Analytic Hierarchy Process 15 Vol. 14 Issue 3 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 for the first subcriteria. In the next step, the second value is chosen to be the value for the second subcriteria, and so on. The total for both values must be less than 100. Lastly, the value for the third subcriteria was assigned as the balance (to make up the total of 100). The randomization process was done 100 times for all candidates and the average values were calculated and adjusted for all criteria and subcriteria. Table 10 Scores of proposed criteria based on Monte Carlo simulation method Criteria C1 C2 C3 C4 C5 C6 C7 Job Performance ๐‘‰111 55 10 48 21 32 50 41 ๐‘‰112 37 56 23 28 30 17 17 ๐‘‰113 8 31 26 48 38 33 42 ๐‘‰121 65 49 17 28 42 18 46 ๐‘‰122 22 13 51 39 32 43 21 ๐‘‰123 13 38 32 33 26 39 33 Work Ability ๐‘‰211 22 86 58 37 54 47 22 ๐‘‰212 50 22 38 78 73 72 46 ๐‘‰213 108 52 54 60 23 37 53 ๐‘‰221 27 15 36 15 0 5 3 ๐‘‰222 34 44 31 47 8 6 6 ๐‘‰223 39 41 28 18 2 4 11 Work Attitude ๐‘‰311 10 34 16 21 17 34 12 ๐‘‰312 43 34 25 41 31 18 35 ๐‘‰313 32 17 34 18 17 18 3 ๐‘‰321 34 31 25 47 44 12 40 ๐‘‰322 44 38 38 40 11 59 37 ๐‘‰323 22 31 37 13 45 29 23 3.3 Results from the AHP The AHP is primarily based on pairwise comparison matrices, which are utilized by a decision maker to generate preferences between alternatives for various criteria and rating systems. The approach considers both qualitative and quantitative factors while making decisions. There are four steps to calculate the rank of personnel using the AHP: 1) forming the decision matrix (A), 2) normalizing the decision matrix, 3) generating a weighted normalized decision matrix and 4) identifying a performance score. Table 11 shows the score of the personnel after adding all the weighted normalized performance values of each personnel. Rank is allocated based on the performance IJAHP Article: Shamshol Bahri, Shariff, Yahya/ Comparative analysis on decision criteria for port personnel using hybrid Analytical Hierarchy Process (H-AHP) International Journal of the Analytic Hierarchy Process 16 Vol. 14 Issue 3 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 score. From the table, it can be concluded that C2 is the best with a performance score of 0.167 (the highest) and is ranked first Table 11 Performance scores of personnel Personnel Performance Score Rank C1 0.155 2 C2 0.167 1 C3 0.143 4 C4 0.155 3 C5 0.131 6 C6 0.132 5 C7 0.117 7 3.4 Results from TOPSIS The TOPSIS approach is a multi-criteria decision-making tool developed by Hwang and Yoon (1981). The TOPSIS approach is based on the largest distance between the positive and negative ideal solutions. The TOPSIS method consists of six steps: 1) forming the decision matrix (A), 2) forming the normalized decision matrix (R), 3) forming the weighted standard decision matrix (V), 4) finding the ideal (๐ด+) and negative ideal (๐ดโˆ’) solutions, 5) calculating the distance between alternatives, and 6) calculating the relative proximity to the ideal solution. Table 12 shows the ranking of personnel based on their ideal solution value. According to the results shown in Table 12, the highest scoring personnel is C2 while the lowest is C6. Table 12 Ranking of personnel using TOPSIS Personnel P Rank C2 0.631 1 C4 0.593 2 C1 0.530 3 C3 0.507 4 C6 0.488 5 C5 0.466 6 C7 0.397 7 IJAHP Article: Shamshol Bahri, Shariff, Yahya/ Comparative analysis on decision criteria for port personnel using hybrid Analytical Hierarchy Process (H-AHP) International Journal of the Analytic Hierarchy Process 17 Vol. 14 Issue 3 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 3.5 Result from PROMETHEE Outranking methods are well-suited to challenges in which a limited number of alternatives must be ranked based on a set of criteria. Preference Ranking Organization Method for Enrichment Evaluation (PROMETHEE) has been determined to be the most stable of the outranking approaches (Turcksin et al., 2011; Kolios et al., 2016). The information on the weights of the criterion and the decision maker's preference functions are required for PROMETHEE to be implemented. There are no explicit standards for determining the criteria weights in PROMETHEE. One of the most extensively used MCDM methods for applications such as selection, evaluation, allocations, prioritization, and ranking is the Analytic Hierarchy Process (AHP). Macharis et al. (2004) examined the strengths and shortcomings of the AHP and PROMETHEE and presented a method to improve PROMETHEE by incorporating the positive aspects of the AHP. Similar to other methods, the AHP is used to assign the criteria weights and PROMETHEE II is used to obtain the final ranking of the personnel. Table 13 shows the net outranking flow values for every personnel and their relative ranking. The best personnel that scored the highest net outranking flow is C2 while the worst personnel who scored the lowest net outranking flow is C7. Table 13 Net outranking values for every personnel Personnel Net outranking flow Rank C1 0.067 3 C2 0.152 1 C3 -0.033 4 C4 0.113 2 C5 -0.052 5 C6 -0.117 6 C7 -0.131 7 3.6 Choosing best MCDM method In order to choose the best MCDM method, a Kendallโ€™s Tau test was implemented. Kendall-Tau is a non-parametric correlation coefficient that can be used to evaluate and analyze correlations between ordinal variables that are not scaled in intervals. The Kendall tau correlation coefficient is sometimes abbreviated with the Greek letter (tau). The Spearman rank correlation coefficient and the Kendall tau correlation coefficient are thought to be comparable. The Spearman rank correlation coefficient is similar to the Pearson correlation coefficient, except that it is calculated using rankings, whereas the Kendall tau correlation is a probability. Table 14 shows the results of the first part of the calculations to choose the best MCDM method from among the AHP, TOPSIS, and PROMETHEE. Kendall's Tau, Goodman and Kruskal's Gamma, and Logistic Regression all use concordant (C) and discordant (D) pairs. They are used to determine if there is agreement (or disagreement) between scores for ordinal (ordered) variables. The data must be organized and grouped into pairs before concordance or discordance can be IJAHP Article: Shamshol Bahri, Shariff, Yahya/ Comparative analysis on decision criteria for port personnel using hybrid Analytical Hierarchy Process (H-AHP) International Journal of the Analytic Hierarchy Process 18 Vol. 14 Issue 3 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 calculated. Concordant pairs are the number of observed ranks below a particular rank which are larger than that particular rank while discordant pairs are the number of observed ranks below a particular rank which are smaller in value than that particular rank. The rankings from Table 14 are sorted from the lowest to highest. The data are compared with the literature review (Demirci & Kฤฑlฤฑรง, 2019). For example, in the AHP ranking, Demirci & Kฤฑlฤฑรง (2019) gave C1 the first rank and the AHP calculation gave it second; this is not perfectly concordant. Therefore, to calculate the concordant and discordant values, the number of rankings that are larger in value from the calculation are compared to the rankings given in the literature. Thus, with the AHP there are five rankings that are larger than 2, except personnel (C2) which is ranked as number 1. Then, the concordant is 5 and discordant is 1. The total number of concordant pairs for AHP is 17 and there are 4 discordant pairs. The same method is used for TOPSIS and PROMETHEE where the total value for concordant and discordant are C=16, D=5 and C=15, D=6, respectively. The Kendallโ€™s tau is calculated as follows: The Kendallโ€™s tau coefficient for the AHP is 0.619 while for TOPSIS and PROMETHEE it is 0.524 and 0.429, respectively. These test values range from -1 to +1, with the sign telling the direction of the relationship. Minus (-) means that as one increases the other decreases. While plus (+) means that as one goes up so does the other. The closer the value to +1 or -1, the stronger the relationship is. As the correlation coefficient value goes towards 0, the relationship between the two variables will be weaker; therefore, the best alternative between the three MCDM approaches is the AHP method because it has the highest Kendall's Tau coefficient that is closest to +1 (Ramsey, 1989). Table 14 Kendallโ€™s Tau coefficient for MCDM approaches AHP TOPSIS PROMETHEE LR Rank C D Rank C D Rank C D C1 1 2 5 1 3 4 2 3 4 2 C4 2 3 4 1 2 4 1 2 4 1 C2 3 1 4 0 1 4 0 1 4 0 C6 4 5 2 1 5 2 1 6 1 2 C5 5 6 1 1 6 1 1 5 1 1 C3 6 4 1 0 4 1 0 4 1 0 C7 7 7 7 7 Sum 17 4 Sum 16 5 Sum 15 6 0.619 0.524 0.429 IJAHP Article: Shamshol Bahri, Shariff, Yahya/ Comparative analysis on decision criteria for port personnel using hybrid Analytical Hierarchy Process (H-AHP) International Journal of the Analytic Hierarchy Process 19 Vol. 14 Issue 3 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 4. Conclusion As the marine logistics industry has become a critical component in the commercial link, talented manpower is needed to support the industry and provide a competitive advantage to the nation. This study explores the criteria for needed talents from the perspective of port logistics experts using the following MCDM approaches: AHP, TOPSIS and PROMETHEE. The three approaches were selected based on their ability to simultaneously consider many criteria as well as consider the subjective judgement of importance from the participating experts. Next, the weightage was calculated using the AHP method and the highest weightage is Work Attitude, followed by Job Performance and Work Ability. Work Attitude scored more than 50% of weightage compared to Job performance and Work Ability. Therefore, the experts believed that Work Attitude is the most important criterion in selecting and evaluating personnel in the port logistics industry. Work Attitude refers to the tendencies of personnel in a behavioral evaluation. In recruitment information, sense of responsibility and group spirit are very common, and should be given more emphasis in the logistics industry. Furthermore, being cautious and working hard are essential characteristics for logistics professionals. Job Performance scored the second highest weightage and refers to the goal of performance management. According to the literature, employers prefer individuals who have prior work experience. Employees who have worked for more than a year are expected to hold 76.1% of jobs. This indicates that recent graduates are unable to match employersโ€™ demands in terms of practical experience and that more employers are ready to select better workers from a job market brimming with inexperienced workers. The largest need for job experience is in production logistics. Work Ability is the least preferred criteria by the experts in evaluating the performance of personnel. Work Ability refers to a person possessing a set of standardized requirements. Communication skills, computer applications skills, English skills, and coordination skills have all become vital traits for a logistics expert in the current employment market. Furthermore, stress tolerance is a crucial consideration for businesses when hiring logistics personnel. Finally, the personnel were ranked based on their performance score using the following different approaches: AHP, TOPSIS and PROMETHEE. Based on the selected data from the literature, the results show that the best MCDM approach is the AHP method because it has the highest Kendallโ€™s Tau coefficient compared to TOPSIS and PROMETHEE. The same calculation method of Kendallโ€™s Tau rank correlation was used to reconfirm that the proposed weightage of the criteria agrees with the actual evaluation at the actual organization. The results of Kendallโ€™s Tau coefficient are 0.490 which means the variables are correlated. IJAHP Article: Shamshol Bahri, Shariff, Yahya/ Comparative analysis on decision criteria for port personnel using hybrid Analytical Hierarchy Process (H-AHP) International Journal of the Analytic Hierarchy Process 20 Vol. 14 Issue 3 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 REFERENCES AbdHamid, A.S. (2020, February 24). Plug the gaps to make Malaysia a maritime nation, Borneo Post online. https://www.theborneopost.com/2020/02/24/plug-the- gaps-to-make-malaysia-a-maritime-nation/ Benjamin, R. (2019, April 9). Mismatch of skills and jobs, The Star. https://www.thestar.com.my/opinion/letters/2019/04/09/mismatch-of-skills-and-jobs Cicek, K., Akyuz, E., & Celik, M. (2019). Future skills requirements analysis in maritime industry. Procedia Computer Science, 158, 270-274. Doi: https://doi.org/10.1016/j.procs.2019.09.051 Demirci, A. E., & Kฤฑlฤฑรง, H. S. (2019). Personnel selection based on integrated multi- criteria decision making techniques. International Journal of Advances in Engineering and Pure Sciences, 31(2), 163-178. Heidary Dahooie, J., Beheshti Jazan Abadi, E., Vanaki, A. S., & Firoozfar, H. R. (2018). Competencyโ€based IT personnel selection using a hybrid SWARA and ARASโ€G methodology. Human Factors and Ergonomics in Manufacturing & Service Industries, 28(1), 5-16. Doi: https://doi.org/10.1002/hfm.20713 Herrera-Viedma, E., Palomares, I., Li, C. C., Cabrerizo, F. J., Dong, Y., Chiclana, F., & Herrera, F. (2020). Revisiting fuzzy and linguistic decision making: Scenarios and challenges for making wiser decisions in a better way. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 51(1), 191-208. Doi: https://doi.org/10.1109/tsmc.2020.3043016 Hwang, C. L., & Yoon, K. (1981). Methods for multiple attribute decision making. Multiple attribute decision making: methods and applications a state-of-the-art survey. Multiple Attribute Decision Making, 1, 58-191. Doi: https://doi.org/10.1007/978-3-642-48318-9_3 Ighravwe, D. E., Babatunde, M. O., Mosetlhe, T. C., Aikhuele, D., & Akinyele, D. (2021). A MCDM-based framework for the selection of renewable energy system simulation tool for teaching and learning at university level. Environment, Development and Sustainability, 24(11), 1-22. Doi: https://doi.org/10.1007/s10668- 021-01981-1 Industry Growth Insights. (2021). Global maritime logistics market by type (General cargo maritime logistics, bulk cargo maritime logistics, maritime logistic), by application (port service, coastal service, other) and by region (North America, Latin America, Europe, Asia Pacific and Middle East & Africa), forecast from 2022 to 2030. https://industrygrowthinsights.com/report/maritime-logistics-market/, retrieved November 8, 2022. Ingwersen, W., Cabezas, H., Weisbrod, A. V., Eason, T., Demeke, B., Ma, X., ... & Ceja, M. (2014). Integrated metrics for improving the life cycle approach to assessing product system sustainability. Sustainability, 6(3), 1386-1413. Kahraman, C., AteลŸ, N. Y., ร‡evik, S., Gรผlbay, M., & ErdoฤŸan, S. A. (2007). Hierarchical fuzzy TOPSIS model for selection among logistics information https://www.theborneopost.com/2020/02/24/plug-the-gaps-to-make-malaysia-a-maritime-nation/ https://www.theborneopost.com/2020/02/24/plug-the-gaps-to-make-malaysia-a-maritime-nation/ https://www.thestar.com.my/opinion/letters/2019/04/09/mismatch-of-skills-and-jobs https://industrygrowthinsights.com/report/maritime-logistics-market/ IJAHP Article: Shamshol Bahri, Shariff, Yahya/ Comparative analysis on decision criteria for port personnel using hybrid Analytical Hierarchy Process (H-AHP) International Journal of the Analytic Hierarchy Process 21 Vol. 14 Issue 3 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 technologies. Journal of Enterprise Information Management, 20(2), 143-168. Doi: https://doi.org/10.1108/17410390710725742 Kolios, A., Mytilinou, V., Lozano-Minguez, E., & Salonitis, K. (2016). A comparative study of multiple-criteria decision-making methods under stochastic inputs. Energies, 9(7), 566. Doi: https://doi.org/10.3390/en9070566 Korkmaz, O. (2019). Personnel selection method based on TOPSIS multi-criteria decision making method. Uluslararasฤฑ ฤฐktisadi ve ฤฐdari ฤฐncelemeler Dergisi, 23, 1-16. Doi: https://doi.org/10.18092/ulikidince.468486 Lima-Junior, F. R., & Carpinetti, L. C. R. (2020). An adaptive network-based fuzzy inference system to supply chain performance evaluation based on SCORยฎ metrics. Computers & Industrial Engineering, 139, 106191. Doi: https://doi.org/10.1016/j.cie.2019.106191 Macharis, C., Springael, J., De Brucker, K., & Verbeke, A. (2004). PROMETHEE and AHP: The design of operational synergies in multicriteria analysis.: Strengthening PROMETHEE with ideas of AHP. European Journal of Operational Research, 153(2), 307-317. Doi: https://doi.org/10.1016/s0377-2217(03)00153-x Mesghouni, K., Pesin, P., Trentesaux, D., Hammadi, S., Tahon, C., & Borne, P. (1999). Hybrid approach to decision-making for job-shop scheduling. Production Planning & Control, 10, 690โ€“706. Doi: https://doi.org/10.1080/095372899232768 Meybodi, M. Z. (2015). Consistency of strategic and tactical benchmarking performance measures. Benchmarking: An International Journal, 22(6), 1019-1032. Doi: https://doi.org/10.1108/bij-07-2013-0074 Mi, X., Tang, M., Liao, H., Shen, W., & Lev, B. (2019). The state-of-the-art survey on integrations and applications of the best worst method in decision making: Why, what, what for and what's next?. Omega, 87, 205-225. Doi: https://doi.org/10.1016/j.omega.2019.01.009 Othman, M. K., Rahman, N. S. F. A., Ismail, A., & Saharuddin, A. H. (2020). Factors contributing to the imbalances of cargo flows in Malaysia large-scale minor ports using a fuzzy analytical hierarchy process (FAHP) approach. The Asian Journal of Shipping and Logistics, 36(3), 113-126. Doi: https://doi.org/10.1016/j.ajsl.2019.12.012 Pamuฤar, D., Steviฤ‡, ลฝ., & Sremac, S. (2018). A new model for determining weight coefficients of criteria in MCDM models: Full consistency method (fucom). Symmetry, 10(9), 393. Doi: https://doi.org/10.3390/sym10090393 Qu, Q., Chen, K. Y., Wei, Y. M., Liu, Y., Tsai, S. B., & Dong, W. (2015). Using hybrid model to evaluate performance of innovation and technology professionals in marine logistics industry. Mathematical Problems in Engineering, 2015. Doi: https://doi.org/10.1155/2015/361275 Rahul, B., Dey, P. K., Ghosh, S. K., & Konstantinos, P. (2018). Strategic maintenance technique selection using combined quality function deployment, the analytic hierarchy process and the benefit of doubt approach. International Journal https://doi.org/10.18092/ulikidince.468486 IJAHP Article: Shamshol Bahri, Shariff, Yahya/ Comparative analysis on decision criteria for port personnel using hybrid Analytical Hierarchy Process (H-AHP) International Journal of the Analytic Hierarchy Process 22 Vol. 14 Issue 3 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 of Advanced Manufacturing Technology, 94(1-4), 31-44. Doi: https://doi.org/10.1007/s00170-016-9540-1 Ramsey, P. H. (1989) Critical values for Spearmanโ€™s rank-order correlation. Journal of Educational Statistics, 14(3), 245-253. Doi: https://doi.org/10.3102/10769986014003245 Saaty, T. L., & Ergu, D. (2015). When is a decision-making method trustworthy? criteria for evaluating multi-criteria decision-making methods. International Journal of Information Technology & Decision Making, 14, 1171โ€“1187. Doi: https://doi.org/10.1142/s021962201550025x Saaty, T.L., (1990). How to make a decision: The Analytic Hierarchy Process, European Journal of Operational Research, 48, 9-26. Doi: https://doi.org/10.1016/0377-2217(90)90057-i Sharma, M., & Sehrawat, R. (2020). A hybrid multi-criteria decision-making method for cloud adoption: Evidence from the healthcare sector. Technology in Society, 61, 101258. Doi: https://doi.org/10.1016/j.techsoc.2020.101258 Shyur, H.-J., & Shih, H.-S. (2006). A hybrid MCDM model for strategic vendor selection. Mathematical and Computer Modelling, 44, 749โ€“761. Doi: https://doi.org/10.1016/j.mcm.2005.04.018 Solangi, Y. A., Longsheng, C., & Shah, S. A. A. (2021). Assessing and overcoming the renewable energy barriers for sustainable development in Pakistan: An integrated AHP and fuzzy TOPSIS approach. Renewable Energy, 173, 209-222. Doi: https://doi.org/10.1016/j.renene.2021.03.141 Turcksin, L., Bernardini, A., & Macharis, C. (2011). A combined AHP- PROMETHEE approach for selecting the most appropriate policy scenario to stimulate a clean vehicle fleet. Procedia-Social and Behavioral Sciences, 20, 954- 965. Doi: https://doi.org/10.1016/j.sbspro.2011.08.104 Van Der Vleuten, C. P., & Schuwirth, L. W. (2005). Assessing professional competence: from methods to programmes. Medical Education, 39(3), 309-317. Doi: https://doi.org/10.1111/j.1365-2929.2005.02094.x Yadav, A., Anis, M., Ali, M., & Tuladhar, S. (2015). Analytical hierarchy process (AHP) for analysis: selection of passenger airlines for Gulf country. International Journal of Scientific & Engineering Research, 6(3), 379-389. Zavadskas, E. K., Govindan, K., Antucheviciene, J., & Turskis, Z. (2016). Hybrid multiple criteria decision-making methods: A review of applications for sustainability issues. Economic Research-Ekonomska Istraลพivanja, 29(1), 857-887. Doi: https://doi.org/10.1080/1331677x.2016.1237302 Zhu, Y., Tian, D., & Yan, F. (2020). Effectiveness of entropy weight method in decision-making. Mathematical Problems in Engineering, 2020.