12018 FACTA UNIVERSITATIS Series: Mechanical Engineering Vol. 23, No 2, 2025, pp. 335 - 349 https://doi.org/10.22190/FUME230730050B © 2025 by University of Niš, Serbia | Creative Commons License: CC BY-NC-ND Original scientific paper A NEW CRITERIA IMPORTANCE ASSESSMENT (CIMAS) METHOD IN MULTI-CRITERIA GROUP DECISION-MAKING: CRITERIA EVALUATION FOR SUPPLIER SELECTION Sara Bošković1, Stefan Jovčić1, Vladimir Simic2,3, Libor Švadlenka1, Momčilo Dobrodolac2, Nebojša Bacanin4,5 1University of Pardubice, Faculty of Transport Engineering, Department of Transport Management, Marketing, and Logistics, Czech Republic 2University of Belgrade, Faculty of Transport and Traffic Engineering, Serbia 3Yuan Ze University, College of Engineering, Department of Industrial Engineering and Management, Taiwan 4Singidunum University, Faculty of Informatics and Computing, Serbia 5MEU Research Unit, Middle East University, Amman, Jordan ORCID iDs: Sara Bošković https://orcid.org/0000-0002-0331-4262 Stefan Jovčić https://orcid.org/0000-0002-9162-2133 Vladimir Simic https://orcid.org/0000-0001-5709-3744 Libor Švadlenka https://orcid.org/0000-0001-6484-6660 Momčilo Dobrodolac https://orcid.org/0000-0001-8112-8329 Nebojša Bacanin https://orcid.org/0000-0002-2062-924X Abstract. Decision-making is a challenging task for logistics managers when solving the supplier selection problem. It is usually affected by numerous conflicting criteria that are not equally important to all decision-makers. Criteria evaluation is one of the crucial parts here. The primary purpose of this paper is to propose a novel integrated criteria importance assessment method based on objective judgment and group decision-making. The developed method is applied to the criteria importance evaluation for supplier selection. First, we proposed the criteria importance assessment (CIMAS) method based on the expert’s opinion, where the years of experts’ experience were given in the form of an expert’s weight. Second, the obtained criteria weights are further integrated within the well-known CRITIC method, and the hybrid criteria weights are determined. The input data matrix is based on the experts’ criteria evaluation on the one-to-ten-point scale. The data were further analyzed by the novel CIMAS method and were utilized within the CRITIC (objective) method. The paper's main contribution is the proposal of the novel CIMAS method. Another contribution is coupling the subjective (CIMAS) and the objective (CRITIC) methods. The results reveal that the most important criterion for supplier selection is on-time distribution, followed by distribution cost, external image and appearance in public, social responsibility rate, and air pollution, respectively. The sensitivity and comparative analysis were also performed, and the technique confirmed a high level of stability. Key words: Decision-Making, Experts, Supplier Evaluation, CIMAS, CRITIC Received: July 30, 2023 / Accepted December 18, 2023 Corresponding author: Stefan Jovčić The University of Pardubice, Faculty of Transport Engineering, Department of Transport Management, Marketing, and Logistics, Studentská 95, 532 10 Pardubice, Czech Republic. E-mail: stefan.jovcic@upce.cz https://orcid.org/0000-0002-0331-4262 https://orcid.org/0000-0002-9162-2133 https://orcid.org/0000-0001-5709-3744 https://orcid.org/0000-0001-6484-6660 https://orcid.org/0000-0001-8112-8329 https://orcid.org/0000-0002-2062-924X mailto:stefan.jovcic@upce.cz 336 S. BOŠKOVIĆ, S. JOVČIĆ, V. SIMIĆ, L. ŠVADLENKA, M. DOBRODOLAC, N. BACANIN 1. INTRODUCTION The Multi-Criteria Decision-Making (MCDM) process has always been an intriguing issue for both researchers and managers all around the world. Each decision-making problem in life is affected by many criteria that interact with each other. Only some of the requirements are equally important to each decision-maker. For that reason, it is necessary to assign the proper importance to them. Many methods, that are used to evaluate the criteria, have been suggested by various authors in the MCDM field [1,2]. Two sorts of methods in the criteria weighting exist, the Subjective Weighting Methods (Direct Rating (DR), Simple Multi-Attribute Rating Technique (SMART), Delphi, Full Consistency Method (FUCOM), Best-Worst Method (BWM), Analytic Hierarchy Process (AHP), Stepwise Weight Assessment Ratio Analysis (SWARA), etc.) and the Objective Weighting Methods (Entropy, CRITIC, Standard Deviation, etc.). In this paper, the authors conducted research on the subjective methods that were proposed in the literature. The Web of Science database was this survey’s primary source of information. It is noticed from the literature that there are a bunch of publications regarding the criteria evaluation methods. Saaty [3] proposed the AHP method. Diakoulaki [1] developed the CRiteria Importance Through Intercriteria Correlation (CRITIC) method. Gabus and Fontela [4] developed the Decision- making Trial and Evaluation Laboratory (DEMATEL) method. Rezaei [5] developed the BWM. Keršuliene et al. [6] proposed the SWARA method. Pamučar et al. [2] developed the Full Consistency Method (FUCOM). All the proposed methods found their application in various fields. For instance, Xie et al. [7] applied DEMATEL for the site selection of supermarkets. Ortega et al. [8] used the BWM to find a sustainable park-and-ride location. Kant and Gupta [9] evaluated urban freight strategies using the BWM method. Regarding the AHP method, it is applied in various contexts such, as forest fire hazard modeling [10], risk assessment of metro systems [11], project post-evaluation [12], etc. The SWARA method has applications to various problems such as supplier evaluation [13], road sections ranking [14], risk assessment in coal supply chain management [15], glasshouse location selection [16]. When it comes to the FUCOM method, it has been applied to various branches such as in the manufacturing environment [17], human resources [18], landfill site selection [19], forklift efficiency analysis [20], etc. Besides MCDM methods for criteria importance assessment, in recent years, the field of MCDM has been growing rapidly. For instance, Bošković et al. [21] developed the AROMAN method to select the best electric vehicle alternative for last-mile delivery. This method has been further developed and applied to cargo bike delivery concept selection by Bošković et al. [22]. In addition, Yalçın et al. [23] developed an Intuitionistic Fuzzy-based AROMAN model for performance evaluation of EcoPorts. Švadlenka et al. [24] developed a picture-fuzzy decision-making approach for sustainable last-mile delivery. Simić et al. [25] developed a neutrosophic LOPCOW-ARAS model for prioritizing industry 4.0-based material handling technologies in smart and sustainable warehouse management systems. Tirkolaee et al. [26] proposed an integrated decision-support framework for resilient supply chain network design. They used the BWM and WASPAS methods that were further integrated within the type-2 neutrosophic fuzzy numbers (T2NN). Čubranić-Dobrodolac et al. [27] proposed a fuzzy-AROMAN-Fuller technique to select a professional driver. On the other hand, Nikolić et al. [28] applied an Interval Type-2 Fuzzy AROMAN Decision-Making method to improve the sustainability of the postal network in rural areas. Pamucar et al. [29] A New Criteria Importance Assessment (CIMAS) Method in Multi-Criteria Group Decision-Making... 337 prioritized sustainable mobility-sharing systems using integrated fuzzy DIBR and fuzzy-rough EDAS models. In this paper, a novel integrated criteria importance assessment method based on objective judgment and group decision-making is developed. First, we proposed the criteria importance assessment (CIMAS) method based on the expert’s opinion, where the years of experience of each expert are given in the form of an expert’s weight. Thus, the obtained criteria weights are further integrated within the well-known CRITIC method, and the hybrid criteria weights are determined. The authors of this paper followed the logic that the more years of experience one expert has, the more critical his criteria assessment decision is. To assess the criteria, the experts use a one-to-ten-point scale and thus formulate an input data matrix. The proposed integrated approach is to help decision-makers determine the criteria weights in multi-criteria decision-making problems, including a team of experts. The first part of the proposed method belongs to a group of subjective methods since a team of experts assesses the criteria in a decision-making process. The second part integrates the well-known CRITIC method, which belongs to the objective methods. The CRITIC method belongs to the objective methods developed and verified by Diakoulaki et al. in 1995 [1]. The correlation analysis obtains the contrasts among criteria [1]. This paper's main contribution lies in coupling subjective and objective methods to get more precise and accurate criteria assessments that are further used in decision-making. In addition, this paper emphasizes its further contributions: i) The original subjective method (CIMAS) to assess the criteria importance is proposed; ii) The proposed CIMAS method appears for the first time in the literature; iii) The method is general (includes the experts’ judgment) and is applicable to assess any criteria in real-life problems. iv) The proposed method is applied to evaluate the criteria for sustainable supplier selection. (v) Coupling the subjective (CIMAS) with the well-known objective (CRITIC) method, an integrated approach gives more precise and confident results. The rest of the paper is structured as follows: Section 2 thoroughly explains the proposed CIMAS method. In Section 3, the CIMAS method is applied to assess the criteria that affect the supplier selection problem. Sub-section 3.1 is the sensitivity analysis on the CIMAS method, where three scenarios are performed. Sub-section 3.2 is the comparative analysis with the CRITIC method. Sub-section 3.3 describes the hybrid criteria weights based on combining the CIMAS and the CRITIC methods. The main contribution should be found in this sub-section. Section 3.4 gives some managerial recommendations regarding the newly proposed method. Section 4 concludes the paper and gives some future research directions. 2. CRITERIA IMPORTANCE ASSESSMENT METHOD This paper proposes a new approach, the Criteria Importance Assessment (CIMAS), in decision-making. This method should be used to identify the criteria’ importance in a decision-making process using experts’ assessment. One important fact regarding the expert knowledge and experience is that this method considers the experts’ experience in the years spent in the considered field. As mentioned above, the authors of this paper followed the logic that the more years of experience one expert has, the more critical the criteria assessment decision is. The experts’ weights are further used in the criteria assessment. The calculated expert weights should be significant later in the decision-making process. In addition, the experts use a one-to-ten-point scale to formulate an input data matrix where 338 S. BOŠKOVIĆ, S. JOVČIĆ, V. SIMIĆ, L. ŠVADLENKA, M. DOBRODOLAC, N. BACANIN the most significant importance of one criterion is denoted by 10. In contrast, the lowest one is denoted by 1. A flowchart of the proposed method is depicted in Fig. 1. Fig. 1 A flowchart of the proposed method The CIMAS method is described through the following steps: Step 1. Define a set of evaluation criteria: The first step in a decision-making process is to define the criteria by which importance should be assessed. The criteria may be identified either from the literature or by consulting experts. Step 2. Identify the number of experts: Decision-maker identifies the number of experts involved in the field and collects information regarding their years of experience. Step 3. Establish the experts’ experience (years): The experts clarify their experience in the number of years spent in the field, and the decision-maker defines the importance of each expert. Step 4. Calculate the importance of each expert: The importance of each expert is calculated by following this logic: let us suppose that “q” experts participate in the criteria assessment procedure. Expert 1 has five years of experience in the considered field, Expert 2 has three years of experience, while Expert q participates with seven years of experience. The importance of experts’ evaluations (the expert weights) should be determined as 5/15 for Expert 1, 3/15 for Expert 2, and 7/15 for Expert q. In the general case, the expert’s importance can be calculated by Eq. (1). 𝑊𝐸𝑖 = 𝐸𝑖 ∑ 𝐸𝑖 𝑞 𝑖=1 , 𝑖 = 1, 2, … , 𝑞 (1) A New Criteria Importance Assessment (CIMAS) Method in Multi-Criteria Group Decision-Making... 339 Step 5. Formulate an input data matrix based on the experts’ assessment (1-10 Scale): In this step, an input decision-making matrix is defined. The experts use a one-to-ten- point scale to formulate an input data matrix where the most significant importance of one criterion is denoted by 10, while the lowest one is denoted by 1. The structure of the input data matrix is presented in Table 1. 𝐸𝑖 represents one of the q experts involved in a decision- making, 𝐶𝑗 is one of the p considered evaluation criteria, 𝑥𝑖𝑗 is the expert’s number i assessment of criteria number j importance (on a scale of 1-10), and finally, the values from 𝑊𝐸1 to 𝑊𝐸𝑞 are the experts’ weights. Table 1 Input Data Matrix Experts/Criteria 𝐶1 𝐶2 … 𝐶𝑗 … 𝐶𝑝 Experts’ Weights 𝐸1 𝑥11 𝑥12 … … … 𝑥1𝑝 𝑊𝐸1 𝐸2 𝑥21 𝑥22 … … … 𝑥2𝑝 𝑊𝐸2 𝐸𝑖 … … … 𝑥𝑖𝑗 … … … 𝐸𝑞 𝑥𝑞1 𝑥𝑞2 … … … 𝑥𝑞𝑝 𝑊𝐸𝑞 Step 6. Normalize the input-data matrix: After the input data matrix is formulated, step 6 is data normalization. It means that the input data are structured in intervals 0 and 1. It further facilitates the decision-making process. In this method, the normalization technique is applied by Eq. (2) and presented in Table 2. 𝑥𝑖𝑗 ∗ = 𝑥𝑖𝑗 ∑ 𝑥𝑖𝑗 𝑞 𝑖=1 , 𝑖 = 1, 2, … , 𝑞; 𝑗 = 1, 2, … , 𝑝 (2) Table 2 Normalized Input-Data matrix Experts/Criteria 𝐶1 𝐶2 𝐶𝑗 𝐶𝑃 𝐸1 𝑥11 ∗ 𝑥12 ∗ … 𝑥1𝑝 ∗ 𝐸2 𝑥21 ∗ 𝑥22 ∗ … 𝑥2𝑝 ∗ 𝐸𝑖 … … 𝑥𝑖𝑗 ∗ … 𝐸𝑞 𝑥𝑞1 ∗ 𝑥𝑞2 ∗ … 𝑥𝑞𝑝 ∗ Step 7. Multiply each value of the normalized input data matrix by the importance of each expert (expert-weighted matrix): In this step, the normalized input data are multiplied by the experts’ weights obtained in Step 4. It is calculated by Eq. (3) and presented in Table 3. 𝑥 ∗̂ 𝑖𝑗 = 𝑥𝑖𝑗 ∗ · 𝑊𝐸𝑖 , 𝑖 = 1,2, … 𝑞; 𝑗 = 1, 2, … , 𝑝 (3) Table 3 Expert-Weighted Matrix Experts/Criteria 𝐶1 𝐶2 … 𝐶𝑗 … 𝐶𝑃 𝐸1 𝑥 ∗̂ 11 𝑥 ∗̂ 12 … … … 𝑥 ∗̂ 1𝑝 𝐸2 𝑥 ∗̂ 21 𝑥 ∗̂ 22 … … … 𝑥 ∗̂ 2𝑝 𝐸𝑖 … … … 𝑥 ∗̂ 𝑖𝑗 … … 𝐸𝑞 𝑥 ∗̂ 𝑞1 𝑥 ∗̂ 𝑞2 … … … 𝑥 ∗̂ 𝑞𝑝 340 S. BOŠKOVIĆ, S. JOVČIĆ, V. SIMIĆ, L. ŠVADLENKA, M. DOBRODOLAC, N. BACANIN Step 8. Identify the maximum and minimum value of each criterion in the expert- weighted matrix: The primary purpose of this step is to identify the maximum (𝑅𝑗 𝑚𝑎𝑥) and minimum value (𝑅𝑗 𝑚𝑖𝑛) of each criterion by columns. It is calculated by Eqs. (4) and (5) and presented in Table 4. 𝑅𝑗 𝑚𝑎𝑥 = max 𝑖 𝑥 ∗̂ 𝑖𝑗, 𝑗 = 1, 2, … , 𝑝, (4) 𝑅𝑗 𝑚𝑖𝑛 = min 𝑖 𝑥 ∗̂ 𝑖𝑗 , 𝑗 = 1, 2, … , 𝑝. (5) Table 4 Maximum and Minimum Value of Each Criterion in the Expert-Weighted Matrix Experts/Criteria 𝐶1 𝐶2 … 𝐶𝑗 … 𝐶𝑃 𝐸1 𝑥 ∗̂ 11 𝑥 ∗̂ 12 … … … 𝑥 ∗̂ 1𝑝 𝐸2 𝑥 ∗̂ 21 𝑥 ∗̂ 22 … … … 𝑥 ∗̂ 2𝑝 𝐸𝑖 … … … 𝑥 ∗̂ 𝑖𝑗 … … 𝐸𝑞 𝑥 ∗̂ 𝑞1 𝑥 ∗̂ 𝑞2 … … … 𝑥 ∗̂ 𝑞𝑝 𝑅𝑗 𝑚𝑎𝑥 𝑅1 𝑚𝑎𝑥 𝑅2 𝑚𝑎𝑥 … 𝑅𝑗 𝑚𝑎𝑥 … 𝑅 𝑝 𝑚𝑎𝑥 𝑅𝑗 𝑚𝑖𝑛 𝑅1 𝑚𝑖𝑛 𝑅2 𝑚𝑖𝑛 … 𝑅𝑗 𝑚𝑖𝑛 … 𝑅𝑝 𝑚𝑖𝑛 Step 9. Calculate the difference between minimum and maximum values: This step calculates the difference (𝐵𝑗) between the minimum and maximum values from the previous step by applying Eq. (6). 𝐵𝑗 = 𝑅𝑗 𝑚𝑎𝑥 − 𝑅𝑗 𝑚𝑖𝑛 , 𝑗 = 1, 2, … , 𝑝 (6) Step 10. Apply the final ranking formula: This step obtains the criteria importance (𝐿𝑗) by using the final ranking equation, i.e. Eq. (7). 𝐿𝑗 = 𝐵𝑗 ∑ 𝐵𝑗 𝑝 𝑗=1 , 𝑗 = 1, 2, … , 𝑝 (7) 2.1. Checking Reliability Index (RI) When it comes to subjective methods, which in this case is the CIMAS method, an important issue is measuring the reliability of the collected answers by the experts. In the AHP method, the authors proposed a well-known approach to measure the inconsistency rate. In that case, the inconsistency rate should be less than 0.1 to consider the results reliable. In the case of CIMAS, this approach cannot be utilized. In the case of CIMAS, the authors applied another approach [27] for measuring the reliability index (RI). The procedure implies a second round of interviewing the experts, where they do not know the results of the first round regarding the criteria assessed. In the second round, the experts are asked to consider the criteria on a scale from 0 to 100 % and thus determine the percentage importance of each criterion. The experts should be careful that the evaluation for all “n” criteria must give the sum of 100 %. Finally, the results from both rounds are compared and they lead to the conclusion regarding the reliability of the results. If we A New Criteria Importance Assessment (CIMAS) Method in Multi-Criteria Group Decision-Making... 341 denote the answers from the second round of the interview by 𝑃𝑗 and previously obtained CIMAS weights by 𝐿𝑗, then the reliability index (RI) can be calculated by Eq. (8). 𝑅𝐼 = ∑ |𝐿𝑗∗100−𝑃𝑗|𝑛 𝑗=1 100 (8) If the RI is less than 0.1, the results should be considered reliable. Contrary, the experts should repeat the criteria assessment procedure. 3. RESULTS AND DISCUSSION This section demonstrates the applicability of a newly proposed CIMAS method in the case of the criteria assessment for the supplier selection process. Five criteria have been identified from the literature: two Economic (distribution costs [30-33], and on-time distribution [30, 31,34, 35]), one Environmental (air pollution [30, 33, 34, 36, 37]), and two Social (external image and appearance in public [30, 31, 33, 34] and social responsibility rate [30, 31, 33, 34]). The criteria assessment problem is depicted in Fig. 2. Fig. 2 Illustration of the problem The first economic criterion is distribution cost. It is the cost that possible suppliers would request from the company for the distribution process. The second economic criterion is on- time distribution, which means whether the supplier respects the promised timeframes. The third criterion is air pollution which belongs to the environmental pillar. The fourth social criterion is the external image and appearance in public. It presents the general impression of possible suppliers in public, such as their building, transport fleet, logo, behavior toward customers, etc. The fifth and last social criterion is the social responsibility rate. Five experts participated in the criteria assessment. All these experts belong to the logistics field with various years of experience (Table 5). Experts 1 to 3 are the CEOs of the logistics companies in the Czech Republic. Experts 4 and 5 are the CEOs of the middle- sized logistics companies in Slovakia. The names of the companies are not mentioned because of their internal policy. The experts were interviewed by telephone due to the ongoing COVID-19 crisis. They were asked to give the importance of each criterion by following the one-to-ten-point scale, where the highest priority of one criterion is denoted by 10. At the same time, the lowest one is represented by 1. In addition, the experts’ weights 342 S. BOŠKOVIĆ, S. JOVČIĆ, V. SIMIĆ, L. ŠVADLENKA, M. DOBRODOLAC, N. BACANIN are determined based on their experience in the field. Therefore, the data were generated from the experts’ criteria assessments for this case study. According to the CIMAS method, the following results are obtained (Table 5 - Table 7). The criteria rank is depicted in Fig. 3. Table 5 Input Data Matrix Distribution Cost On-time Distribution Air Pollution External Image and Appearance in Public Social Responsibility Rate Expert – Years of Experience Expert- Weights E1 9 6 6 6 4 2 𝑊𝐸1 =0.0909 E2 8 8 7 9 6 4 𝑊𝐸2 =0.1818 E3 10 9 6 7 7 1 𝑊𝐸3 =0.0455 E4 9 6 8 9 8 5 𝑊𝐸4 =0.2273 E5 10 9 6 8 6 10 𝑊𝐸5 =0.4545 ∑=22 Table 6 Normalized input data matrix Distribution Cost On-time Distribution Air Pollution External Image and Appearance in Public Social Responsibility Rate E1 0.1957 0.1579 0.1818 0.1538 0.1290 E2 0.1739 0.2105 0.2121 0.2308 0.1935 E3 0.2174 0.2368 0.1818 0.1795 0.2258 E4 0.1957 0.1579 0.2424 0.2308 0.2581 E5 0.2174 0.2368 0.1818 0.2051 0.1935 The results of the criteria assessment for supplier selection are given in the following ranking order: the highest importance was assigned to the distribution on time, followed by the cost of distribution, external image, and appearance in public, social responsibility rate, and air pollution, respectively. To check the reliability of the obtained results, we interviewed the experts in the second round to collect information about the percentage distribution of criteria importance. The results are shown in Table 8. As it can be concluded, the rate of inconsistency is below 0.1 (RI = 0.06445), which means that reliability is satisfactory. Table 7 The maximum and minimum value of each criterion, its difference obtained final weights. Distribution Cost On-time Distribution Air Pollution External Image and Appearance in Public Social Responsibility Rate E1 0.0178 0.0144 0.0165 0.0140 0.0117 E2 0.0316 0.0383 0.0386 0.0420 0.0352 E3 0.0099 0.0108 0.0083 0.0082 0.0103 E4 0.0445 0.0359 0.0551 0.0524 0.0587 E5 0.0988 0.1077 0.0826 0.0932 0.0880 𝑅𝑗 𝑚𝑎𝑥 0.0988 0.1077 0.0826 0.0932 0.0880 𝑅𝑗 𝑚𝑖𝑛 0.0099 0.0108 0.0083 0.0082 0.0103 𝐵𝑗 0.0889 0.0969 0.0744 0.0851 0.0777 ∑=0.4230 𝐿𝑗 0.2102 0.2291 0.1758 0.2011 0.1837 A New Criteria Importance Assessment (CIMAS) Method in Multi-Criteria Group Decision-Making... 343 Fig. 3 Criteria Ranking Table 8 Calculation of the reliability index (RI) 𝐿𝑗 E1 (%) E2 (%) E3 (%) E4 (%) E5 (%) Average 𝑃𝑗 |𝑊𝑗*100-𝑃𝑗| 𝑅𝐼𝑗 Distribution Cost 0.2102 25 30 15 15 15 20 1.0244473 0.010244 On-Time Distribution 0.2291 30 25 20 20 35 26 3.0944179 0.030944 Air Pollution 0.1758 25 5 15 30 5 16 1.5840832 0.015841 External Image and Appearance in Public 0.2011 15 15 35 25 10 20 0.1139869 0.00114 Social Responsibility Rate 0.1837 15 25 15 5 35 19 0.6280995 0.006281 0.06445 3.1. Sensitivity Analysis In this section, the sensitivity analysis is performed to check the deviations of the CIMAS method. The sensitivity analysis is performed in a way that the most experienced experts were excluded from the model to observe how the rank of criteria will change. In our case, we examined three scenarios. Scenario 1, where the high-experienced expert (Expert 5) was excluded. Scenario 2, where the second most experienced expert (Expert 4) was excluded, and Scenario 3, where an inexperienced expert (Expert 1) was excluded. The results of the sensitivity analysis are presented in Fig. 4. Fig. 4 Sensitivity Analysis The sensitivity analysis results revealed that when the most experienced expert (Expert 5) was excluded from decision-making, there was a significant change in criteria ranking. The other two scenarios had better compatibility with the original state. However, when 0.2102 0.2291 0.1758 0.2011 0.1837 0.0000 0.0500 0.1000 0.1500 0.2000 0.2500 Distribution Cost On-time Distribution Air Pollution External Image and Appearance in Public Social Responsibility Rate 0 5 10 C1 C2 C3 C4 C5 Original Scenario 1 - Expert 5 Excluded Scenario 2 - Expert 4 Excluded Scenario 3 - Expert 1 Excluded 344 S. BOŠKOVIĆ, S. JOVČIĆ, V. SIMIĆ, L. ŠVADLENKA, M. DOBRODOLAC, N. BACANIN the least experienced expert (Expert 1) was excluded, this did not significantly affect the ranking order. It may be concluded that the more experienced experts significantly influence the criteria to rank more than those inexperienced. 3.2. Comparative Analysis with the CRITIC method In this sub-section, the newly proposed CIMAS method was compared to the well-known CRITIC method proposed by Diakoulaki et al. [1]. The CRITIC method, as mentioned at the beginning of the paper, belongs to the objective methods used to obtain the criteria weights. The authors of this paper utilized the same input data (in the CRITIC method) obtained from the five experts, and the results of the comparative analysis are depicted in Fig. 5. It may be noticed from Fig. 5 that both the CRITIC and the CIMAS methods ranked the on-time distribution (C2) criterion with the highest importance. On the other hand, the CRITIC method ranked the Distribution Cost (C1) as the second best, while the CIMAS ranked the C1 as the third best. In the next section, given the noticeable similarity between the two methods, the paper’s authors coupled the newly proposed method with the CRITIC one to obtain hybrid criteria weights that should be utilized further in decision-making. Fig. 5 Comparative analysis between the CRITIC and the CIMAS methods 3.3. Hybrid weights based on the integration of the CRITIC and CIMAS methods In this sub-section, the obtained criteria weights from the CIMAS and CRITIC methods are effectively coupled, and the hybrid criteria weights are obtained. The hybrid criteria weights are calculated by applying the equation of an arithmetic mean: 𝑊𝑗 ℎ𝑦𝑏𝑟𝑖𝑑 = 𝑊𝑗 𝐶𝑅+𝐿𝑗 2 , 𝑗 = 1, 2, … , 𝑝 (9) where: 𝑊𝑗 𝐶𝑅 are the criteria importance obtained by the CRITIC method, while 𝐿𝑗 are the criteria obtained by the CIMAS method. The obtained hybrid criteria weights are presented in Table 9 and Fig. 6. 0.2407 0.2571 0.2171 0.1678 0.1173 0.2102 0.2291 0.1758 0.2011 0.1837 0.0000 0.0500 0.1000 0.1500 0.2000 0.2500 0.3000 Distribution Cost On-time Distribution Air Pollution External Image and Appearance in Public Social Responsibility Rate Lj Wcr A New Criteria Importance Assessment (CIMAS) Method in Multi-Criteria Group Decision-Making... 345 Table 9 Hybrid criteria weights 𝑊𝑗 𝐶𝑅 𝐿𝑗 𝑊𝑗 ℎ𝑦𝑏𝑟𝑖𝑑 Distribution Cost 0.2407 0.2102 0.2255 On-time Distribution 0.2571 0.2291 0.2431 Air Pollution 0.2171 0.1758 0.1965 External Image and Appearance in Public 0.1678 0.2011 0.1844 Social Responsibility Rate 0.1173 0.1837 0.1505 Fig. 6 shows that the obtained hybrid criteria weights (the red line) are in between the CRITIC and newly proposed CIMAS method, which may indicate more accurate results in the decision-making process. This should be seen as a powerful technique that combines objective and subjective criteria weights for better decisions. Fig. 6 Hybrid Criteria Weights 3.3.1. Sensitivity analysis of the integrated model The sensitivity analysis is proposed to prove the robustness of the newly proposed framework with the hybrid criteria weights. The authors introduced the parameter λ (between the intervals 0 and 1) and tested the robustness of the hybrid CRITIC-CIMAS model. The robustness of the model was tested by applying Eq. (10). The results are presented in Fig. 7. 𝑊𝑗 𝐻𝑦𝑏𝑟𝑖𝑑 (𝜆) = 𝜆𝑊𝑗 𝐶𝑅+(1−𝜆)𝐿𝑗 2 , 𝑗 = 1, 2, … , 𝑝 (10) 346 S. BOŠKOVIĆ, S. JOVČIĆ, V. SIMIĆ, L. ŠVADLENKA, M. DOBRODOLAC, N. BACANIN Fig. 7 Sensitivity analysis on the hybrid criteria weights Applying Eq. 10 allows two scenarios where different methods were used. In the first one, when λ=0, only the criteria weights obtained by the CIMAS method ( 𝐿𝑗 2 ) are considered. In the second one, when λ=1, only the criteria weights obtained by the CRITIC method ( 𝑊𝑗 𝐶𝑅 2 ) are considered. Both criteria weights are considered in all other cases, λ = (0.1-0.9). It may be noticed from Fig. 7 that the criteria weights are stable when λ is between 0.4 and 1. In the rest, minimal overlaps are noticed. It can be concluded that the criteria ranks are very stable to variations in the operational parameter λ. 3.4. Managerial Implications The evaluating criteria weights for an effective supplier selection process is an initial phase and crucial to all logistics decision-makers. The criteria weights may be determined differently since many multi-criteria decision-making methods were developed. Sometimes, the criteria weights may be obtained either by objective or subjective methods. To reduce the level of uncertainty and boost more precise final ranking decisions, we developed an integrated criteria importance assessment method based on objective judgment and group decision-making. This integrated approach may serve managers to assess the criteria efficiently and effectively. One of the possibilities of this approach is that the managers may choose either to apply the CIMAS method or the CRITIC one to find the criteria's importance. The recommendation to managers may be to use an integrated approach due to its high level of robustness. Apart from the logistics industry, the integrated approach may be applied by managers to assess the criteria in other branches of industry such as supply chain, medicine, education, automotive, construction, etc. One of the advantages of the integrated approach is its simplicity of implementation, which leads to the expectation that managers worldwide would be able to apply it to solve their decision-making problems. A New Criteria Importance Assessment (CIMAS) Method in Multi-Criteria Group Decision-Making... 347 4. CONCLUSIONS The primary aim of this paper was to propose a novel Criteria Importance Assessment (CIMAS) method to help decision-makers assess the criteria when dealing with complex MCDM problems. As its name suggests, the novel method should be suitable in cases when multiple criteria affect the decision on the best alternative. The main advantage of the proposed method is to help managers and decision-makers identify the weights of each criterion and rank them in descending order. The experts' knowledge and experience are essential to the newly proposed method. The final rank of the criteria exclusively depends on the experts' knowledge and experience, which sometimes could be a disadvantage if the decision is made by an inexperienced expert. The paper’s authors performed the sensitivity analysis to observe how the criteria to rank would change if some experts were excluded from the criteria evaluation (Fig. 4). The sensitivity analysis results revealed that the most experienced expert significantly impacted decision-making if his knowledge and experience had been omitted from decision-making. On the contrary, if the least experienced expert had been excluded, that did not significantly affect the criteria ranking order. The secondary aim of this paper was to integrate the newly proposed CIMAS method within objective one (CRITIC) and thus obtain more accurate hybrid criteria weights. The proposed integrated approach was applied to evaluate the hybrid criteria weights affecting supplier selection. The obtained results within the integrated approach presented the following hybrid ranking order (Table 10, Fig. 6): the highest importance was assigned to the on-time distribution, followed by the cost of distribution, air pollution, external image and appearance in public, and social responsibility, respectively. In addition, the sensitivity analysis was performed to check the stability of the newly integrated approach where the criteria rank presented a high level of robustness (Fig. 7). This paper highlights its significant contributions: i) The original MCDM method, named the Criteria Importance Assessment (CIMAS), is proposed; ii) The CIMAS method belongs to the subjective methods and is presented for the first time in the literature; iii) The method is general (includes the expert-knowledge and experience) and applies to any other multi-criteria decision-making problem in many branches of industry; iv) The application of the CIMAS method is demonstrated in evaluating the criteria which are further used for the supplier selection process. v) Coupling the subjective (CIMAS) with the well-known objective (CRITIC) method, an integrated approach gives a more precise and confident criteria rank which was confirmed by the sensitivity analysis. Besides contributions, there is a limitation of the proposed CIMAS method. The method uses only the crisp values, and its application is not suitable in an uncertain environment. The possible future research directions of this paper should be: 1) to extend the methodology on fuzzy sets; 2) to apply the methodology to many different fields; 3) to couple the CIMAS method with some of the existing MCDM methods to obtain criteria weights; 4) Sustainable supplier selection as a crucial part of the whole supply chain should also be further addressed. In this regard, it is essential to carefully monitor and identify the most relevant criteria, evaluate them as most appropriate according to the chosen alternatives, and obtain the best possible ranking alternative order. Acknowledgements: This research was part of the project „CK01000032 – Sustainable Urban Mobility Plans, E-commerce, and Smart City Logistics “, and SGS_2023_017. 348 S. BOŠKOVIĆ, S. JOVČIĆ, V. SIMIĆ, L. ŠVADLENKA, M. DOBRODOLAC, N. BACANIN REFERENCES 1. Diakoulaki, D., Mavrotas, G., Papayannakis, L., 1995, Determining objective weights in multiple criteria problems: The critic method, Computers & Operations Research, 22(7), pp. 763–770. 2. Pamučar, D., Stević, Ž., Sremac, S., 2018, A new model for determining weight coefficients of criteria in MCDM models: full consistency method (FUCOM), Symmetry, 10(9), 393. 3. Saaty, T.L., 1980, The Analytic Hierarchy Process, McGraw-Hill, New York. 4. Gabus, A., Fontela, E., 1972, World Problems an Invitation to Further Thought within the Framework of DEMATEL, Battelle Geneva Research Centre, Geneva. 5. Rezaei, J., 2015, Best-worst multi-criteria decision-making method, Omega, 53, pp. 49–57. 6. Keršuliene, V., Zavadskas, E.K., Turskis, Z., 2010, Selection of rational dispute resolution method by applying new stepwise weight assessment ratio analysis (SWARA), Journal of Business Economics and Management, 11(2), pp. 243–258. 7. Xie, S., Ren, A., Liu, R., 2009, Site selection of supermarket based on DEMATEL, Proc. 16th International Conference on Industrial Engineering and Engineering Management, Beijing, pp. 255–259. 8. Ortega, J., Sarbast, M., János, T., Tamás, P., Palaguachi, J., Paguay, M., 2020, Using Best worst method for sustainable park and ride facility location, Sustainability, 12(23), 10083. 9. Kant, P., Gupta, S., 2020, Sustainable Urban Freight Strategies for Jaipur City, India. In: Golinska-Dawson, P., Tsai, KM., Kosacka-Olejnik M. (eds) Smart and Sustainable Supply Chain and Logistics – Trends, Challenges, Methods, and Best Practices, EcoProduction (Environmental Issues in Logistics and Manufacturing), Springer, Cham. 10. Sharifi, H., Shahram, H., Milaghardan, A., Esmaeily, A., Mojaradi, B., Naseri, F., 2012, Forest fire hazard modeling using hybrid AHP and fuzzy AHP methods using MODIS sensor, Proc. 2012 IEEE International Geoscience and Remote Sensing Symposium, Munich, pp. 931–934. 11. Lyu, H.M., Zhou, W.H., Shen, S.L., Zhou, A.N., 2020, Inundation risk assessment of metro system using AHP and TFN-AHP in Shenzhen, Sustainable Cities and Society, 56, 102103. 12. Sinuany-Stern, Z., Amitai, A., 1994, Project post-evaluation via AHP, Production Planning and Control, 5(4), pp. 337–341. 13. Singh, R.K., Modgil, S., 2020, Supplier selection using SWARA and WASPAS - A case study of the Indian cement industry, Measuring Business Excellence, 24(2), pp. 243–265. 14. Vrtagić, S., Softić, E., Subotić, M., Stević, Ž., Đordević, M., Ponjavić, M., 2021, Ranking road sections based on MCDM model: new improved fuzzy SWARA (IMF SWARA), Axioms, 10(2), 92. 15. Sivageerthi, T., Bathrinath, S., Uthayakumar, M., Bhalaji, R.K.A., 2022, A SWARA method to analyze the risks in coal supply chain management, Materials Today: Proceedings, 50(5), pp. 935-940. 16. Haghnazar, R.K., Zolfani, S.H., Golabchi, M., 2015, Glasshouse locating based on the SWARA-COPRAS approach, International Journal of Strategic Property Management, 19(2), pp. 111–122. 17. Sofuoğlu, M.A., 2020, Fuzzy applications of FUCOM method in manufacturing environment, Journal of Polytechnic, 23(1), pp. 189-195. 18. Stević, Ž., Brković, N., 2020, A novel integrated FUCOM-MARCOS model for evaluation of human resources in a transport company, Logistics, 4(1), 4. 19. Badi, I., Kridish, M., 2020, Landfill site selection using a novel FUCOM-CODAS model: A case study in Libya, Scientific African, 9, e00537. 20. Huskanović, E., Stević, Ž., Nunić, Z., Chatterjee, P., Tanackov, I., 2021, An integrated decision-making model for efficiency analysis of the forklifts in warehousing systems, Facta Universitatis – Series: Mechanical Engineering, 19(3), pp. 537-553. 21. Bošković, S., Švadlenka, L., Jovčić, S., Dobrodolac, M., Simić V., Bacanin, N., 2023, An alternative ranking order method accounting for two-step normalization (AROMAN) – A case study of the electric vehicle selection problem, IEEE Access, 11, pp. 39496-39507. 22. Bošković, S., Švadlenka, L., Dobrodolac, M., Jovčić, S., Zanne, M., 2023, An extended AROMAN method for cargo bike delivery concept selection, Decision Making Advances, 1(1), pp. 1-9. 23. Yalçın, G.C., Kara, K., Toygar, A., Simic, V., Pamucar, D., Köleoğlu, N., 2023, An intuitionistic fuzzy- based model for performance evaluation of EcoPorts, Engineering Applications of Artificial Intelligence, 126, 107192. 24. Švadlenka, L., Simić, V., Dobrodolac, M., Lazarević, D., Todorović, G., 2020, Picture fuzzy decision- making approach for sustainable last-mile delivery, IEEE Access, 8, pp. 209393-209414. 25. Simic, V., Dabic-Miletic, S., Tirkolaee, E.B., Stević, Ž., Ala, A., Amirteimoori, A., 2023, Neutrosophic LOPCOW-ARAS model for prioritizing industry 4.0-based material handling technologies in smart and sustainable warehouse management systems, Applied Soft Computing, 143, 110400. A New Criteria Importance Assessment (CIMAS) Method in Multi-Criteria Group Decision-Making... 349 26. Tirkolaee, E.B., Torkayesh, A.E., Tavana, M., Goli, A., Simic, V., Ding, W., 2023, An integrated decision support framework for resilient vaccine supply chain network design, Engineering Applications of Artificial Intelligence, 126, 106945. 27. Čubranić-Dobrodolac, M., Jovčić, S., Bošković, S., Babić, D., 2023, A decision-making model for professional drivers selection: A hybridized fuzzy–AROMAN–Fuller approach, Mathematics, 11(13), 2831. 28. Nikolić, I., Milutinović, J., Božanić, D., Dobrodolac, M., 2023, Using an interval type-2 fuzzy AROMAN decision-making method to improve the sustainability of the postal network in rural areas, Mathematics, 11(14), 3105. 29. Pamucar, D., Simic, V., Lazarević, D., Dobrodolac, M., Deveci, M., 2022, Prioritization of sustainable mobility sharing systems using integrated fuzzy DIBR and fuzzy-rough EDAS model, Sustainable Cities and Society, 82, 103910. 30. Zhang, J., Li, L., Zhang, J., Chen, L., Chen, G., 2023, Private-label sustainable supplier selection using a fuzzy entropy-VIKOR-based approach, Complex & Intelligent Systems, 9, pp. 2361–2378. 31. Alavi, B., Tavana, M., Mina, H., 2021, A dynamic decision support system for sustainable supplier selection in circular economy, Sustainable Production and Consumption, 27, pp. 905–920. 32. Rahimi, M., Kumar, P., Moomivand, B., Yari, G., 2021, An intuitionistic fuzzy entropy approach for supplier selection, Complex & Intelligent Systems, 7, pp. 1869–1876. 33. Pınar, A., Babak Daneshvar, R., Özdemir, Y.S., 2021, q-Rung orthopair fuzzy TOPSIS method for green supplier selection problem, Sustainability, 13(2), 985. 34. Yazdani, M., Ebadi Torkayesh, A., Stević, Ž., Chatterjee, P., Asgharieh Ahari, S., Doval Hernandez, V., 2021, An interval-valued neutrosophic decision-making structure for sustainable supplier selection, Expert Systems with Applications, 183, 115354. 35. Liu, G., Fan, S., Tu, Y., Wang, G., 2021, Innovative supplier selection from collaboration perspective with a hybrid MCDM model: A case study based on NEVs manufacturer, Symmetry, 13(1), 143. 36. Hoseini, S.A., Fallahpour, A., Wong, K.Y., Mahdiyar, A., Saberi, M., Durdyev, S., 2021, Sustainable supplier selection in construction industry through hybrid fuzzy-based approaches, Sustainability, 13(3), 1413. 37. Alamroshan, L., Yahyaei, M., 2022, The green-agile supplier selection problem for the medical devices: a hybrid fuzzy decision-making approach, Environmental Science and Pollution Research, 29, pp. 6793–6811.