FACTA UNIVERSITATIS Series: Mechanical Engineering Vol. 19, No 3, Special Issue, 2021, pp. 361 - 380 https://doi.org/10.22190/FUME210214031P © 2021 by University of Niš, Serbia | Creative Commons License: CC BY-NC-ND Original scientific paper A NEW LOGARITHM METHODOLOGY OF ADDITIVE WEIGHTS (LMAW) FOR MULTI-CRITERIA DECISION-MAKING: APPLICATION IN LOGISTICS Dragan Pamučar1, Mališa Žižović2, Sanjib Biswas3, Darko Božanić1 1University of Defense in Belgrade, Military Academy, Belgrade, Serbia 2Faculty of Technical Sciences in Čačak, University of Kragujevac, Serbia 3Decision Sciences and Operations Management Area, Calcutta Business School, India Abstract. Logistics management has been playing a significant role in ensuring competitive growth of industries and nations. This study proposes a new Multi-Criteria Decision-making (MCDM) framework for evaluating operational efficiency of logistics service provider (LSP). We present a case study of comparative analysis of six leading LSPs in India using our proposed framework. We consider three operational metrics such as annual overhead expense (OE), annual fuel consumption (FC) and cost of delay (CoD, two qualitative indicators such as innovativeness (IN) which basically indicates process innovation and average customer rating (CR)and one outcome variable such as turnover (TO) as the criteria for comparative analysis. The result shows that the final ranking is a combined effect of all criteria. However, it is evident that IN largely influences the ranking. We carry out a comparative analysis of the results obtained from our proposed method with that derived by using existing established frameworks. We find that our method provides consistent results; it is more stable and does not suffer from rank reversal problem. Key Words: Logarithm Methodology of Additive Weights (LMAW), Bonferroni Aggregator, Operational Performance, Logistics Service Providers, Rank Reversal, Sensitivity Analysis 1. INTRODUCTION Logistics management (LM) encompasses an uninterrupted flow of materials, services, and information related to the movement through seamless integration of all stages of the supply chain connecting the points of source and use [1]. The broader spectrum of LM includes various activities like material handling and storing, inventory Received February 14, 2021 / Accepted March 18, 2021 Corresponding author: Dragan Pamučar University of Defence in Belgrade, Military Academy, Pavla Jurišića Šturma 33, Belgrade, Serbia E-mail: dpamucar@gmail.com 362 D. PAMUČAR, M. ŽIĆOVIĆ, S. BISWAS, D. BOŽANIĆ optimization and management, network planning, transportation arrangement, order processing, distribution planning, channel management, and management of returns [2]. In this era of globalization, LM bridges the interrelated and interdependent supply chains of different partnering organizations and industries spreading over a wide geographical region. LM enables the industries to consolidate their resources for optimization of cost, generate supply chain surplus and offer utmost service quality to the customers [3]. A country’s competitive growth especially for the developing nations like India is significantly contributed by LM activities. According to a recent market research [4], organizations across the globe are increasingly focusing on creating a global production base which largely depends on effective LM. India as a fastest growing economy in the south-east Asia with surpassing demographic dividend and tremendous market size and variety, is significantly positioned as a potential driver of global operations in the coming decades. An effective LM planning and execution can bolster the ambitious initiatives like “Make-in-India” led by the government of India (GOI). A very recent report [5] has estimated a CAGR of 10.5% from 2019 to 2025 for the logistics sector in India which shall draw a notable foreign direct investment (FDI) and cash inflow to the country. Hence, it is quite imperative to mention that logistics is under the spotlight from industrial and country’s growth perspective and as a result, a lot of research works are being conducted by the practitioners and scholars on LM. In this context, logistics service providers (LSP) play a crucial role. In this era of extreme competitions, the organizations are putting more emphasis on strengthening their core competencies for improving performance, reducing operational costs, and capital investments, optimally utilizing resources, and, finally, providing better quality products and services to the customers, thereby increasing return on investment for the shareholders [6-7]. Hence, the importance of LSPs has been increased in the last two decades. Most of the organizations outsource their LM activities to the LSPs. However, as LSPs have become strategic partners to the firms, selection of an appropriate vendor is of paramount importance to the supply chain managers. Selection of a LSP is a complex task that depends on multiple aspects (both subjective and objective) which quite often are conflicting in nature [8]. There have been a sizeable number of research contributions towards developing a measurement framework for assessing LM performance of the service providers. Some of the parameters that are mentioned in extant literature include order fulfillment, on time delivery, faster response, reduction in lead time, improved service quality for customer delight, flexibility and adaptability, convenience, sharing of information, seamless coordination and cooperation, optimization of operational cost, innovativeness, adoption of new technologies, reputation building, and the ability to withstand uncertainties [9-19]. It is evident from the discussions and observations on the past work that the comparative performance assessment of the LSPs is a MCDM issue. For solving real-life complex problems, the decision-makers (DM) are confronted with the requirement of consistent decision-making through rational evaluation of the possible alternatives subject to the influence of conflicting criteria [20]. MCDM frameworks enable the DMs to evaluate available possibilities under the effect of different criteria in a structured and cost effective way with reasonable precision and accuracy to arrive at an acceptable solution [21-22]. As a result, MCDM techniques are frequently used by the researchers and DMs for solving variety of complex problems, for example, related to facility location selection [23], supply chain performance [24-26], investment decision-making A New Logarithm Methodology of Additive Weights (LMAW) for Multi-Criteria Decision-Making... 363 [27]. Over the years researchers have developed various MCDM methods which are dissimilar in nature. The features that differentiate various MCDM methods are formulation of decision matrix, choice of normalization, functionality and applications, type of information (subjective and objective) and computational algorithms. As a result, the selection of an appropriate MCDM technique for solving a given problem is essential to find out optimum solution [28]. The literature is rife with a significant number of valuable contributions by several researchers pertaining to the MCDM domain. The evolution of the stated field has been supported by several algorithms. Some of the popularly used MCDM frameworks are Simple Additive Weighting (SAW) [29], Elimination Et Choice Translating Reality (ELECTRE) [30], Analytical Hierarchy Process (AHP) [31], Multi- criteria Optimization and Compromise Solution (Serbian: Više Kriterijumska optimizacija i Kompromisno Rešenje (VIKOR)) [32-33], Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) [34], Preference Ranking Organization Method for Enrichment Evaluation (PROMETHEE) [35-36], multi-attribute utility function based MCDM [37], Complex Proportional Assessment (COPRAS) [38], Analytic Network Process (ANP) [39], Multi-objective Optimization by Ratio Analysis (MOORA) [40], and its subsequent extension (with full multiplicative form) known as MULTIMOORA [41], Additive Ratio Assessment (ARAS) [42], Step‐wise Weight Assessment Ratio Analysis (SWARA) [43], Multi-objective Optimization on the basis of Simple Ratio Analysis (MOOSRA) [44], Weighted Aggregated Sum Product Assessment (WASPAS) [45], KEmeny Median Indicator Ranks Accordance (KEMIRA) [46], Multi-Attributive Border Approximation Area Comparison (MABAC) [47], Evaluation based on Distance from Average Solution (EDAS) [48], Combinative Distance-based Assessment (CODAS) [49], Pivot Pairwise Relative Criteria Importance Assessment (PIPRECIA) [50], Full consistency method (FUCOM) [51], Combined Compromise Solution (CoCoSo) [52], Level Based Weight Assessment (LBWA) [53], Measurement of Alternatives and Ranking according to COmpromise Solution (MARCOS) [54], and Ranking of Alternatives through Functional mapping of criterion sub-intervals into a Single Interval (RAFSI) [55]. In this paper, we introduce a new MCDM algorithm such as LMAW. The LMAW method presents a new multi-criteria decision-making framework that has a methodology for determining the weight coefficients of the criteria. The LMAW method showed greater stability compared to the TOPSIS method, which is based on similar principles, respectively, the definition of the distance of alternatives in relation to reference points. Compared to the TOPSIS method, the LMAW method showed robustness of results when changing the number of alternatives in the initial decision-making matrix. The TOPSIS model showed that eliminating the worst alternatives from the decision-making matrix led to the change in the existing rank, respectively, to the occurrence of the rank reversal problem. On the other hand, the LMAW method did not cause rank reversal problems. Thus, the LMAW method showed significant stability and reliability of results in a dynamic environment. It is also important to note that in numerous simulations the LMAW method showed stability when processing larger data sets. This was confirmed also by the case study discussed in this paper. In addition to the above mentioned, the following advantages of the LMAW method can be highlighted: (1) mathematical framework of the method remains the same regardless of the number of alternatives and criteria; (2) a possibility of application in the case studies considering a number of alternatives and criteria; (3) a clearly defined range of alternatives expressed in numerical values, which makes it easier to understand the results; and 364 D. PAMUČAR, M. ŽIĆOVIĆ, S. BISWAS, D. BOŽANIĆ (4) the presented methodology allows the evaluation of alternatives expressed by either qualitative or quantitative types of criteria. The rest of the paper is structured as follows. In Section 2, we summarize some of the related work in the field of performance evaluation of LSPs. In Section 3, we elucidate the new methodology and define the computational steps. Section 4 presents the case study of comparative evaluation of logistics service providers in the Indian context wherein we apply the new methodology. Section 5 exhibits the analysis and findings related to validation and sensitivity analysis of the proposed model. Finally, Section 6 concludes the paper while highlighting some of the implications of this research and future scope. 2. LITERATURE REVIEW We notice that several MCDM techniques are applied for comparative performance analysis of the LSPs in umpteen occasions. For instance, in [56] ANP was applied for selection of LSP from growth perspective for a medium-scale FMCG organization. A combination of ANP and TOPSIS was considered in the work of [57]. Some researchers (for example, [11]) have considered qualitative information and applied Delphi method in conjunction with ANP. Optimization is also given due consideration by the contributors. As example, data envelopment analysis (DEA) was used in the work of [58] while in [59], a combination of AHP and goal programming (GP) was applied. Bajec and Tuljak- Suban [19] used a combination of AHP and DEA to solve LSP selection problem. However, Andrejić [60] mentioned the difficulty of precise assessment of logistics performance due to the presence of many conflicting aspects. It is evident from the literature that researchers put due diligence to the issue of impreciseness. We find that a good number of works have been carried out in uncertain environment. In this regard, we observe three strands of literature: the first one applied fuzzy concepts; the second one worked with rough numbers and the final one used grey theory based models. Apart from these, some contributions included a combined approach also. The study of [61] used an integrated fuzzy AHP and integer GP while the authors [62] relied on a combined fuzzy AHP-TOPSIS framework. On a different note, we observe that in [63] logic and rule based reasoning, and compromise solution based algorithms were used for the comparative analysis. In this category, Liu and Wang [64] put forth an integrated Delphi, inference system and linear assignment based framework for solving the LSP selection problem. Causal MCDM techniques like Interpretive Structural Modeling (ISM) have also been used to delve into the interrelationship among the criteria along with the outranking algorithm like fuzzy TOPSIS for the selection of suitable third party LSP (3PL) for the return channel for a battery manufacturer [65]. Akman and Baynal [66] conducted the research on selection of 3PL for a tire manufacturing unit using fuzzy AHP- TOPSIS model. For selecting a reverse logistics partner, Prakash and Barua [67] took help of fuzzy AHP and VIKOR while in a recent work, Li et al. [17] introduced the concept of the prospect theory and applied fuzzy TOPSIS. In the work of [16], we observe that an expert decision-making framework has been used wherein the authors used fuzzy SWARA and COPRAS approach. The combination of fuzzy AHP-TOPSIS is seen as a popular framework [18]. However, some contributors (e.g., [68]) have also considered the degree of indeterminacy and carried out a more granular analysis using hesitant and intuitionist fuzzy sets. For enhancing clarity and preciseness in analysis, the concept of rough numbers has also been used significantly. For instance, Sremac et al. A New Logarithm Methodology of Additive Weights (LMAW) for Multi-Criteria Decision-Making... 365 [15] used rough SWARA–WASPAS model while Pamucar et al. [69] applied interval rough number based Best Worst Method (BWM)-WASPAS-MABAC framework for ranking of 3PLs. Nevertheless, in some cases fuzziness cannot be determined realistically (e.g., opinion based analysis when varying levels of measurement and considerable amount of information is not available explicitly or information loss is present) [70]. Under those circumstances, the Grey Theory [71-72] has been considered by many scholars while applying MCDM models. For instance, in [14] a grey forecasting based analysis was carried out. Mercangoz et al. [73] devised a grey based COPRAS scheme for evaluating competitiveness of LM performance of European Union (EU) member states. 3. NEW MCDM FRAMEWORK: LOGARITHM METHODOLOGY OF ADDITIVE WEIGHTS (LMAW) In the following section, the new Logarithm Methodology of Additive Weights (LMAW) is presented as implemented through six steps: Step 1: Forming initial decision-making matrix (X). In the first step, it is performed the evaluation of m alternatives A = {A1,A2,...,Am} compared to n criteria C = {C1,C2,...,Cn}. The weight coefficients of criteria wj (j = 1,2,...,n) are defined also meeting the condition where 1 1 n j j w = = . It is assumed that the evaluation of the alternatives is performed by k experts E = {E1,E2,...,Ek} based on a predefined linguistic scale. Then, for every expert what is obtained is matrix [ ] e e ij m n X   = (1  e  k), where e ij  presents the value from the defined linguistic scale. Applying Bonferroni aggregator through the expression (1), aggregated initial decision-making matrix X = [ij]mn is obtained: 1 ( ) ( ) 1 1 1 ( ) ( ) ( 1) p q k k x p y q ij ij ij x y y x k k    + = =      =  −      (1) where ij presents the averaged values obtained by applying Bonferroni aggregator (1); p,q  0 present stabilization parameters of the Bonferroni aggregator, while e presents the e-th expert 1  e  k. Step 2: Standardization of the initial decision-making matrix elements. Standardized matrix 11[ ]m n  = is obtained by applying the expression (2). is Benefit, is Cost. ij j ij j j ij ij j ij j ij if C if C          + + −  + =  =  + =  (2) where max( )j ij i   + = , min( )j iji   − = , while ij presents the standardized values of the initial decision-making matrix. 366 D. PAMUČAR, M. ŽIĆOVIĆ, S. BISWAS, D. BOŽANIĆ Step 3: Determining weight coefficients of the criteria. The experts from the group E = {E1,E2,...,Ek} prioritize criteria C = {C1,C2,...,Cn} based on the value from the predefined linguistic scale. Prioritizing is performed by adding a higher value from the linguistic scale to the criterion with higher significance, while adding a lower value from the linguistic scale to the criterion with lower significance. In this way what is obtained is priority vector 1 2 ( , ,.., ) e e e e C C Cn P   = , where e Cn  presents the value from the linguistic scale assigned by expert e (1  e  k) to criterion t C (1  t  n). Step 3.1: Defining absolute anti-ideal point ( AIP ). Absolute anti-ideal point is defined in relation to the minimum values from the priority vector and should be lower than the smallest value from the priority vector. We can define  AIP value as  AIP =  e min / s, where min 1 2 min{ , ,..., } e e e e C C Cn    = , and s is a number greater than the base of logarithm (A). If we take ln as a logarithmic function, then s = 3. Step 3.2: Applying the expression (3), the relation is determined between the elements of the priority vector and absolute anti-ideal point ( AIP ). e e Cn Cn AIP    = (3) Thus we obtain relation vector 1 2 ( , ,.., ) e e e e C C Cn R   = , where e Cn  presents the value from the relation vector which is obtained by applying the expression (3), while Re presents the relation vector of expert e (1  e  k). Step 3.3: Determining the vector of weight coefficients wj = (w1, w2,...,wn) T. Applying the expression (4), the values of the weight coefficients of the criteria are obtained for expert e (1  e  k): log ( ) , 1 log ( ) e e A Cn j e A w A b  =  (4) where e Cn  presents the elements of relation vector R, while 1 n b e Cn j b  = =  . Such obtained values of the weight coefficients meet the condition where 1 1 n e jj w = = . Applying Bonferroni aggregator as in the expression (5), we obtain the aggregated vector of weight coefficients wj = (w1, w2,...,wn) T. 1 ( ) ( ) 1 1 1 ( ) ( ) ( 1) p q k k x p y q j j j x y y x w w w k k + = =      =  −      (5) where p,q  0 present stabilization parameters of Bonferroni aggregator, while e j w presents the weight coefficients obtained based on the evaluations of the e-th expert 1  e  k. Step 4: Calculation of weighted matrix (N). The elements of weighted matrix [ ] ij m n N   = are obtained by applying the expression (6): A New Logarithm Methodology of Additive Weights (LMAW) for Multi-Criteria Decision-Making... 367 2 (2 ) j j j w ij ij w w ij ij     = − + (6) where 1 ln( ) ln ij ij m ij i    = =        (7) while ij presents the elements of standardized matrix 11[ ]m n  = , while wj presents the weight coefficients of the criteria. Step 5: Calculation of the final index for ranking alternatives (Qi). The rank of alternatives is defined based on value Qi. The preferable alternative is with as high as possible value of Qi. 1 n i ij j Q  = =  (8) where ij  presents the elements of weighted matrix [ ] ij m n N   = . 4. COMPARISON OF PERFORMANCE OF SELECTED LOGISTICS SERVICE PROVIDERS IN INDIA 4.1. The Case Study In our case study we consider six large scale multimodal integrated supply chain and logistics service providers in India providing the services like FTL (Full Truck Load), LTL (Less than Truckload), PHH (Project & Heavy Haul), and Rail (for different organizations), people transport, CFS (container freight stations), and warehousing. All these LSPs are having all India presence. Many of them operate worldwide including neighboring countries. These service providers are significantly old. For confidentiality of information, their names are not disclosed in this paper. Let us code the names of these LSPs as A1, A2, … A6. Our objective is to carry out a comparative analysis of their performances using both objective operational metrics and subjective factors. The following table (see Table 1) lists the criteria considered for the comparative analysis. Table 1 Criteria for evaluation of alternatives Criteria Code UOM Effect Direction Turnover (TO) C1 Rs. Cr. (+) Innovativeness (IN) C2 Scale Value (+) Annual Overhead Expenses (OE) C3 Rs. Cr. (-) Annual Fuel Consumption (FC) C4 1000 Lit (-) Cost of Delay (CoD) C5 Rs./Hr. (-) Average Customer Rating (CR) C6 Scale Value (+) 368 D. PAMUČAR, M. ŽIĆOVIĆ, S. BISWAS, D. BOŽANIĆ Here, we consider six criteria. The first criterion (TO) signifies business growth on the basis of revenue generated by providing services to the customers. In other words, it is a proxy measure of customer satisfaction. The growth prospect is not a single day affair. The firm needs to stay agile, flexible, adaptable to changes, and responsive. Most importantly, organizations need to anticipate the changing scenario and customer requirements and be capable to promise service. Therefore, organizations need to be innovative in terms of meeting the changing requirements as well as staying cost effective for providing services at an affordable price. Hence, the second criterion (IN) is of notable importance to the LSPs. Next, we consider criteria related to operational cost (C3 and C4). On time delivery and speed of operation are mandate for the success for the LSPs. Therefore, we include the fifth criterion (CoD). Finally, perception of performance among the customers plays a significant role in retaining existing and/or attracting new business opportunities. Hence, customer rating (CR) is an important aspect that we, with due consideration, include in our analysis (C6). As evident, criteria C1, C3, C4 and C5 represent quantitative criteria, while criteria C2 and C6 belong to the group of qualitative criteria. In order to describe the quantitative group of criteria (C1, C3, C4 and C5) we have used the real indicators collected during the research, while the qualitative group of criteria (C2 and C6) is presented on the basis of expert preferences. A seven-point scale was used to present expert preferences: 1 - Absolutely low (AL), 2 - Very low (VL), 3 - Low (L), 4 - Medium (M), 5 - Medium high (MH), 6 - High (H) and 7 - Very high (VH). 4.2. Results The evaluation of alternatives was performed by applying new Logarithm Methodology of Additive Weights (LMAW) which was implemented through six steps presented in the next section. Step 1: The evaluation of alternatives was performed in relation to the six criteria presented in Table 2. Since criteria C2 and C6 present qualitative criteria, four experts evaluated the alternatives in relation to criteria C2 and C6. Research-based unique values are defined for quantitative criteria. Applying Bonferroni aggregator from the expression (1), the values of the qualitative criteria are aggregated; thus we obtain the initial decision matrix: Table 2 Decision Matrix 1 2 3 4 5 6 1 647.34 6.24 49.87 19.46 212.58 6.75 2 115.64 3.24 16.26 9.69 207.59 3.00 3 373.61 5.00 26.43 12.00 184.62 3.74 4 37.63 2.48 2.85 9.35 142.50 3.24 5 858.01 4.74 62. 6 C C C C C C A A A A A X A = 8 5 45.96 267.95 4.00 222.92 3.00 19.24 21.46 221.38 3.4 9 max max min min min max                    A New Logarithm Methodology of Additive Weights (LMAW) for Multi-Criteria Decision-Making... 369 The values at the position A6-C6 are obtained by averaging expert preferences 1 66 3, = 2 66 4, = 3 66 4 = and 4 66 3 = . Applying Bonferroni aggregator, as in the expression (1), we obtain the averaged value:   1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 66 1 1 1 1 1 1 1 1 1 1 , 1 3 4 3 4 3 3 4 3 4 4 4 31 3, 4, 4, 3 3.49 4(4 1) ... 4 4 4 3 3 3 3 4 3 4 p q  + =    +  +  +  +  +  = = =    − + +  +  +  +  +    The remaining values of the qualitative criteria in matrix X are obtained in a similar way. Step 2: Applying the expression (2), we perform the standardization of the elements of initial decision matrix X ; hence we obtain the standardized matrix, Table 3: Table 3 Standardized Decision Matrix 1 2 3 4 5 6 1 1.75 2.00 1.06 1.48 1.67 2.00 2 1.13 1.52 1.18 1.96 1.69 1.44 3 1.44 1.80 1.11 1.78 1.77 1.55 4 1.04 1.40 2.00 2.00 2.00 1.48 5 2.00 1.76 1.05 1.20 1.53 1.59 6 1.26 1.48 1.15 1.44 1.64 1 C C C C C C A A X A A A A = . 52 max max min min min max                    The values at the positions A1-C1 are obtained by applying the expression (2) as follows: 11 11 647.34 1.75 858.01 858.01 j j     + + + + = = = where the values are 1 1 647.34, 115.64, 373.61, 858.01 1 max( ) max 37.63, 858.0 , 222.92 i i   +   = = =    The remaining elements of the standardized matrix are obtained in a similar way. Step 3: In the following section are calculated the values of the weight coefficients of the criteria. Four experts prioritized the criteria based on the following scale: 1 - Absolutely low (AL), 1.5 - Very low (VL), 2 - Low (L), 2.5 - Medium (M), 3 - Equal (E), 3.5 - Medium high (MH), 4 - High (H), 4.5 - Very high (VH) and 5 - Absolutely high (AH). Considering that the evaluation is performed by four experts, four priority vectors are defined: 370 D. PAMUČAR, M. ŽIĆOVIĆ, S. BISWAS, D. BOŽANIĆ 1 2 3 4 ( ), ( ), ( , , , , , ), ( , , , , , ). , , , , , , , , , , P P P H L L VL H L ML VL MH AH EH VL M E E AH E P A L A H H VL L L M L H H = = = = Step 3.1: Absolute anti-ideal point AIP  is arbitrary defined as value 0.5 AIP  = . Step 3.2: Based on the data from the expert priority vectors and AIP = 0.5, by applying the expression (3), the relation is determined between the elements of the priority vector and absolute anti-ideal point (AIP). In the following section the relations are presented between the elements of the priority vector and the AIP: 1 2 3 4 8, 4, 5, 3, 7, 10 9, 3, 5, 2, 6, 9 8, 4, 4, 3, 6, 10 8, 3, 4, 2, ( ), ( ), ( . 7, 8 ), ( ) R R R R = = = = The elements of vector 1 R are obtained by applying the expression (3) as follows: 1 1 4 8 0.5 C  = = , 1 2 2 4 0.5 C  = = , 1 3 2.5 5 0.5 C  = = , 1 4 1.5 3 0.5 C  = = , 1 5 3.5 7 0.5 C  = = and 1 6 5 10 0.5 C  = = . The elements of remaining vectors R2, R3 and R4 are obtained in a similar way. Step 3.3: Applying the expression (4), the values of the weight coefficients of the criteria by experts are obtained: 1 2 3 4 ( ), ( ), ( ), ( ). 0.200, 0.133, 0.154, 0.105, 0.187, 0.221 0.229, 0.115, 0.168, 0.072, 0.187, 0.229 0.207, 0.138, 0.138, 0.109, 0.178, 0.229 0.224, 0.118, 0.149, 0.075, 0.21, 0.224 j j j j w w w w = = = = The elements of vector 1 j w of the first expert are obtained by applying the expression (4) as follows: 1 1 ln(8) 0.200 ln(33600) w = = , 1 2 ln(4) 0.133 ln(33600) w = = , 1 3 ln(5) 0.154 ln(33600) w = = , 1 4 ln(3) 0.105 ln(33600) w = = , 1 5 ln(7) 0.187 ln(33600) w = = , 1 6 ln(10) 0.221 ln(33600) w = = . where 1 8 4 5 3 7 10 33600b =      = . A New Logarithm Methodology of Additive Weights (LMAW) for Multi-Criteria Decision-Making... 371 The obtained values of the weight coefficients meet the condition where 6 1 1 1 jj w = = . The elements of remaining vectors w2 j , w3 j and w4 j are obtained in a similar way. Applying the expression (5), we obtain the aggregated vector of the weight coefficients 0.215, 0.126, 0.152, 0.09, 0.19, 0.22( )6 T j w = . The value of weight coefficient 1 0.215w = is obtained by averaging values we j (1  e  4) for every expert, respectively, by averaging values w1 j = 0.200, w 2 j = 0.229, w3 j = 0.207 and w4 j = 0.224. Applying the expression (5), we obtain the averaged value: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 , 1 1 {0.200, 0.229, 0.207, 0.224} 0.200 0.229 0.200 0.207 0.200 0.224 ...1 0.215 4(4 1) 0.207 0.224 0.224 0.200 0.224 0.2229 0.224 0.207 p q w + = = =    +  +  + + = =    −  +  +  +    The remaining values of the weight coefficients vectors are obtained in a similar way. Step 4: Applying the expression (6), the elements of weighted matrix (N) are calculated, Table 4: Table 4 Weighted Matrix 1 2 3 4 5 6 1 0.81 0.87 0.72 0.88 0.77 0.78 2 0.65 0.84 0.80 0.91 0.77 0.71 3 0.76 0.86 0.77 0.90 0.78 0.73 4 0.55 0.82 0.93 0.91 0.80 0.72 5 0.83 0.86 0.71 0.85 0.75 0. 74 6 0.71 0.83 0.7 9 0.88 0.77 = C C C C C C A A A A A X A 0.72                    The values at the positions A1-C1 are obtained by applying the expression (6) as follows: 1 1 1 0.215 11 11 0.215 0.215 11 11 2 2 0.28 0.81 (2 0.28) 0.28(2 ) w w w      = = = − +− + where value 11 presents additive normalized weight of the elements of the normalized decision-making matrix at the positions A1-C1, while w1 presents the weight coefficient of criterion C1. Additive normalized weight of elements A1-C1 is calculated as follows: 11 11 1 1 ln( ) ln(1.75) 0.28 ln(7.52) ln( ) m i i    = = = =  where 1 1 1.75 1.13 1.44 1.04 2.00 1.26 7.52 m i i  =     ==  . The remaining weighted decision- making matrices are obtained in a similar way. 372 D. PAMUČAR, M. ŽIĆOVIĆ, S. BISWAS, D. BOŽANIĆ Step 5: Applying the expression (8), the final indices of alternatives are calculated based on which is performed the ranking of alternatives: 1 4.840 2 4.681 3 4.799 4 4.733 5 4.736 6 4.704 i Q A A A A A A           =          Since it is preferable for the alternative to have as high as possible value of i Q , we can define the rank: A1>A3>A5>A4>A6>A2. 5. VALIDATION AND DISCUSSION OF THE RESULTS 5.1. Comparison of the results with other multi-criteria techniques In the following section, the comparison of the results of the LMAW method with other traditional multi-criteria techniques is presented. The comparison is made with the TOPSIS [34], VIKOR (multi-criteria compromise ranking) [32-33], RAFSI [55], COPRAS (COmplex PRoportional ASsessment) [74], and MABAC [47] multi-criteria models. All multi-criteria techniques are applied to the same initial data from the initial decision-making matrix and with the same values of the criteria weights. Numerous studies showed that the application of different models for data normalization could influence the change of the ranking results [75-79]; thus in this analysis are selected the multi-criteria methods, which apply different ways of data normalization. The results of the application of the mentioned methods are presented in Fig. 1. A1 A2 A3 A4 A5 A6 1 2 3 4 5 6 Alternatives R a n k COPRAS TOPSIS LMAW MABAC VIKOR RAFSI Fig. 1 Comparison of the LMAW method with other multi-criteria methods A New Logarithm Methodology of Additive Weights (LMAW) for Multi-Criteria Decision-Making... 373 The VIKOR, TOPSIS and RAFSI methods confirmed the ranks of the LMAW method, with a high correlation, as the Spearman’s coefficient (SCC) for all three methods amounted to 0.943. In the MABAC method there was a slightly lower correlation, compared to the VIKOR, TOPSIS and RAFSI methods, in which the SCC = 0.886. The lowest correlation of results appeared in the COPRAS method where the SCC = 0.714. However, all models confirmed the rank of the first-ranked alternative A1, and the last two ranked alternatives {A2, A6}. For the remaining three alternatives, A3, A4 and A5, different ranks were proposed, with the greatest similarity in the ranks of the MABAC, VIKOR, RAFSI and LMAW methods. The largest deviations in the ranges of alternatives A3, A4 and A5 occurred in the COPRAS and TOPSIS methods. Such a result was the consequence of the application of different data normalization methods, respectively, vector normalization (TOPSIS) and additive normalization (COPRAS). In order to confirm this fact, an experiment was performed in which the same way of data normalization as in the LMAW model was applied in both COPRAS and TOPSIS models. At the same time, the rest of the algorithm of the COPRAS and the TOPSIS model remained unchanged. After changing the way of normalization, identical ranks were obtained in all models. Based on the presented results, we can conclude and confirm robustness of the LMAW model as well as that the LMAW model provided credible and reliable results. 5.2. Rank reversal problem Robust multi-criteria models provide stable solutions in the conditions of changing the number of alternatives, respectively, by introducing new alternatives to the set or by eliminating bad alternatives from the set. In such conditions, the model is not expected to show logical contradictions that may appear in the form of unwanted changes in the ranks of alternatives. If such anomalies occur, then reasonable fear can be expressed indicating a problem with the mathematical apparatus of the applied method. Rank reversal problem (RRP) is one of the most significant problems in multi-criteria decision-making that can lead to illogical and controversial decisions [80]. Significant attention has been paid to the research of the RRP in the literature [55, 75-76]. Therefore, the resistance of the LMAW model to the RRP is analyzed in the following section. The experiment was conducted through five scenarios. In every scenario, one of the worst alternatives from the set of considered alternatives was eliminated and the influence of the change in the number of alternatives on the change of ranks and criteria functions of the alternatives was analyzed. The ranks of the alternatives are presented through five scenarios in Table 5. Table 5 Ranking of alternatives by scenarios - LMAW model Alt. S0 S1 S2 S3 S4 S5 A1 1 1 1 1 1 1 A3 2 2 2 2 2 A5 3 3 3 3 A4 4 4 4 A6 5 5 A2 6 374 D. PAMUČAR, M. ŽIĆOVIĆ, S. BISWAS, D. BOŽANIĆ It can be clearly noted from Table 5 that the LMAW model provides valid results in a dynamic environment. At the same time, the MABAC, VIKOR, RAFSI, COPRAS and TOPSIS models were applied in the same experiment. The results showed that MABAC, VIKOR, RAFSI and COPRAS models provided stable results, while the RRP appeared in the TOPSIS method. The results of the TOPSIS method application are shown in Table 6. The TOPSIS, VIKOR and COPRAS models were used under the same conditions. All three models showed stability and resistance to rank changes. However, in all four models, the values of the criteria functions changed through the scenarios. Accordingly, it can be concluded that for other values in the initial decision-making matrix, in the TOPSIS, VIKOR and COPRAS models, changes in ranks can be expected, which is analyzed in the second experiment presented in the next section of the paper. Based on the presented analysis, it can be summarized that there is a rank reversal problem in the TOPSIS model, which can lead to the appearance of illogical results in the conditions of variable input parameters in the initial decision-making matrix. At the same time, it can be concluded that the MABAC, VIKOR, RAFSI, COPRAS and LMAW models show resistance to the rank reversal problem in the presented experiment. From this analysis it can be concluded that the LMAW model contributes to a realistic and stable assessment of alternatives in solving real world problems. Table 6 Ranks of alternatives by scenarios - TOPSIS model Alt. S0 S1 S2 S3 S4 S5 A1 1 1 1 1 1 1 A5 2 2 3 3 A3 3 3 2 2 2 A4 4 4 4 A2 5 5 A6 6 5.3. Influence of changing parameters p and q on ranking results Mathematical formulation of the Bonferroni function clearly indicates that the change in the values of parameters p and q affects the change in the aggregated values [81], and thus the change in the final values of the indices of alternatives of the LMAW model. Therefore, in order to validate the results, in the following section is analyzed the impact of changes in parameters p, q on the ranking results. The analysis of the change in the value of parameters p and q was performed through a total of 300 scenarios during which the change of parameters p and q in the interval was simulated. The limit for variation of the values of parameters p and q were the values of p = 300 and q = 3000. Based on a large number of simulations of the values of parameters p and q, it was noticed that for the values of parameters over 300 there were no significant changes in the ranks of alternatives. The results of the influence of parameters p and q on the ranking results are shown in Fig. 2. As the values of parameters p and q increase, the Bonferroni function becomes more complex since several relations between the criteria are considered at the same time. The decision makers choose the values of these two parameters according to their preferences. When making decisions in real conditions and in real time, it is recommended for the value of both parameters to be p = q = 1. This simplifies the decision-making process and A New Logarithm Methodology of Additive Weights (LMAW) for Multi-Criteria Decision-Making... 375 at the same time allows the consideration of internal relations between attributes. Fig. 2 shows that when parameters p and q have different values, the score function changes, but these changes do not cause any changes in the ranks of the alternatives. This confirmed that there was sufficient mutual advantage between the alternatives just as it confirmed the initial ranking. 0 50 100 150 200 250 300 4.55 4.6 4.65 4.7 4.75 4.8 4.85 Scenarios: 1 ≤ p,q ≤ 300 S c o re f u n c ti o n o f L M A W m o d e l A1 A2 A3 A4 A5 A6 Fig. 2 Influence of parameters p and q on the ranking results 5.4. Influence of changing criteria weights on the ranking results The next section presents the analysis of the influence of the change of the most significant criterion (C6) on the ranking results. The change of the weight coefficient of criterion C6 through 50 scenarios was simulated. The scenarios were made based on the proportion: * * 6 6 : (1 ) : (1 ) n n w w w w− = − (1) where w*6 presents corrected value of the weight coefficient of criterion C6, w * n presents reduced value of the considered criterion, wn presents original value of the considered criterion and w6 presents original value of criterion C6. In the first scenario, the value of criterion C6 is reduced by 1%, while the values of the remaining criteria were proportionally corrected applying the shown proportion. In every subsequent scenario, the value of criterion C6 was corrected by 2%, while correcting, at the same time, the value of the remaining criteria. Thus, 50 new vectors of weight coefficients were obtained, as in Fig. 3. Once the new vectors of the weight coefficients of the criteria (Fig. 3) were formed, the values of the indices of the alternatives of the LMAW model were obtained, as in Fig. 4. It can be observed from Fig. 4 that the change in the value of criterion C6 affects the change in the index value of the LMAW model alternatives. In the scenarios S1-S40, the initial rank of alternatives A1>A3>A5>A4>A6>A2 was retained. In the scenarios S40- 376 D. PAMUČAR, M. ŽIĆOVIĆ, S. BISWAS, D. BOŽANIĆ S50, there was a change in the ranks of the first two-ranked alternatives, A1 and A3, respectively, the rank A3>A1>A5>A4>A6>A2 was obtained. 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 10 20 30 40 50 Criteria weights S c e n a ri o s C1 C2 C3 C4 C5 C6 Fig. 3 Weight coefficients of the criteria through 50 scenarios 0 5 10 15 20 25 30 35 40 45 50 4.65 4.7 4.75 4.8 4.85 4.9 Scenarios S c o re f u n c ti o n o f L M A W m o d e l A1 A2 A3 A4 A5 A6 Fig. 4 Influence of the change of criterion C6 to the change of the indices of the alternatives of the LMAW model By the above-presented analysis it is shown that the changes in the values of the weight coefficients significantly affected the change in the value of the index of alternatives of the LMAW model, which further confirmed the sensitivity of the LMAW model. Based on the presented analysis it can also be concluded that the initial rank of the alternatives is confirmed and that alternatives {A1, A3} are indicated as good solutions, with the confirmed advantage of alternative A1 over alternative A3. A New Logarithm Methodology of Additive Weights (LMAW) for Multi-Criteria Decision-Making... 377 6. CONCLUSION In this paper, we present a new additive MCDM approach using logarithms and the Bonferroni function. We apply the proposed methodology for solving a real-life problem such as a comparative performance analysis of LSPs in Indian context. We observe that our method performs well as compared with the widely popular MCDM framework such as TOPSIS. Our method provides a more stable result and may be applied for solving complex real-life issues which involve a considerable number of conflicting criteria. However, this work has some limitations which may be treated as the scopes for future work. For example, we have only considered the operational metrics related to turnover and cost. In a typical complex scenario, one may include the criteria like order fulfillment, disruption risk loss, human resource productivity, market innovation, R&D expense, etc. Further, we have considered only six alternatives. One may check the robustness of this method considering a large set of alternatives and criteria. Further, the other functions like Einstein aggregation, Heronian mean function may be used to check the results. LMAW is proposed in this paper only. Therefore, one may be curious to develop some extended models in uncertain domain using fuzzy and rough sets. Nevertheless, we believe that these future scopes do not undermine the usefulness our proposed method. This easy-to-use methodology can be used to solve various complex engineering, basic science and management related problems. REFERENCES 1. Biswas, S., Anand, O.P., 2020, Logistics Competitiveness Index-Based Comparison of BRICS and G7 Countries: An Integrated PSI-PIV Approach, IUP Journal of Supply Chain Management, 17(2), pp. 32-57. 2. Tanyaş, M., Serdar, S., 2003, A comparison of quality performance criteria of logistics service providers and those of their customers, Proc. of the International logistics Congress, Istanbul, Turkey. 3. Yan, J., Chaudhry, P.E., Chaudhry, S.S., 2003, A model of a decision support system based on case based reasoning for third party logistics evaluation, Expert Systems, 20(4), pp. 196-207. 4. Deloitte Report, 2018, India - On the Cusp of a Logistics Revolution, Retrieved from https://www2.deloitte. com/content/dam/Deloitte/in/Documents/public-sector/in-ps-india-on-cusp-of-a-logistics-revolution-noexp.pdf (last access: 03.02.2021) 5. https://www.reportlinker.com/p05878611/Indian-Logistics-Industry-Outlook.html?utm_source=GNW (last access: 02.02.2021) 6. Liu, C.L., Lyons, A.C., 2011, An analysis of third-party logistics performance and service provision, Transportation Research Part E: Logistics and Transportation Review, 47(4), pp. 547-570. 7. Rashidi, K., Cullinane, K., 2019, Evaluating the sustainability of national logistics performance using Data Envelopment Analysis, Transport Policy, 74, pp. 35-46. 8. Cakir, E., Tozan, H., Vayvay, O., 2009, A method for selecting third party logistic service provider using fuzzy AHP, Journal of Naval Science and Engineering, 5(3), pp. 38-54. 9. Meade, L., Sarkis, J., 2002, A conceptual model for selecting and evaluating third‐party reverse logistics providers, Supply Chain Management: An International Journal, 7(5), pp. 283-295. 10. Efendigil, T., Önüt, S., Kongar, E., 2008, A holistic approach for selecting a third-party reverse logistics provider in the presence of vagueness, Computers & Industrial Engineering, 54(2), pp. 269-287. 11. Chen, K.Y., Wu, W.T., 2011, Applying analytic network process in Logistics service provider selection-A case study of the Industry Investing in Southeast Asia, International Journal of Electronic Business Management, 9(1), pp. 24-36. 12. Aguezzoul, A., 2014, Third-party logistics selection problem: A literature review on criteria and methods, Omega, 49, pp. 69-78. 13. Guarnieri, P., Sobreiro, V.A., Nagano, M.S., Serrano, A.L.M., 2015, The challenge of selecting and evaluating third-party reverse logistics providers in a multi-criteria perspective: a Brazilian case, Journal of Cleaner Production, 96, pp. 209-219. 14. Yu, M.C., Wang, C.N., Ho, N.N.Y., 2016, A grey forecasting approach for the sustainability performance of logistics companies, Sustainability, 8(9), 866. https://www2.deloitte.com/content/dam/Deloitte/in/Documents/public-sector/in-ps-india-on-cusp-of-a-logistics-revolution-noexp.pdf https://www2.deloitte.com/content/dam/Deloitte/in/Documents/public-sector/in-ps-india-on-cusp-of-a-logistics-revolution-noexp.pdf https://www.reportlinker.com/p05878611/Indian-Logistics-Industry-Outlook.html?utm_source=GNW 378 D. PAMUČAR, M. ŽIĆOVIĆ, S. BISWAS, D. BOŽANIĆ 15. Sremac, S., Stević, Ž., Pamučar, D., Arsić, M., Matić, B., 2018, Evaluation of a third-party logistics (3PL) provider using a rough SWARA–WASPAS model based on a new rough dombi aggregator, Symmetry, 10(8), 305. 16. Zarbakhshnia, N., Soleimani, H., Ghaderi, H., 2018, Sustainable third-party reverse logistics provider evaluation and selection using fuzzy SWARA and developed fuzzy COPRAS in the presence of risk criteria, Applied Soft Computing, 65, pp. 307-319. 17. Li, Y.L., Ying, C.S., Chin, K.S., Yang, H.T., Xu, J., 2018, Third-party reverse logistics provider selection approach based on hybrid-information MCDM and cumulative prospect theory, Journal of Cleaner Production, 195, pp. 573-584. 18. Jovčić, S., Průša, P., Dobrodolac, M., Švadlenka, L., 2019, A proposal for a decision-making tool in third-party logistics (3PL) provider selection based on multi-criteria analysis and the fuzzy approach, Sustainability, 11(15), 4236. 19. Bajec, P., Tuljak-Suban, D., 2019, An Integrated Analytic Hierarchy Process—Slack Based Measure- Data Envelopment Analysis Model for Evaluating the Efficiency of Logistics Service Providers Considering Undesirable Performance Criteria, Sustainability, 11(8), 2330. 20. Yannis, G., Kopsacheili, A., Dragomanovits, A., Petraki, V., 2020, State-of-the-art review on multi- criteria decision-making in the transport sector, Journal of Traffic and Transportation Engineering, 7(4), pp. 413-431. 21. Ha, M.H., Yang, Z., Lam, J.S.L., 2019, Port performance in container transport logistics: A multi- stakeholder perspective, Transport Policy, 73, pp. 25-40. 22. Wang, H., Jiang, Z., Zhang, H., Wang, Y., Yang, Y., Li, Y., 2019, An integrated MCDM approach considering demands-matching for reverse logistics, Journal of Cleaner Production, 208, pp. 199-210. 23. Biswas, S., Pamucar, D., 2020, Facility location selection for B-schools in Indian context: A multi- criteria group decision based analysis, Axioms, 9(3), 77. 24. Ghosh, I., Biswas, S., 2017, A novel framework of ERP implementation in Indian SMEs: Kernel principal component analysis and intuitionistic Fuzzy TOPSIS driven approach, Accounting, 3(2), pp. 107-118. 25. Ghosh, I., Biswas, S., 2016. A comparative analysis of multi-criteria decision models for ERP package selection for improving supply chain performance, Asia-Pacific Journal of Management Research and Innovation, 12(3-4), pp. 250-270. 26. Biswas, S., 2020, Measuring performance of healthcare supply chains in India: A comparative analysis of multi-criteria decision-making methods, Decision-making: Applications in Management and Engineering, 3(2), pp. 162-189. 27. Biswas, S., Bandyopadhyay, G., Guha, B., Bhattacharjee, M., 2019. An ensemble approach for portfolio selection in a multi-criteria decision-making framework. Decision-making: Applications in Management and Engineering, 2(2), pp. 138-158. 28. Zolfani, S., Yazdani, M., Pamucar, D., Zarate, P., 2020, A VIKOR and TOPSIS focused reanalysis of the MADM methods based on logarithmic normalization, Facta Universitatis-Series Mechanical Engineering, 18(3), pp. 341-355. 29. MacCrimmon, K.R., 1968. Decisionmaking among multiple-attribute alternatives: a survey and consolidated approach, Rand Corp Santa Monica Ca. 30. Roy, B., 1968, Classement et choix en présence de points de vue multiples, Revue française d'informatique et de recherche opérationnelle, 2(8), pp. 57-75. 31. Saaty, T.L., 1980, The Analytic Hierarchy Process, McGraw-Hill: New York, NY, USA. 32. Duckstein, L., Opricovic, S., 1980, Multiobjective optimization in river basin development, Water resources research, 16(1), pp. 14-20. 33. Opricovic, S., Tzeng, G.H., 2004, Compromise solution by MCDM methods: A comparative analysis of VIKOR and TOPSIS, European Journal of Operational Research, 156(2), pp. 445-455. 34. Hwang, C.L., Yoon, K., 1981, Multiple Attribute Decision-making: Methods and Applications, Springer: New York, NY, USA. 35. Brans, J.P., 1982, L'ingénierie de la décision: l'élaboration d'instruments d'aide a la decision, Université Laval, Faculté des sciences de l'administration, Canada. 36. Brans, J.P., Mareschal, B., 1992, PROMETHEE V: MCDM problems with segmentation constraints, INFOR: Information Systems and Operational Research, 30(2), pp. 85-96. 37. Keeney, R.L., Raiffa, H., Meyer, R.F., 1993, Decisions with multiple objectives: preferences and value trade-offs, Cambridge university press. 38. Zavadskas, E.K., Kaklauskas, A., Sarka, V., 1994, The new method of multi-criteria complex proportional assessment of projects, Technological and Economic Development of Economy, 1(3), pp. 131-139. A New Logarithm Methodology of Additive Weights (LMAW) for Multi-Criteria Decision-Making... 379 39. Saaty, R.W., 2003, Decision-making in Complex Environment: The Analytic Hierarchy Process (AHP) for Decision-making and the Analytic Network Process (ANP) for Decision-making with Dependence and Feedback, Super Decisions, Pittsburgh, USA. 40. Brauers, W.K., Zavadskas, E.K., 2006, The MOORA method and its application to privatization in a transition economy, Control and cybernetics, 35, pp. 445-469. 41. Brauers, W.K.M., Zavadskas, E.K., 2010, Project management by MULTIMOORA as an instrument for transition economies, Technological and Economic Development of Economy, 16(1), pp. 5-24. 42. Zavadskas, E.K., Turskis, Z., 2010, A new additive ratio assessment (ARAS) method in multicriteria decision‐making, Technological and Economic Development of Economy, 16(2), pp. 159-172. 43. Keršuliene, V., Zavadskas, E.K., Turskis, Z., 2010, Selection of rational dispute resolution method by applying new step‐wise weight assessment ratio analysis (SWARA), Journal of business economics and management, 11(2), pp. 243-258. 44. Das, M.C., Sarkar, B., Ray, S., 2012, Decision-making under conflicting environment: a new MCDM method, International Journal of Applied Decision Sciences, 5(2), pp. 142-162. 45. Zavadskas, E.K., Turskis, Z., Antucheviciene, J., Zakarevicius, A., 2012, Optimization of weighted aggregated sum product assessment, Elektronika ir elektrotechnika, 122(6), pp. 3-6. 46. Krylovas, A., Zavadskas, E.K., Kosareva, N., Dadelo, S., 2014, New KEMIRA method for determining criteria priority and weights in solving MCDM problem, International Journal of Information Technology & Decision-making, 13(06), pp. 1119-1133. 47. Pamučar, D., Ćirović, G., 2015, The selection of transport and handling resources in logistics centers using Multi-Attributive Border Approximation area Comparison (MABAC), Expert systems with applications, 42(6), pp. 3016-3028. 48. Keshavarz Ghorabaee, M., Zavadskas, E.K., Olfat, L., Turskis, Z., 2015, Multi-criteria inventory classification using a new method of evaluation based on distance from average solution (EDAS). Informatica, 26(3), pp. 435-451. 49. Keshavarz Ghorabaee, M., Zavadskas, E.K., Turskis, Z., Antucheviciene, J., 2016, A new combinative distance-based assessment (CODAS) method for multi-criteria decision-making, Economic Computation & Economic Cybernetics Studies & Research, 50(3), pp. 25-44. 50. Stanujkic, D., Zavadskas, E. K., Karabasevic, D., Smarandache, F., Turskis, Z. (2017), The Use Of The Pivot Pairwise Relative Criteria Importance Assessment Method For Determining The Weights Of Criteria, Journal for Economic Forecasting, 4, pp. 116-133. 51. Pamučar, D., Stević, Ž., Sremac, S., 2018, A new model for determining weight coefficients of criteria in mcdm models: Full consistency method (FUCOM), Symmetry, 10(9), 393. 52. Yazdani, M., Zarate, P., Zavadskas, E.K., Turskis, Z., 2019, A Combined Compromise Solution (CoCoSo) method for multi-criteria decision-making problems, Management Decision, 57(9), pp. 2501-2519. 53. Žižović, M., Pamucar, D., 2019, New model for determining criteria weights: Level Based Weight Assessment (LBWA) model, Decision-making: Applications in Management and Engineering, 2(2), pp. 126-137. 54. Stević, Ž., Pamučar, D., Puška, A., Chatterjee, P., 2020, Sustainable supplier selection in healthcare industries using a new MCDM method: Measurement of alternatives and ranking according to COmpromise solution (MARCOS), Computers & Industrial Engineering, 140, 106231. 55. Žižović, M., Pamučar, D., Albijanić, M., Chatterjee, P., Pribićević, I., 2020, Eliminating Rank Reversal Problem Using a New Multi-Attribute Model—The RAFSI Method, Mathematics, 8(6), 1015. 56. Jharkharia, S., Shankar, R., 2007, Selection of logistics service provider: An analytic network process (ANP) approach, Omega, 35(3), pp. 274-289. 57. Qureshi, M.N., Kumar, D., Kumar, P., 2007, Performance evaluation of 3PL Services provider using AHP and TOPSIS: a case study, The ICFAI Journal of Supply Chain Management, 4(3), pp. 20-38. 58. Hamdan, A., Rogers, K.J., 2008, Evaluating the efficiency of 3PL logistics operations, International Journal of Production Economics, 113(1), pp. 235-244. 59. Sheng, X.H., Yang, W.P., Chen, L.H., Yang, H.Y., 2012, Research on the choice of the third-party reverse logistics enterprise based on the method of AHP and Goal Programming, Advanced Materials Research, 452-453, pp. 581-585. 60. Andrejić, M.M., 2013,, Measuring efficiency in logistics, Military Technical Courier, 61(2), pp. 84-104. 61. Wong, J.T., 2012, DSS for 3PL provider selection in global supply chain: combining the multi-objective optimization model with experts’ opinions, Journal of Intelligent Manufacturing, 23(3), pp. 599-614. 62. Senthil, S., Srirangacharyulu, B., Ramesh, A., 2014, A robust hybrid multi-criteria decision-making methodology for contractor evaluation and selection in third-party reverse logistics, Expert Systems with Applications, 41(1), pp. 50-58. 380 D. PAMUČAR, M. ŽIĆOVIĆ, S. BISWAS, D. BOŽANIĆ 63. Işıklar, G., Alptekin, E., Büyüközkan, G., 2007, Application of a hybrid intelligent decision support model in logistics outsourcing, Computers & Operations Research, 34(12), pp. 3701-3714. 64. Liu, H.T., Wang, W.K., 2009, An integrated fuzzy approach for provider evaluation and selection in third-party logistics, Expert Systems with Applications, 36(3), pp. 4387-4398. 65. Kannan, G., Pokharel, S., Kumar, P.S., 2009, A hybrid approach using ISM and fuzzy TOPSIS for the selection of reverse logistics provider, Resources, conservation and recycling, 54(1), pp. 28-36. 66. Akman, G., Baynal, K., 2014, Logistics service provider selection through an integrated fuzzy multicriteria decision-making approach, Journal of industrial Engineering, 2014, 794918. 67. Prakash, C., Barua, M.K., 2016, A combined MCDM approach for evaluation and selection of third-party reverse logistics partner for Indian electronics industry, Sustainable Production and Consumption, 7, pp. 66-78. 68. Liu, Y., Zhou, P., Li, L., Zhu, F., 2020, An interactive decision-making method for third-party logistics provider selection under hybrid multi-criteria, Symmetry, 12(5), 729. 69. Pamucar, D., Chatterjee, K., Zavadskas, E.K., 2019, Assessment of third-party logistics provider using multi-criteria decision-making approach based on interval rough numbers, Computers & Industrial Engineering, 127, pp. 383-407. 70. Chithambaranathan, P., Subramanian, N., Gunasekaran, A., Palaniappan, P.K., 2015, Service supply chain environmental performance evaluation using grey based hybrid MCDM approach, International Journal of Production Economics, 166, pp. 163-176. 71. Julong, D., 1982, Control problems of grey systems, Systems & control letters, 1(5), pp. 288-294. 72. Julong, D., 1989, Introduction to grey system theory, The Journal of grey system, 1(1), pp. 1-24. 73. Mercangoz, B.A., Yildirim, B.F., Yildirim, S.K., 2020, Time period based COPRAS-G method: application on the Logistics Performance Index, LogForum, 16(2), pp. 239-250. 74. Zavadskas, E.K., Kaklauskas, A., Turskis, Z., Tamošaitien, J., 2008, Selection of the effective dwelling house walls by applying attributes values determined at intervals, Journal of Civil Engineering and Management, 14, pp. 85-93. 75. Pamucar, D., Božanić, D., Ranđelović, A., 2017, Multi-criteria decision-making: An example of sensitivity analysis, Serbian journal of management, 11(1), pp. 1-27. 76. Mukhametzyanov, I., Pamucar, D., 2018, A sensitivity analysis in MCDM problems: A statistical approach, Decision-making: Applications in Management and Engineering, 1(2), pp. 51-80. 77. Bobar, Z., Božanić, D., Đurić-Atanasievski, K., Pamučar, D., 2020, Ranking and Assessment of the Efficiency of Social Media using the Fuzzy AHP-Z Number Model - Fuzzy MABAC, Acta Polytechnica Hungarica, 17(3), pp. 43-70. 78. Pamučar D., Božanić D., Kurtov D., 2016, Fuzzification of the Saaty’s scale and a presentation of the hybrid fuzzy AHP-TOPSIS model: an example of the selection of a Brigade Artillery Group firing position in a defensive operation, Military Technical Courier, 64(4), pp. 966-986. 79. Pamučar, D., Ćirović, G., Božanić, D., 2019, Application of interval valued fuzzy-rough numbers in multi-criteria decision-making: The IVFRN-MAIRCA model, Yugoslav Journal of Operations Research, 29(2), pp. 221-247. 80. Belton, V., Gear, T., 1985, The legitimacy of rank reversal--A comment, Omega, 13(3), pp. 143-144. 81. Ecer, F., Pamucar, D., 2020, Sustainable supplier selection: A novel integrated fuzzy best worst method (F-BWM) and fuzzy CoCoSo with Bonferroni (CoCoSo'B) multi-criteria model, Journal of Cleaner Production, 266, 121981.