Operational Research in Engineering Sciences: Theory and Applications Vol. 3, Issue 3, 2020, pp. 101-122 ISSN: 2620-1607 eISSN: 2620-1747 DOI: https://doi.org/10.31181/oresta20303101k kysem24@gmail.com; semakayapinar@munzur.edu.tr EVALUATION OF THE EFFECT OF COVID-19 ON COUNTRIES’ SUSTAINABLE DEVELOPMENT LEVEL: A COMPARATIVE MCDM FRAMEWORK Sema Kayapinar Kaya Department of Industrial Engineering, Munzur University, Tunceli, Turkey Received: 02 November 2020 Accepted: 01 December 2020 First online: 06 December 2020 Original scientific paper Abstract: The extent of the outbreak of coronavirus disease (COVID-19) had a major impact on health, social life, economic and environmental activities in almost every country over the world. It has disrupted the sustainable development of countries and brought many uncertainties for their future capabilities. In this study, the effects of the COVID-19 on OECD countries' sustainable development were investigated, and the sustainable development performance of the countries was evaluated by the MultiAttributive Ideal-Real Comparative Analysis (MAIRCA) method. Data for the second quarter of 2020 and the same quarter of the previous year is considered. Then, the results obtained by the MAIRCA method were compared with two different multi-criteria decision-making (MCDM) methods called MABAC (Multi‐Attributive Border Approximation Area Comparison) and WASPAS (Weighted Aggregated Sum Product Assessment). The effectiveness and validity of the results obtained from these methods were tested with Spearman's correlation coefficient. Finally, to examine the effect of COVID-19 on the indicators of sustainable development, a non-parametric Wilcoxon signed-rank test was applied. As a result, it was concluded that COVID-19 negatively affected the sustainable development of countries. However, sustainable development performances of developed countries have been observed to be better than developing countries. Keywords: Pandemic, COVID-19, Sustainability, MAIRCA, Multi-Criteria Decision Making 1. Introduction In December 2019, Chinese Center for Disease Control and Prevention and Wuhan city health authorities reported an unknown pneumonia outbreak in Wuhan City, Hubei Province, China. On January 7, 2020, the center detected a new type of coronavirus that has never been seen in humans from the lower respiratory tract samples of patients (Wang et al., 2020; Li et al., 2020). Samples from the first patients Kaya/Oper. Res. Eng. Sci. Theor. Appl. 3 (3) (2020) 101-122 102 were tested with many known pathogens. The new type of coronavirus showed similarity to respiratory diseases such as Severe Acute Respiratory Syndrome (SARS- CoV) and Middle East Respiratory Syndrome (MERS) (Lai et al., 2020). Symptoms of the new type of coronavirus include fever, cough, shortness of breath, and dyspnea. However, these symptoms differ from person to person. While most infected people develop mild to moderate symptoms, some patients experience severe pneumonia, pulmonary edema, and multiple organ failure, leading to death. This infectious virus has been officially named as Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) by the World Health Organization (WHO). WHO used the term COVID-19 to describe the disease caused by the virus (WHO, 2020). This disease, the first case of which appeared in China in the last days of 2019, later began to occur in countries such as Japan, South Korea, and Thailand (Chen et al., 2020). The disease spread rapidly around the world in January 2020, and cases of virus began to be reported in several countries in Europe, North America, and Asia- Pacific (CDC, 2020; Hui et al., 2020; Bedford et al., 2020). On March 11, 2020, WHO declared COVID-19 as a global pandemic (WHO, 2020). At the time of writing this paper, the COVID-19 epidemic affected 213 countries and regions worldwide, infected more than 11 million 600 thousand people and the number of people who died globally has exceeded 530 thousand (COVID-19 Virus Pandemic, 2020). After WHO announced that the epicenter of the COVID-19 outbreak was Europe in the spring of 2020, the loss of life caused by COVID-19 increased in many countries, especially in Italy, France, Spain, and the United Kingdom. Again, at the time of writing this paper, it was announced by WHO that the new epicenter of COVID-19 was the continent of America and Asia. It was also reported that the pandemic risk continues largely; the USA, with more than 7 million cases, and India, with 6 million cases, have been the two countries in the world with the most COVID-19 cases. The number of deaths in both countries is increasing day by day and 200 thousand people in the USA and 96 thousand people in India died due to COVID-19 (Coronavirus Update, 2020). Although the COVID-19 pandemic was initially seen as a global health crisis, the situation has changed as the extent of the pandemic increased, and COVID-19 has become a deep political, economic, social, and environmental crisis in every country it touched. Due to the pandemic, in almost 90% of the world a wide range of social isolation and curfews have been implemented, many businesses have been closed, and domestic and international transportation services have been disrupted. Social restrictions and home isolation negatively affected many sectors especially production, health, transportation, tourism, real estate, education, energy, banking, etc. (Deloitte | Annual Turkish M&A Review, 2019). Even in the largest economies of the world, an economic contraction is foreseen that exceeds the estimates. According to International Monetary Fund (IMF) report titled "great isolation" dated April 14, 2020, the world is expected to experience the biggest global economic crisis since the "great depression" in 1929. In the report published by IMF, the growth expectation of the global economy has been revised as 3% shrinkage instead of 3.3% growth for 2020, and it is expected that global trade will decrease by 11% and oil prices by 42%. Due to the pandemic, a great decrease has occurred in the production and service sector and this made developing countries face high inflation and increasing unemployment. The gross domestic product, one of the most important economic indicators, fell by 1.8% in the OECD region in the first quarter of 2020 (World Economic Outlook, 2020). Evaluation of the Effect of COVID-19 on Countries’ Sustainable Development Level: A comparative MCDM framework 103 However, it is not exactly known how dramatic the effects of the COVID-19 outbreak on the global economy will be. In The World Economic Forum report, it was stated that in addition to the economic problems, many countries would face many multidimensional problems in tourism, the housing market, demand for commercial products, transportation, unemployment, education, energy consumption, and impact on social life (World Economic Outlook, 2020). Tourism and service transportation is one of the sectors heavily hit by COVID-19. Transportation and transport activities have almost come to a halt during the quarantine process. The World Travel and Tourism Council stated that 50 million jobs operating in the global tourism and travel industry are at risk (News Article | World Travel & Tourism Council (WTTC), 2020). With the decrease in production in the pandemic period, the amount of energy needs to be decreased, as a result of which a decrease in energy production and investment was experienced. The positive aspect of the pandemic is that despite the decline in energy investments, renewable energy has resisted and continued to grow against the pandemic (IEA, 2020). Even though the disruption of the education process of children was prevented by initiating the distance / online education processes in the quarantine process, the discount / free meal application given in schools in many countries, which is especially important for disadvantaged people, was disrupted, and some of the students suspended their education as they could not connect to the internet, and this situation brought about socio-economic inequalities. As can be seen, the pandemic is a multidimensional global crisis that affects the economic, social, and environmental factors of the countries and disrupts its sustainable development. The United Nations (UN) stated that all the work done during and after this crisis should focus on building more resilient, equal, inclusive, and sustainable economies in the face of the challenges we face. Also, it was emphasized that the countries' recovery and sustainable development goals should be taken into consideration more than ever before to cope with the shocks that may be encountered in the pandemic in the future (UNDP, 2020). Only a few studies have addressed the threat of the coronavirus pandemic to the sustainable development levels of countries. This paper attempts to investigate how the COVID-19 pandemic has changed the level of sustainable development for developed and less developed countries. Additionally, MCDM has become widely used in different sustainable development context over the past few years (Perez-Gladish et al., 2020). For this purpose, this study is to evaluate and compare the level of sustainable development of the OECD countries by using the MAIRCA model. MAIRCA is an effective MCDM method that takes into account the concept of the positive and negative ideal solution. The results obtained with the MAIRCA method were compared with new multi-criteria decision-making methods such as MABAC and WASPAS. The efficacy and validity of the results obtained in the three methods were tested with Spearman's correlation coefficient. Finally, the Wilcoxon signed-rank test was performed to examine the impact of COVID-19 on the indicators of sustainable development. The rest of the paper is organized as follows: In Section 2, literature review on MAIRCA has presented. The steps of the MAIRCA method is explained in Section 3. Evaluation of the effect of COVID-19 on countries’ sustainable development level based on the MAIRCA method is given in the fourth section. The results obtained to test the effectiveness and validity of MAIRCA method are illustrated in Section 5. Next Kaya/Oper. Res. Eng. Sci. Theor. Appl. 3 (3) (2020) 101-122 104 section, the results of the Non-Parametric Wilcoxon signed-rank test are presented. Results and some limitations are discussed in detail in Section 7.. 2. Literature review MAIRCA is a popular method within the group of MCDM methods which is developed by Professor Dragan Pamucar in the Logistics Research Centre at the University of Defence in Belgrade (Pamucar et al., 2014). MAIRCA is easy to use in computation procedure and its calculation steps are similar to the ideal and non-ideal solution approach in the technique for order of preference by similarity to ideal solution (TOPSIS) method. (Gul and Ak, 2020). The MAIRCA model is a considerable new decision-making method that can be very successfully combined with different MCDM methods. Related literature has been evaluated over the years. Gigović et al. (2016) aimed to determine the appropriate location for the ammunition depots by using the Geographic Information System (GIS) and MAIRCA methods together. To do this, the priority weights criteria of depots were determined by DEMATEL-ANP, and then the ranking of alternative regions was performed by MAIRCA. In the study by (Pamučar et al., 2017a), using a hybrid approach, the tenderers of the public procurement tender were evaluated by means of rough number based on DEMATEL, ANP, and MAIRCA methods. In another study, Pamucar et al. (2018) used the Full Consistency Method (FUCOM) and MAIRCA integrated methods in the location selection of level crossings to reduce the number of traffic accidents. Pamučar et al. (2019) defined six alternatives in determining the landing departure point of the vehicles in combat operations and they ranked their priorities with MAIRCA using interval-valued fuzzy-rough numbers. In their research, Badi and Ballem (2018) evaluated the supplier selection process by applying the integration of the rough numbers with the Best-Worst Method (BWM) and MAIRCA methods. As a result, it is determined that the cost, quality, and company profile are the three most important criteria. Chatterjee et al. (2018) evaluated the suppliers' performances considering the green supply chain criteria with the help of rough DEMATEL, Analytic Network Process (ANP) and MAIRCA methods. Pamucar et al. (2018) performed the location selection for a multi-model logistics facility that took into account sustainability criteria with the help of DEMATEL-MAIRCA methods. Based on the two main criteria that affect the ergonomic risk level, Ekinci and Can (2018) developed the CRITIC-MAIRCA method to achieve a combined risk level by taking into consideration the evaluation results made for the sub-criteria of these main criteria. Boral et al. (2020) listed the types of errors seen in the production facility of a small and medium-sized (SME) company operating in the automobile industry using the fuzzy MAIRCA method. Ulutaş (2019) used the Step-Wise Weight Assessment Ratio Analysis (SWARA) and MAIRCA integrated method in the selection of the catering company. Aycin (2020) used The criteria importance through intercriteria correlation (CRITIC) and MAIRCA methods in the selection of personnel to work in the IT department of a company operating in the logistics industry. Arsić et al. (2019) made a menu evaluation for a restaurant with BWM and rough MAIRCA methods. In the study by (Chatterjee et al., 2020), the MAIRCA method was used to evaluate the alternatives in lightweight environmentally friendly materials in the automotive industry. Pirbasti et al. (2020) selected the waste disposal facility location of eight hospitals using a hybrid approach with fuzzy SWARA and GIS-MAIRCA. In the study by Evaluation of the Effect of COVID-19 on Countries’ Sustainable Development Level: A comparative MCDM framework 105 Pamučar and Savin (2020), BWM and MAIRCA methods were utilized together for the selection of military land vehicles, taking into account the 11 criteria defined. Gul and Ak (2020) used BWM and MAIRCA methods under fuzzy conditions to analyze potential risks in the marble factory. The relative importance of the three risk factors in the traditional Fine-Kinney method was calculated with fuzzy BWM, and the identified risks were ranked by fuzzy - MAIRCA. 3. MAIRCA method MAIRCA, which has been added to MCDM literature by Gigovic et al., 2016, is a method based on defining the gaps between ideal and empirical ratings. By the addition of the gaps for each criterion, the total gap for decision alternatives is obtained. At the end of the application process, the alternative that is the closest to the ideal ratings according to most of the criteria, or in other words, the alternative with the lowest total gap value is determined as the best alternative (Gigović et al., 2016; Pamučar et al., 2017). The MAIRCA method has an implementation process consisting of eight steps (Pamucar et al., 2018). Step 1: Creating the Initial Decision Matrix (X): The criteria(Cj) values obtained from each alternative (Ai) are shown in Equation (1). 𝐶1 𝐶2 … 𝐶𝑛 𝑋 = [ 𝑥11 𝑥12 … 𝑥1𝑛 𝑥21 𝑥22 … 𝑥2𝑛 ⋮ ⋮ ⋱ ⋮ 𝑥𝑚1 𝑥𝑚2 … 𝑥𝑚𝑛 ] (0) Step 2: Determining the Priorities of Alternatives: The absence of a priority in the alternative selection process of the decision-maker is an assumption of the method, m as the total number of alternatives, i. The priority of the alternative 𝑃𝑟𝐴𝑖 is calculated as shown in Equation (2). 𝑃𝑟𝐴𝑖 = 1 𝑚 ; ∑ 𝑃𝑟𝐴𝑖 𝑚 𝑖=1 = 1 𝑖 = 1, 2, … , 𝑚 (2) The decision-maker is equidistant to any alternative. Therefore, all priorities are equal, as shown in Equation (3). 𝑃𝑟𝐴1 = 𝑃𝑟𝐴2 = ⋯ = 𝑃𝑟𝐴𝑚 (3) Stage 3: Construction of the Theoretical Rating Matrix (𝑻𝒑): The elements of the matrix (𝑡𝑝𝑖𝑗 ) are calculated by multiplying the priorities of alternatives (𝑃𝑟𝐴𝑖 ) and the criterion weights (𝑤𝑗 ), as shown in Equation (4). Kaya/Oper. Res. Eng. Sci. Theor. Appl. 3 (3) (2020) 101-122 106 𝑻𝒑 = [ PrA1. w1 PrA1. w2 … PrA1. wn PrA2. w1 PrA2. w2 … PrA2. wn ⋮ ⋮ ⋱ ⋮ PrAm. w1 PrAm. w2 … PrAm. wn ] (4) Stage 4: Defining the Real Rating Matrix (𝑻𝒓): In order to obtain 𝑇𝑟 matrix, theoretical grading matrix 𝑇𝑝 and initial decision matrix 𝑋 are used. Matrix elements should be calculated by using Equation (5) for maximization criteria and Equation (6) for minimization criteria. trij = tpij . ( xij − xij − xij + − xij −) (5) trij = tpij . ( xij − xij + xij − − xij +) (6) 𝑥𝑖𝑗 + is the highest value of the criterion from the alternative (𝑥𝑖𝑗 + = max (𝑥1, 𝑥2, … , 𝑥𝑚 )), 𝑥𝑖𝑗 − is the lowest value of the criterion from the alternative (𝑥𝑖𝑗 − = min (𝑥1, 𝑥2, … , 𝑥𝑚 )). The actual rating matrix to be obtained as a result of calculations is shown in Eq. (7). 𝐶1 𝐶2 … 𝐶𝑛 𝑇𝑟 = [ tr11 tr12 … tr1n tr21 tr22 … tr2n ⋮ ⋮ ⋱ ⋮ trm1 trm2 … trmn ] (7) Stage 5: Computation of Total Gap Matrix (G) With the help of Gap Matrix (𝐺), Equation (8), the difference between the theoretical rating matrix (𝑇𝑝) and the actual grading matrix (𝑇𝑟 ) is obtained as shown in Eq. (9). 𝑔𝑖𝑗 = 𝑡𝑝𝑖𝑗 − 𝑡𝑟𝑖𝑗 𝑔𝑖𝑗 ∈ [0, ∞) (8) 𝐺 = 𝑇𝑝 − 𝑇𝑟 = [ 𝑔11 𝑔12 … 𝑔1𝑛 𝑔21 𝑔22 … 𝑔2𝑛 ⋮ ⋮ ⋱ ⋮ 𝑔𝑚1 𝑔𝑚2 … 𝑔𝑚𝑛 ] (9) Stage 6: Determining the Total Gap with Alternatives If theoretical rating (𝑡𝑝𝑖𝑗 ) and real rating (𝑡𝑟𝑖𝑗 ) of an alternative (𝐴𝑖 ) for a criterion (𝐶𝑗 ) are equal and different from zero, the gap will be zero (𝑔𝑖𝑗 = 0). In this case, this alternative (𝐴𝑖) would be the ideal alternative (𝐴𝑖 +) for this criterion (𝐶𝑗 ). If the theoretical rating (𝑡𝑝𝑖𝑗 ) of an alternative (𝐴𝑖) for a criterion (𝐶𝑗 ) equals zero (𝑡𝑝𝑖𝑗 = 𝑡𝑟𝑖𝑗 = 𝑔𝑖𝑗 = 0), then the gap for the alternative (𝐴𝑖 ) for the criterion (𝐶𝑗 ) is (𝑔𝑖𝑗 = 0). In this case, this alternative (𝐴𝑖 ) will be the worst alternative (𝐴𝑖 −) for this criterion (𝐶𝑗 ). Evaluation of the Effect of COVID-19 on Countries’ Sustainable Development Level: A comparative MCDM framework 107 Stage 7: Calculation of the Value (𝑸𝒊) of the Final Criteria Functions of Alternatives The value of the criteria functions is calculated to take advantage of Equation (10) for each alternative. 𝑄𝑖 = ∑ 𝑔𝑖𝑗 𝑛 𝑗=1 , 𝑖 = 1, 2, … , 𝑚 (10) 𝑄𝑖 values are ranked from small to a large value, and alternatives are obtained. 4. Evaluate the effects of the COVID-19 pandemic on sustainable development performance of OECD countries The Organization for Economic Co-operation and Development, or OECD in short, is an international platform that works jointly to solve the economic, social, and management problems of member countries. This establishment was founded in Paris in 1961 and was originally established with 20 countries. Later, the number of OECD member countries increased to 37 with the participation in developmentally and socioeconomically different countries (Our global reach - OECD, 2020). The main purpose of OECD is to support countries in ensuring sustainable economic growth, increasing employment, raising living standards, ensuring economic stability and contributing to the growth of world trade. Member states of the organization constitute 63 percent of GDP, three-quarters of world trade, 95 percent of world official development aid and more than half of world energy consumption in today's world (What is OECD, 2020). In the study, Colombia, Luxemburg, Israel and New Zealand were excluded from this study due to the unavailability of some data for this method. The impact of COVID- 19 on the sustainable development performances of a total of 33 OECD countries were analyzed using MAIRCA considering the data of the second quarter (Q2) of 2020 and the data of the same quarter of the previous year (2019). To test the validity and effectiveness of this method, the ranking results of MAIRCA were compared with the results obtained from novel MCDM models such as MABAC and WASPAS. Additionally, the Wilcoxon signed-rank test is conducted to determine whether the sustainable development indicators of OECD countries differ between the Q2/2019 and Q2/2020. The general framework of this study is summarized in Figure 1. Kaya/Oper. Res. Eng. Sci. Theor. Appl. 3 (3) (2020) 101-122 108 Figure 1. General framework of study Sustainable development indicators of OECD countries have been determined in line with the sustainable development goals of OECD, the European Union, and the United Nations and literature review. In addition, all the indicators used in this study Collect data from OECD and IEA database Identify sustainable development indicators Comparison and sensitivity analysis using the MABAC and WASPAS method Explore assumptions for the statistical hypothesis test Comparison of data for the same quarter of two different year Discuss the result of Wilcoxon signed-rank test Step 1: Describe the problem and Identify indicators Step 2: MAIRCA method Step 3. Sensitivity Analysis Step 4. Wilcoxon test Evaluation of the Effect of COVID-19 on Countries’ Sustainable Development Level: A comparative MCDM framework 109 were designed to take into account all dimensions, namely the economy, environment, and social, which constitute the sustainable development model. Eight different sustainable development indicators comprising Total Electricity Production, Renewable Energy Production, Merchandise Trade, Customer Price Index (CPI), Analytical House Price Indicators (rent price), Gross Domestic Product (GDP), Producer Price Index (PPI), unemployment rate, aged 15 and over were selected in this study. The indicators were analyzed for their ability to measure the economic, environmental and social dimensions of sustainable development. Data on economic and social indicators (I3, I4, I5, I6, I7, I8) were obtained from the OECD databases, and environmental indicators (I1 and I2) were obtained from reports published by the International Energy Agency (IEA). Definitions, indices, and periods considered in the study regarding indicators of sustainable development are shown in Table 1. Table 1. Indicators of sustainable development Not. Indicator Unit Quarter / Year References I1 Total Electricity Production GWh Q2/2019 - Q2/2020 (Ding et al., 2016); (Sustainable Development Goals, 2020) I2 Renewable Energy production GWh Q2/2019 - Q2/2020 (Mateusz et al., 2018); (Kothari et al., 2010); (Sustainable Development Goals, 2020); (Sathaye et al., 2011) I3 Merchandise Trade US Dollar, Billions Q2/2019 - Q2/2020 (Ding et al., 2016); (Sustainable Development Goals, 2020) I4 Customer price index Index, 2015=100 Q2/2019 - Q2/2020 (Gaspar et al., 2017); (Sustainable Development Goals, 2020) I5 Analytical House Rent Price indicators Index Q2/2019 - Q2/2020 (Zavadskas et al., 2017); (Sustainable Development Goals, 2020) I6 Gross Domestic Product Annual growth rate (%) Q2/2019 - Q2/2020 (Bali Swain and Yang-Wallentin, 2020); (Balcerzak and Pietrzak, 2016); (Ding et al., 2016): (Gaspar et al., 2017); (Sustainable Development Goals, 2020) I7 Producer price index Index, 2015=100 Q2/2019 - Q2/2020 (Sustainable Development Goals, 2020) I8 The unemployment rate, aged 15 and over % Q2/2019 - Q2/2020 (Bali Swain and Yang-Wallentin, 2020); (Balcerzak and Pietrzak, 2016); (Ding et al., 2016); (Mateusz et al., 2018); (Gaspar et al., 2017) Total Electricity Production (I1): Electricity generated different type of energy resources such as fossil fuels, nuclear power plants, hydropower plants (excluding pumped storage), geothermal systems, solar panels, biofuels, wind, etc. Renewable Energy production (I2): Renewable Energy is the energy received from the energy flow that exists in the natural processes that continue continuously. They are hydro energy, wind, solar, geothermal, and other renewable energy sources. Merchandise Trade (I3): Goods that add or subtract from the stock of material resources of a country by entering (imports) or leaving (exports) its economic territory. Kaya/Oper. Res. Eng. Sci. Theor. Appl. 3 (3) (2020) 101-122 110 Customer price index (I4): Defined as the change in the prices of a basket of goods and services that are typically purchased by specific groups of households. Analytical House Rent Price indicators (I5): House price indices (rent prices), are index numbers that measure the rent prices of residential properties over time. Gross Domestic Product (I6): The standard measure of the value-added created through the production of goods and services in a country during a certain period. Producer price index (I7): The rate of change in the prices of products sold as they leave the producer. The unemployment rate, aged 15 and over (I8): The number of unemployed people as a percentage of the labor force, where the latter consists of the unemployed plus those in paid or self-employment. Table 2. Initial decision matrix Indicators Countries I1 I2 I3 I4 I5 I6 I7 I8 Australia 21041 4304 69.61 106.6 102.5 1223934.95 111.1 5.231 Austria 6357 5842 44.75 106.7 114.4 469791.799 105.1 4.500 Belgium 6891 1376 112.24 107.9 103.9 553085.807 113.5 5.467 Canada 47322 31979 114.90 107.7 104.6 1723905.35 106.4 5.567 Chile 6804 2560 17.06 111.1 117.8 445180.531 108.8 6.980 Czech Republic 6613 925 50.17 108.1 109.7 406561.381 103.0 1.967 Denmark 2292 1955 27.84 103.1 104.9 308065.246 105.9 4.933 Estonia 489 137 4.11 109.9 130.7 45784.7699 109.7 4.867 Finland 5202 2999 18.65 103.4 108.2 254103.301 104.7 6.800 France 44439 9888 145.44 104.4 100.6 2908013.64 103.3 8.500 Germany 45887 20169 371.46 105.5 105.4 4150471.67 105.4 3.067 Greece 3117 1227 9.66 101.9 92.1 306790.238 104.7 17.333 Hungary 2258 358 30.36 109.4 123.4 307584.948 115.0 3.433 Iceland 1660 1660 1.28 109.4 120.0 18920.8022 99.0 3.367 Ireland 2293 625 42.61 102.1 115.9 413134.441 100.6 5.200 Italy 22141 10296 133.67 103.0 101.0 2335523.24 103.5 10.000 Japan 73035 36284 181.29 101.7 99.3 5356221.8 101.0 2.367 Korea 43570 3031 136.20 104.9 104.0 2150833.19 102.7 4.000 Latvia 402 244 3.94 109.4 107.0 55221.0561 110.5 6.367 Lithuania 282 206 8.41 110.4 126.0 96063.1159 107.2 6.100 Mexico 27884 5347 117.21 117.9 110.0 2388177.33 126.3 3.549 Netherlands 9374 1796 176.64 106.0 108.2 930705.86 109.4 3.333 Norway 9770 9502 26.31 110.6 107.6 331933.394 114.0 3.433 Poland 12052 1848 66.00 105.7 112.8 1204174.54 109.7 3.333 Portugal 4037 2176 16.89 104.2 107.5 339634.903 104.8 6.600 Slovak Republic 2264 695 22.21 106.0 101.3 181242.251 103.7 5.767 Slovenia 1419 588 11.31 105.4 118.4 75067.7421 103.1 4.367 Spain 20784 8803 84.80 104.6 103.0 1790431.14 105.3 14.200 Sweden 12703 7655 40.11 106.8 104.1 519697.892 112.1 6.533 Switzerland 5744 3551 61.12 102.0 102.1 569152.648 100.4 4.455 Turkey 23989 14407 44.52 158.2 141.4 2330974.19 186.2 13.867 United Kingdom 23914 8185 108.49 107.8 103.8 2948323.99 111.5 3.767 United States 332986 74378 408.63 108.0 115.3 19900185.1 106.94 3.633 AIM MAX MAX MAX MIN MIN MIN MIN MIN Evaluation of the Effect of COVID-19 on Countries’ Sustainable Development Level: A comparative MCDM framework 111 In this study, one of the expert is an engineer with expertise in Regional development at the Ankara Development Agency in Turkey, and the others are two academics working in the economics and business department of the university. In line with the opinion received from three experts, as a result of interview with experts, the sustainable indicator should equal the importance weight. In the application of the MAIRCA method, data of 33 OECD countries are obtained from the OECD and IEA databases. The initial decision matrix for the second quarter of 2020 and the aim of each indicator are presented in Table 2. Due to the nature of the method, the decision-maker should not have a priority in the choice of an alternative. Since there are 33 alternative countries (n), the priority (𝑃𝑟𝐴𝑖 ) of each alternative is calculated as shown in Equation 2. 𝑃𝑟𝐴𝑖 = 1 𝑛 = 1 33 = 0.030 (11) 𝑃𝑟𝐴1 =𝑃𝑟𝐴2 = 𝑃𝑟𝐴3 = 𝑃𝑟𝐴4 =……………………=𝑃𝑟𝐴33 =0.030 (12) The theoretical rating matrix was calculated by Eq. (3) and Eq. (4), respectively. Matrix elements are obtained by multiplying the chosen alternative preferences (𝑃𝑟𝐴𝑖 ) and the coefficients (𝑤𝑖 ) of the weights of the indicators. Experts assumed that the indicator weights were of equal coefficient. The theoretical rating matrix (𝑇𝑝) is shown in Table A1 (Appendix). After this matrix was calculated, the real evaluation matrix (𝑇𝑟 ) was created, as given in Table A2 (Appendix). The actual evaluation matrix element is found by multiplying the theoretical rating matrix element with the normalized start matrix element. The normalized initial matrix was calculated using the Eq. (6) and Eq. (7). The total gap matrix (G) was obtained by subtracting the real rating matrix (𝑡𝑟𝑖𝑗 ) from the theoretical rating matrix (𝑡𝑝𝑖𝑗 ), as shown in Eq. (8) and Eq. (9). It is preferred that the gap value be close to zero. The gap matrix is shown in Table A3 in Annex. In the last step of the method, by using the total gap matrix in Table A3 (Appendix), the criterion function values of decision alternatives were calculated by using Equation (10). The function values (𝑄𝑖 ) of the criteria obtained for the second quarter of 2019 with the MAIRCA method and the ranking of the OECD countries are shown in Table 3. Similar steps were followed using data from the second quarter of 2020, when COVID-19 started to spread worldwide, and the ranking of the sustainability performances of OECD countries obtained by MAIRCA method for the second quarter of 2019 and 2020 is shown in Figure 1. According to the results given in Table 3, the country with the best sustainability performance among OECD countries is the United States. This country is followed by Japan, Germany, Canada, Korea, and the Netherlands, respectively. Kaya/Oper. Res. Eng. Sci. Theor. Appl. 3 (3) (2020) 101-122 112 Table 3. The ranking of OECD countries using the MAIRCA method (Q2/2020) Countries Qi Rank Countries Qi Rank United States 0.0090 1 Slovak Republic 0.0136 18 Germany 0.0091 2 Austria 0.0137 19 Japan 0.0095 3 Portugal 0.0138 20 Korea 0.0114 4 Finland 0.0138 21 Netherlands 0.0117 5 Sweden 0.0139 22 France 0.0119 6 Slovenia 0.0139 23 Switzerland 0.0121 7 Latvia 0.0146 24 United Kingdom 0.0123 8 Iceland 0.0146 25 Italy 0.0123 9 Mexico 0.0147 26 Belgium 0.0127 10 Spain 0.0148 27 Canada 0.0129 11 Hungary 0.0148 28 Czech Republic 0.0130 12 Greece 0.0150 29 Australia 0.0131 13 Estonia 0.0153 30 Denmark 0.0133 14 Lithuania 0.0157 31 Ireland 0.0133 15 Chile 0.0166 32 Norway 0.0135 16 Turkey 0.0249 33 Poland 0.0135 17 The sustainability performance rankings of the countries in the second quarter of 2020, in which the COVID-19 pandemic spread worldwide, were compared in the same period of the previous year, as shown in Figure 2. According to the results obtained with MAIRCA in the second quarter of 2020, the USA belongs to the best sustainable development level among alternative countries. This country is followed by Germany, Japan, Korea and the Netherlands, respectively. Countries with the worst sustainable development performance of the same period, the lowest ranking countries in terms of sustainability performance, are Turkey (33rd), Chile (32nd), Lithuania (31st), Estonia (30th), and Greece (29th). To examine how the COVID-19 pandemic has affected the sustainable development goals of OECD countries, data from the same period of the previous year were used and the MAIRCA method was resolved again. The comparison of the sustainable performance levels for both quarters is shown in Figure 2. Accordingly, the country with the highest sustainable development performance in April-May-June 2019 was America, followed by Japan, Germany, Canada and Korea, respectively. In the ranking results, Canada ranked 4th in 2019, ranked 11th during the pandemic period. The pandemic has been shown to seriously affect Canada's level of sustainable development. It can be seen that for Hungary, Turkey, Greece, Lithuania, Australia, Denmark, Norway, Poland, Sweden, in terms of development sustainability, rankings are stable. Evaluation of the Effect of COVID-19 on Countries’ Sustainable Development Level: A comparative MCDM framework 113 Figure. 2. Comparison of sustainability performance rankings (Q2/2019-Q2/2020) 5. Sensitivity Analysis The reliability of the results obtained from the MAIRCA model should be tested to ensure the validity of the selected alternatives. For this purpose, the reliability and validity of the model were analyzed by using the MABAC method and WASPAS method. Results obtained with MAIRCA, MABAC and WASPAS methods are quite similar to each other. A comparison of results obtained using three MCDM methods are illustrated in Figure 3 and Figure 4, respectively. As can be seen from Figure 3, in all three methods, the United States has the best sustainable development performance. In the second quarter of 2019, Japan, Germany, Canada and Korea have the same rank in all three methods. The Spearman correlation coefficient was used to determine the relationships between these methods. The Spearman correlation coefficient is used to measure the similarity between two group rankings. This method with a higher Spearman's rank relationship coefficient is accepted to be more significant than one with a lower Spearman's rank connection coefficient since it has better concurrences with other MCDM methods (Gang Kou, Yanqun Lu, Yi Peng, & Yong Shi, 2012). Spearman correlation coefficients for both years are shown in Table 4. According to the validity results, the correlation coefficient is above 87.2% and it has a high correlation. This confirms that the MAIRCA method is in agreement with other MCDM methods and its results are reliable. 0 5 10 15 20 25 30 35 Q2/2019 Q2/2020 Kaya/Oper. Res. Eng. Sci. Theor. Appl. 3 (3) (2020) 101-122 114 Figure 3. The ranking of OECD countries (Q2/2019) Figure 4. The ranking of OECD countries (Q2/2020) Table 4. Correlation values of methods Spearman's coefficient MABAC WASPAS Average value MAIRCA (Q2/2019) 1.000 0.872 0.936 MAIRCA (Q2/2020) 0.644 0.881 0.762 0 5 10 15 20 25 30 35 MAIRCA MABAC WASPAS 0 5 10 15 20 25 30 35 A u st ra li a A u st ri a B e lg iu m C a n a d a C h il e C z e c h R e p u b li c D e n m a rk E st o n ia F in la n d F ra n c e G e rm a n y G re e c e H u n g a ry Ic e la n d Ir e la n d It a ly Ja p a n K o re a L a tv ia L it h u a n ia M e x ic o N e th e rl a n d s N o rw a y P o la n d P o rt u g a l S lo v a k R e p u b li c S lo v e n ia S p a in S w e d e n S w it z e rl a n d T u rk e y U n it e d K in g d o m U n it e d S ta te s MAIRCA MABAC WASPAS Evaluation of the Effect of COVID-19 on Countries’ Sustainable Development Level: A comparative MCDM framework 115 6. Wilcoxon Signed-Rank Test The Non-Parametric Wilcoxon signed-rank test was applied to determine whether there is a significant difference between the second quarter of 2020 and the same quarter of the previous year in terms of the indicators of the sustainable development of OECD countries. It can be clearly seen in Table 5, p values of all indicators are less than 0.05 value. According to Wilcoxon test hypothesis, if p-value is less than zero, the null hypothesis is rejected and it is concluded that there is a significant difference between the period (April-June 2019) and (April-June 2020). Accordingly, Table 5 shows that as the p values of “Total Electricity Production (I1), Renewable Energy Production (I2), Merchandise Trade (I3), Customer Price Index (I4), Analytical house rent price indicators (I5), GDP (I6)", Producer Price Index (I7), Unemployment rate, aged 15 and over (I8) are less than 0.05, results demonstrated that there are differences between before and during the COVID-19 pandemic. Test results are clearly expressed in Table 5 and 6, respectively. Table 5. Wilcoxon signed-rank test results I1(2020)- I1(2019) I2(2020) - I2(2019) I3(2020) - I3(2019) I4(2020)- I4(2019) I5(2020)- I5(2019) I6(2020- I6(2019) I7(2020)- I7(2019) I8(2020)- I8(2019) Z -3.815b -2.10b -5.012b -2.124c -3.293c -5.012b -3.475c -3.726c p 0.000 0.036 0.000 0.034 0.001 0.000 0.001 0.000 a. Wilcoxon Signed-Rank Test, b. Based on positive ranks, c. Based on negative ranks As seen in Table 6, the total electricity production of 26 OECD countries decreased in the second quarter of 2020. It is observed that the COVID-19 pandemic has negatively affected the energy production of OECD countries. However, this situation has changed in the amount of renewable energy generation. In the second quarter of 2020, which was heavily affected by the pandemic, the amount of renewable energy production of 22 countries increased compared to the second quarter of the previous year. It is interesting to note that COVID-19 pandemic has a positive effect on renewable energy goals. The global COVID-19 novel coronavirus pandemic has severe negative impacts on the global economy. GDP is an important indicator to bring coherence to the sustainable development goals. When the result on house rent prices were analyzed, the consumer price index of 27 countries increased compared to the second quarter of 2019. According to Table 6, Merchandise trade and GDP of all OECD countries plunged in the second quarter of 2020 as compared to the same period last year. The COVID-19 has prevented countries from achieving their sustainable development goals. The producer price index means the average change over time in selling prices received by domestic producers of goods and services. The producer price index of 29 countries decreased compared to the same quarter of the previous year. The other important sustainable development indicator is the unemployment rate; the test result indicates that the unemployment rate of 26 countries has increased compared to the same period of the previous year. Coronavirus has hit unemployment in OECD countries. The results found that all sustainable development indicators, except renewable energy production, have been severely affected by the COVID-19 pandemic. Kaya/Oper. Res. Eng. Sci. Theor. Appl. 3 (3) (2020) 101-122 116 Table 6. Cash Ratio Ranks Number Mean Rank Sum of Ranks I1 (2019) – I1(2020) Total Electricity production Negative Ranks 26 19.00 494.0 Positive Ranks 7 9.57 67.0 I2 (2019) – I1(2020) Renewable energy Negative Ranks 11 14.82 163.0 Positive Ranks 22 18.09 398.0 I3 (2019) – I1(2020) Merchandise trade Negative Ranks 33 17.00 561.0 Positive Ranks 0 0.00 0 I4 (2019) – I1(2020) CPI Negative Ranks 12 12.54 150.5 Positive Ranks 20 18.88 377.5 I5 (2019) – I1(2020) Rent price Negative Ranks 4 20.00 80.0 Positive Ranks 27 15.41 416.0 I6 (2019) – I1(2020) GDP Negative Ranks 33 17.00 561.0 Positive Ranks 0 0.00 0.0 I7 (2019) – I1(2020) Producer price index Negative Ranks 29 17.59 475.0 Positive Ranks 6 14.33 86.0 I8 (2019) – I1(2020) Unemployment rate Negative Ranks 7 10.29 72.0 Positive Ranks 26 18.81 489.0 7. Results and Limitations World economies have faced serious health problems and socio-economic crises due to the COVID-19 pandemic. The COVID-19 pandemic continues to threaten life, to suppress the world economy, and to have a profound impact on social and environmental issues. National and international community organizations emphasized that countries should pay more attention to sustainable development goals in the post-COVID-19 recovery phase in order to reduce the destructive effect of the COVID-19 crisis. In this study, the effect of the COVID-19 pandemic on the sustainable development of OECD countries was investigated with a novel MCDM method. For this purpose, the MAIRCA method was used to rank the sustainability performance of OECD countries and test its validity and reliability with MABAC and WASPAS methods. Moreover, statistical analysis was implemented and obtained results were discussed in view of sustainable development. The analysis leads to the following conclusions: United States, Germany, Japan, France, and South Korea are with the best development performance while countries with the worst performance are Turkey, Chile, Lithuania, and Estonia for the same quarter of 2019 and 2020. Developed countries are in the top position in the ranking of sustainable development performance compared to developing countries, and this situation did not change with the appearance of the COVID-19 pandemic; the rankings score was the same. In order to test the validity and effectiveness of the MAIRCA method, the ranking results of MAIRCA were compared with the results obtained from novel MCDM models such as MABAC and WASPAS. It has been observed that all MCDM methods used give effective results to determine the ranking of countries under a sustainable development level. Furthermore, a Non-parametric Wilcoxon Signed- Rank test was used to determine whether the direction of changes of each sustainable development indicator was different between the pre-COVID-19 and COVID-19. Evaluation of the Effect of COVID-19 on Countries’ Sustainable Development Level: A comparative MCDM framework 117 Accordingly, results demonstrated that there were significant differences between before and during the COVID-19 pandemic. Importantly, our results provide evidence that, except for renewable energy production, all sustainable indicators have adversely been affected by the COVID-19 pandemic. However, this study has revealed that COVID 19 has had an innovator effect by changing the direction of energy production resources. The pandemic has tripped the scale in favor of renewable energy. There are a number of limitations for this study. One of the main limitations is the missing dataset. Due to the continuing COVID-19 pandemic, there are missing and uncompleted sustainable development indicators such as the "number of hospital beds", "attendance at school", "inequality in education", "life expectancy", "gender inequality", etc. Further study will reevaluate with a different type of sustainable indicators. Another limitation of the study is that it only takes into account the impact of the pandemic on OECD countries' sustainable development performances. In the future study, new research is planned with different countries included in OPEC, G20, and BRIC countries. Appendix Table A1. Theoretical Evaluation Matrix (Tp) Countries I1 I2 I3 I4 I5 I6 I7 I8 Australia 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 Austria 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 Belgium 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 Canada 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 Chile 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 Czech Republic 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 Denmark 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 Estonia 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 Finland 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 France 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 Germany 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 Greece 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 Hungary 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 Iceland 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 Ireland 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 Italy 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 Japan 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 Korea 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 Latvia 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 Lithuania 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 Mexico 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 Netherlands 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 Norway 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 Poland 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 Portugal 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 Slovak Republic 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 Slovenia 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 Spain 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 Sweden 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 Kaya/Oper. Res. Eng. Sci. Theor. Appl. 3 (3) (2020) 101-122 118 Switzerland 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 Turkey 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 United Kingdom 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 United States 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 0.0038 Table A2. Real Evaluation Matrix (Tr) Countries I1 I2 I3 I4 I5 I6 I7 I8 Australia 0.0002 0.0002 0.0006 0.0035 0.0030 0.0036 0.0033 0.0030 Austria 0.0001 0.0003 0.0004 0.0035 0.0021 0.0037 0.0035 0.0032 Belgium 0.0001 0.0001 0.0010 0.0034 0.0029 0.0037 0.0032 0.0029 Canada 0.0005 0.0016 0.0011 0.0034 0.0028 0.0035 0.0035 0.0029 Chile 0.0001 0.0001 0.0001 0.0032 0.0018 0.0037 0.0034 0.0026 Czech Republic 0.0001 0.0000 0.0005 0.0034 0.0024 0.0037 0.0036 0.0038 Denmark 0.0000 0.0001 0.0002 0.0037 0.0028 0.0037 0.0035 0.0031 Estonia 0.0000 0.0000 0.0000 0.0032 0.0008 0.0038 0.0033 0.0031 Finland 0.0001 0.0001 0.0002 0.0037 0.0025 0.0037 0.0035 0.0026 France 0.0005 0.0005 0.0013 0.0036 0.0031 0.0032 0.0036 0.0022 Germany 0.0005 0.0010 0.0034 0.0035 0.0028 0.0030 0.0035 0.0035 Greece 0.0000 0.0001 0.0001 0.0038 0.0038 0.0037 0.0035 0.0000 Hungary 0.0000 0.0000 0.0003 0.0033 0.0014 0.0037 0.0031 0.0034 Iceland 0.0000 0.0001 0.0000 0.0033 0.0016 0.0038 0.0038 0.0034 Ireland 0.0000 0.0000 0.0004 0.0038 0.0020 0.0037 0.0037 0.0030 Italy 0.0002 0.0005 0.0012 0.0037 0.0031 0.0033 0.0036 0.0018 Japan 0.0008 0.0018 0.0017 0.0038 0.0032 0.0028 0.0037 0.0037 Korea 0.0005 0.0001 0.0013 0.0036 0.0029 0.0034 0.0036 0.0033 Latvia 0.0000 0.0000 0.0000 0.0033 0.0026 0.0038 0.0033 0.0027 Lithuania 0.0000 0.0000 0.0001 0.0032 0.0012 0.0038 0.0034 0.0028 Mexico 0.0003 0.0003 0.0011 0.0027 0.0024 0.0033 0.0026 0.0034 Netherlands 0.0001 0.0001 0.0016 0.0035 0.0026 0.0036 0.0033 0.0035 Norway 0.0001 0.0005 0.0002 0.0032 0.0026 0.0037 0.0031 0.0034 Poland 0.0001 0.0001 0.0006 0.0035 0.0022 0.0036 0.0033 0.0035 Portugal 0.0000 0.0001 0.0001 0.0036 0.0026 0.0037 0.0035 0.0026 Slovak Republic 0.0000 0.0000 0.0002 0.0035 0.0031 0.0038 0.0036 0.0029 Slovenia 0.0000 0.0000 0.0001 0.0035 0.0018 0.0038 0.0036 0.0032 Spain 0.0002 0.0004 0.0008 0.0036 0.0030 0.0035 0.0035 0.0008 Sweden 0.0001 0.0004 0.0004 0.0035 0.0029 0.0037 0.0032 0.0027 Switzerland 0.0001 0.0002 0.0006 0.0038 0.0030 0.0037 0.0037 0.0032 Turkey 0.0003 0.0007 0.0004 0.0000 0.0000 0.0033 0.0000 0.0009 United Kingdom 0.0003 0.0004 0.0010 0.0034 0.0029 0.0032 0.0032 0.0033 United States 0.0038 0.0038 0.0038 0.0034 0.0020 0.0000 0.0034 0.0034 Table A3. Total Gap Matrix Countries I1 I2 I3 I4 I5 I6 I7 I8 Australia 0.0036 0.0036 0.0032 0.0003 0.0008 0.0002 0.0005 0.0008 Austria 0.0037 0.0035 0.0034 0.0003 0.0017 0.0001 0.0003 0.0006 Belgium 0.0037 0.0037 0.0028 0.0004 0.0009 0.0001 0.0006 0.0009 Canada 0.0033 0.0022 0.0027 0.0004 0.0010 0.0003 0.0003 0.0009 Chile 0.0037 0.0037 0.0036 0.0006 0.0020 0.0001 0.0004 0.0012 Czech Republic 0.0037 0.0037 0.0033 0.0004 0.0014 0.0001 0.0002 0.0000 Denmark 0.0038 0.0037 0.0035 0.0001 0.0010 0.0001 0.0003 0.0007 Estonia 0.0038 0.0038 0.0038 0.0005 0.0030 0.0000 0.0005 0.0007 Finland 0.0037 0.0036 0.0036 0.0001 0.0012 0.0000 0.0002 0.0012 Evaluation of the Effect of COVID-19 on Countries’ Sustainable Development Level: A comparative MCDM framework 119 France 0.0033 0.0033 0.0024 0.0002 0.0007 0.0006 0.0002 0.0016 Germany 0.0033 0.0028 0.0003 0.0002 0.0010 0.0008 0.0003 0.0003 Greece 0.0038 0.0037 0.0037 0.0000 0.0000 0.0001 0.0002 0.0038 Hungary 0.0038 0.0038 0.0035 0.0005 0.0024 0.0001 0.0007 0.0004 Iceland 0.0038 0.0037 0.0038 0.0005 0.0021 0.0000 0.0000 0.0003 Ireland 0.0038 0.0038 0.0034 0.0000 0.0018 0.0001 0.0001 0.0008 Italy 0.0035 0.0033 0.0026 0.0001 0.0007 0.0004 0.0002 0.0020 Japan 0.0030 0.0019 0.0021 0.0000 0.0006 0.0010 0.0001 0.0001 Korea 0.0033 0.0036 0.0025 0.0002 0.0009 0.0004 0.0002 0.0005 Latvia 0.0038 0.0038 0.0038 0.0005 0.0011 0.0000 0.0005 0.0011 Lithuania 0.0038 0.0038 0.0037 0.0006 0.0026 0.0000 0.0004 0.0010 Mexico 0.0035 0.0035 0.0027 0.0011 0.0014 0.0005 0.0012 0.0004 Netherlands 0.0037 0.0037 0.0022 0.0003 0.0012 0.0002 0.0005 0.0003 Norway 0.0037 0.0033 0.0036 0.0006 0.0012 0.0001 0.0007 0.0004 Poland 0.0037 0.0037 0.0032 0.0003 0.0016 0.0002 0.0005 0.0003 Portugal 0.0037 0.0037 0.0036 0.0002 0.0012 0.0001 0.0003 0.0011 Slovak Republic 0.0038 0.0038 0.0036 0.0003 0.0007 0.0000 0.0002 0.0009 Slovenia 0.0038 0.0038 0.0037 0.0002 0.0020 0.0000 0.0002 0.0006 Spain 0.0036 0.0033 0.0030 0.0002 0.0008 0.0003 0.0003 0.0030 Sweden 0.0036 0.0034 0.0034 0.0003 0.0009 0.0001 0.0006 0.0011 Switzerland 0.0037 0.0036 0.0032 0.0000 0.0008 0.0001 0.0001 0.0006 Turkey 0.0035 0.0031 0.0034 0.0038 0.0038 0.0004 0.0038 0.0029 United Kingdom 0.0035 0.0034 0.0028 0.0004 0.0009 0.0006 0.0005 0.0004 United States 0.0000 0.0000 0.0000 0.0004 0.0018 0.0038 0.0003 0.0004 References Arsić, S.N., Pamučar, D., Suknović, M., Janošević, M., (2019). Menu evaluation based on rough MAIRCA and BW methods. Serbian Journal of Management 14, 27–48. https://doi.org/10.5937/sjm14-18736 AYÇİN, E., (2020). Using CRITIC and MAIRCA Methods in Personnel Selection Process. The Business Journal 1, 1–12. Badi, I., Ballem, M., (2018). Supplier selection using the rough BWM-MAIRCA model: A case study in pharmaceutical supplying in Libya. Decision Making: Applications in Management and Engineering 1, 16–33. Balcerzak, A.P., Pietrzak, M.B., (2016). Application of TOPSIS Method for Analysis of Sustainable Development in European Union Countries Institute of Economic Research Working Papers. Bali Swain, R., Yang-Wallentin, F., (2020). Achieving sustainable development goals: predicaments and strategies. International Journal of Sustainable Development & World Ecology 27, 96–106. Bedford, J., Enria, D., Giesecke, J., Heymann, D.L., Ihekweazu, C., Kobinger, G., Lane, H.C., Memish, Z., Oh, M., Sall, A.A., Schuchat, A., Ungchusak, K., Wieler, L.H., (2020). COVID- Kaya/Oper. Res. Eng. Sci. Theor. Appl. 3 (3) (2020) 101-122 120 19: towards controlling of a pandemic. The Lancet 395, 1015–1018. https://doi.org/10.1016/S0140-6736(20)30673-5 Boral, S., Howard, I., Chaturvedi, S.K., McKee, K., Naikan, V.N.A., (2020). An integrated approach for fuzzy failure modes and effects analysis using fuzzy AHP and fuzzy MAIRCA. Engineering Failure Analysis 108, 104195. CDC, (2020). International Locations with Confirmed COVID-19 Cases. Centers for Disease Control and Prevention. Available at: https://www.cdc.gov/coronavirus/2019-ncov/global-covid-19/world-map.html (accessed 5.29.20). Chatterjee, K., Pamucar, D., Zavadskas, E.K., (2018). Evaluating the performance of suppliers based on using the R’AMATEL-MAIRCA method for green supply chain implementation in electronics industry. Journal of Cleaner Production 184, 101–129. https://doi.org/10.1016/j.jclepro.2018.02.186 Chatterjee, P., Mandal, N., Dhar, S., Chatterjee, S., Chakraborty, S., (2020). A novel decision-making approach for light weight environment friendly material selection. Materials Today: Proceedings, 2nd International Conference on Materials Manufacturing and Modelling, ICMMM – 2019, VIT University, Vellore, 29th - 31st March 2019 22, 1460–1469. https://doi.org/10.1016/j.matpr.2020.01.504 Chen, N., Zhou, M., Dong, X., Qu, J., Gong, F., Han, Y., Qiu, Y., Wang, J., Liu, Y., Wei, Y., (2020). Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study. The Lancet 395, 507– 513. Coronavirus Update (Live): 13,612,973 Cases and 584,165 Deaths from COVID-19 Virus Pandemic - Worldometer, Available at: https://www.worldometers.info/coronavirus/?utm_campaign=homeADemocracyno w(2020)%20dvegas1? (Accessed 7.15.2020). COVID-19, (2020). IEA. URL https://www.iea.org/topics/covid-19 (accessed 7.16.20). Deloitte Annual Turkish M&A Review, 2019. URL https://www2.deloitte.com/tr/en.html (accessed 5.29.20). Ding, L., Shao, Z., Zhang, H., Xu, C., Wu, D., (2016). A comprehensive evaluation of urban sustainable development in China based on the TOPSIS-entropy method. Sustainability 8, 746. Ekinci, E.B.M., Can, G.F., (2018). Evaluation of Workers’ Ergonomıc Rısk Levels Consıderıng Workıng Postures And Perceıved Workload Wıth Multı-Crıterıa Decısıon Makıng Approach. Ergonomics 1(2), 77–91. Gang Kou, Yanqun Lu, Yi Peng, Yong Shı, (2012). Evaluation of Classifıcation Algorithms Using MCDM and Rank Correlation. International Journal of Information Technology & Decision Making 11, 197–225. Gaspar, J. dos S., Marques, A.C., Fuinhas, J.A., (2017). The traditional energy-growth nexus: A comparison between sustainable development and economic growth approaches. Ecological Indicators 75, 286–296. Gigović, L., Pamučar, D., Bajić, Z., Milićević, M., (2016). The combination of expert judgment and GIS-MAIRCA analysis for the selection of sites for ammunition depots. Sustainability 8, 372. Gul, M., Ak, M.F., (2020). Assessment of occupational risks from human health and environmental perspectives: a new integrated approach and its application using Evaluation of the Effect of COVID-19 on Countries’ Sustainable Development Level: A comparative MCDM framework 121 fuzzy BWM and fuzzy MAIRCA. Stochastic Environmental Research And Rısk Assessment. Hui, D.S., Azhar, E.I., Madani, T.A., Ntoumi, F., Kock, R., Dar, O., Ippolito, G., Mchugh, T.D., Memish, Z.A., Drosten, C., Zumla, A., Petersen, E., (2020). The continuing 2019-nCoV epidemic threat of novel coronaviruses to global health — The latest 2019 novel coronavirus outbreak in Wuhan, China. International Journal of Infectious Diseases 91, 264–266. https://doi.org/10.1016/j.ijid.2020.01.009 Kothari, R., Tyagi, V.V., Pathak, A., (2010). Waste-to-energy: A way from renewable energy sources to sustainable development. Renewable and Sustainable Energy Reviews 14, 3164–3170. Lai, C.-C., Shih, T.-P., Ko, W.-C., Tang, H.-J., Hsueh, P.-R., (2020). Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and coronavirus disease-2019 (COVID-19): The epidemic and the challenges. International Journal of Antimicrobial Agents 55, 105924. Li, Q., Guan, X., Wu, P., Wang, X., Zhou, L., Tong, Y., Ren, R., Leung, K.S.M., Lau, E.H.Y., Wong, J.Y., Xing, X., Xiang, N., Wu, Y., Li, C., Chen, Q., Li, D., Liu, T., Zhao, J., Liu, M., Tu, W., Chen, C., Jin, L., Yang, R., Wang, Q., Zhou, S., Wang, R., Liu, H., Luo, Y., Liu, Y., Shao, G., Li, H., Tao, Z., Yang, Y., Deng, Z., Liu, B., Ma, Z., Zhang, Y., Shi, G., Lam, T.T.Y., Wu, J.T., Gao, G.F., Cowling, B.J., Yang, B., Leung, G.M., Feng, Z., (2020). Early Transmission Dynamics in Wuhan, China, of Novel Coronavirus–Infected Pneumonia. New England Journal of Medicine 382, 1199–1207. https://doi.org/10.1056/NEJMoa2001316 Mateusz, P., Danuta, M., Ma\lgorzata, \Latuszyńska, Mariusz, B., Kesra, N., (2018). TOPSIS and VIKOR methods in study of sustainable development in the EU countries. Procedia Computer Science 126, 1683–1692. News Article | World Travel & Tourism Council (WTTC), (2020). World Travel Tourism Council, UK. What is OECD?, (2020). Ekonomist. Available at: https://www.ekonomist.com.tr/ekonomist/oecd-nedir.html (Accessed 6.20.2020). Pamučar, D., Mihajlović, M., Obradović, R., Atanasković, P., (2017a). Novel approach to group multi-criteria decision making based on interval rough numbers: Hybrid DEMA℡-ANP-MAIRCA model. Expert Systems with Applications 88, 58–80. Pamučar, D., Mihajlović, M., Obradović, R., Atanasković, P., (2017b). Novel approach to group multi-criteria decision making based on interval rough numbers: Hybrid DEMATEL-ANP-MAIRCA model. Expert Systems with Applications 88, 58–80. https://doi.org/10.1016/j.eswa.2017.06.037 Pamučar, D.S., Ćirović, G., Božanić, D., (2019). Application of interval valued fuzzy- rough numbers in multi-criteria decision making: the IVFRN-MAIRCA model. Yugoslav Journal of Operations Research 29, 221–247. Pamucar, D.S., Pejcic Tarle, S., Parezanovic, T., (2018a). New hybrid multi-criteria decision-making DEMATEL MAIRCA model: sustainable selection of a location for the development of multimodal logistics centre. Economic research-Ekonomska istraživanja 31, 1641–1665. Pamučar, D.S., Savin, L.M., (2020). Multiple-criteria model for optimal off-road vehicle selection for passenger transportation: BWM-COPRAS model. Vojnotehnički glasnik 68, 28–64. Kaya/Oper. Res. Eng. Sci. Theor. Appl. 3 (3) (2020) 101-122 122 Pirbasti, F.R.H., Modiri, M., Hafashjani, K.F., Komijan, A.R., (2020). Location of Hospital Waste Disposal Site With FSWARA-GIS-MAIRCA Hybrid Algorithm. Sathaye, J., Lucon, O., Rahman, A., Christensen, J., Denton, F., Fujino, J., Heath, G., Mirza, M., Rudnick, H., Schlaepfer, A., Shmakin, A., Angerer, G., Bauer, C., Bazilian, M., Brecha, R., Burgherr, P., Clarke, L., Creutzig, F., Edmonds, J., Hagelüken, C., Hansen, G., Hultman, N., Jakob, M., Kadner, S., Lenzen, M., Macknick, J., Masanet, E., Nagai, Y., Olhoff, A., Olsen, K., Pahle, M., Rabl, A., Richels, R., Roy, J., Schei, T., Stechow, C. von, Steckel, J., Warner, E., Wilbanks, T., Zhang, Y., (2011). Renewable Energy in the Context of Sustainable Development. Renewable Energy Sources and Climate Change Mitigation. Sustainable Development Goals, (2020). United Nations Sustainable Development. URL Available at: https://www.un.org/sustainabledevelopment/sustainable- development-goals/ (accessed 11.21.20). Ulutaş, A., (2019). The Selection of Caterıng Fırm with Swara And Maırca Methods.Business & Management Studies: An International Journal. 7(4), 1467–1479. https://doi.org/10.15295/bmij.v7i4.1166 UNDP, (2020). UNDP. URL Available at: https://www.tr.undp.org/content/turkey/tr/home/presscenter/pressreleases/202 0/04/COVID19-haritasi.html (accessed 7.16.20). Wang, C., PW. Horby, FG Hayden, GF Gao, (2020). A novel coronavirus outbreak of global health concern. The Lancet 395, 470-473. WHO, (2020). Naming the coronavirus disease (COVID-19) and the virus that causes it. Available at: https://www.who.int/emergencies/diseases/novel-coronavirus- 2019/technical-guidance/naming-the-coronavirus-disease-(covid-2019)-and-the- virus-that-causes-it (Accessed 7.8.20). World Economic Outlook, (2020). The Great Lockdown, Available at: https://www.imf.org/en/Publications/WEO/Issues/2020/04/14/weo-april-2020 (accessed 5.31.20). Zavadskas, E.K., Cavallaro, F., Podvezko, V., Ubarte, I., Kaklauskas, A., (2017). MCDM Assessment of a Healthy and Safe Built Environment According to Sustainable Development Principles: A Practical Neighborhood Approach in Vilnius. Sustainability 9, 702. https://doi.org/10.3390/su9050702 © 2020 by the authors. Submitted for possible open access publication under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/). Evaluation of the Effect of COVID-19 on Countries’ Sustainable Development Level: A comparative MCDM framework Sema Kayapinar Kaya 1. Introduction 2. Literature review 3. MAIRCA method 4. Evaluate the effects of the COVID-19 pandemic on sustainable development performance of OECD countries 5. Sensitivity Analysis 6. Wilcoxon Signed-Rank Test 7. Results and Limitations Appendix References