Operational Research in Engineering Sciences: Theory and Applications
Vol. 4, Issue 3, 2021, pp. 59-81
ISSN: 2620-1607
eISSN: 2620-1747
DOI: https://doi.org/10.31181/oresta081221059t
* Corresponding author.
spturkoglu@ybu.edu.tr (S. Türkoğlu), stuzcu@politics.ankara.edu.tr (S. Tuzcu)
ASSESSING COUNTRY PERFORMANCES DURING THE
COVID-19 PANDEMIC: A STANDARD DEVIATION BASED
RANGE OF VALUE METHOD
Serap Pelin Türkoğlu1, Sevgi Eda Tuzcu2*
1 Department of Management and Organization, Ankara Yıldırım Beyazıt University,
Ankara, Turkey
2 Department of Business Administration, Ankara University, Ankara, Turkey
Received: 22 April 2021
Accepted: 24 September 2021
First online: 09 December 2021
Original scientific paper
Abstract: In this paper, we compare the pandemic management performance of 22
countries that belong to the middle-high income class based on criteria including the
pandemic data, population characteristics, and health system capacity. The
management of the COVID-19 pandemic requires considering many and often
conflicting aspects at the same time which necessitates an MCDM approach. We use a
standard deviation (SDV) based range of value (ROV) method which coincides with the
black-box nature of the disease. The weights obtained from the SDV method reveal that
the number of COVID-19 deaths, current health expenditure, and deaths due to
cardiovascular diseases are the most important criteria. The ROV method indicates that
most Asian countries are ranked in higher positions due to their strong healthcare
systems and quick implementation of social distancing rules. The lowest performances
belong to Bulgaria, Montenegro, and Bosnia and Herzegovina. They have experienced
an elevated number of deaths due to having an elderly population and inefficient usage
of healthcare resources. We also show that extreme poverty is an important determinant
of country performance. In countries where poverty is higher, as the case with Indonesia,
implementing the social distancing rules becomes almost impossible which affects the
overall country performance significantly.
Keywords: COVID-19 pandemic, ROV method, SDV method, MCDM, middle-high income
countries
1. Introduction
It has been more than a year since the World Health Organization first declared the
new Coronavirus disease (COVID-19) was a pandemic. After one year and several
Türkoğlu and Tuzcu/Oper. Res. Eng. Sci. Theor. Appl. 4 (3) (2021) 59-81
60
waves, COVID-19 caused 113,051,293 cases and more than 2,500,000 deaths1.
Although we have learned many things about the disease, and several COVID-19
vaccines have been developed, the importance of the studies to clarify the effects of
the disease has not been reduced. Any research that helps reduce the burden on the
healthcare system and flatten the spread rate is still valuable. In this study, we aim to
compare the performance of a relatively homogenous group of countries in their fight
against the COVID-19 during the ongoing pandemic. More specifically, we rank the
pandemic management performance of 22 countries that belong to the middle-high
income class according to the World Bank classifications based on many criteria,
namely pandemic data, population characteristics, and health system capacity. The
existence of various factors and often conflicting criteria at the same time makes the
issue of ranking country performances as a Multi-Criteria Decision Making (MCDM)
problem. For this aim, we use a combination of two MCDM techniques, namely the
standard deviation approach (SDV) for the criteria weights and range of value (ROV)
method in order to compare country performances against COVID-19.
Regarding the aim of our study, we note two related fields of research: The first
field investigates how well countries all over the world cope with the pandemic.
Adabavazeh et al. (2020) note that benchmarking is essential for the healthcare
systems to improve their working process and to be prepared for future public health
crises. Hence, it is natural to observe a specific line of research in the COVID-19
literature. These studies concentrate on the performance rankings of different
countries with various methodologies. Jamison et al. (2020) examine the
performances of 35 countries according to the doubling times of confirmed cases and
deaths at three different points in time during the beginning of the pandemic. They
demonstrate that although the variation among the cross-country performances is
modest initially, the difference between countries gets wider between later days.
Aydin and Yurdakul (2020) rank the country performances against COVID-19 by using
an extension of data envelopment analysis (DEA) and different machine learning
algorithms. They aim to show which factors affect the number of confirmed cases and
deaths the most. Maroko et al. (2020) aim to compare the neighborhoods in New York
City, the USA, in terms of population characteristics. Olivia et al. (2020) evaluate the
current pandemic management of Indonesia by comparing its performance with
Lombardy (Italy) and New York (the USA) as well as other Asian countries.
The second group of related works is the ones that apply different decision-making
approaches to COVID-19 studies. Among those studies, Shirazi et al., (2020) rank the
performance of hospitals in Iran and their capability to meet patients’ needs during
the pandemic by using FAHP – PROMETHEE methods. Samanlioglu and Kaya (2020)
evaluate the intervention strategies that countries have adopted to flatten the
epidemiological curve by using the hesitant fuzzy AHP method. Kayapinar Kaya (2020)
assesses the impact of COVID-19 on the sustainable development levels of the OECD
countries by employing the MAIRCA technique and compares the findings with two
other MCDM models, i.e. MABAC and WASPAS. Yiğit (2020) analyzed the performance
of OECD countries during the current pandemic by employing the TOPSIS approach.
In this study, the healthcare system performance proxies, such as health expenditures,
the number of medical staff, and COVID-19 related indicators are employed as criteria,
1 Data from: https://www.worldometers.info/coronavirus/ Access Date: 24.02.2021
https://www.worldometers.info/coronavirus/
Assessing country performances during the COVID-19 pandemic: a standard deviation based
range of value method
61
while 36 OECD countries constitute the alternatives. This study reveals that Asian
Pacific countries are the more successful ones in this struggle whereas the European
and Middle Eastern countries demonstrate bad performance.
Determining a good country performance against COVID-19 requires considering
many types of criteria at the same time. Population characteristics, such as age,
accompanying chronic diseases, the level of poverty are important factors in this
struggle. Still, the biggest burden is on the healthcare system components. Among the
primary determinants of pandemic management success, we can observe the timely
detection of the newly infected ones, providing good healthcare, and limiting the
number of deaths. The existence of various and conflicting criteria creates a need for
MCDM techniques. Hence, we use two MCDM techniques combinedly, namely the SDV
and the ROV approaches, to determine the rankings of the countries. To the extent of
our knowledge, this is the first study that applies an SDV based ROV method to analyze
the country’s performances in the battle against the current pandemic. By doing so,
we aim to contribute to both lines of the above-mentioned researches.
MCDM is a very common methodology to rank the decision options according to a
group of criteria (Hajkowicz & Higgins, 2008). This particular nature allows us to
compare the country’s performances in the fight against COVID-19. In this sense, the
aim of our study is related to the paper by Yiğit (2020). Here, we must mention the
significant differences between this paper and ours. Yiğit (2020) considers the early
phases of the pandemic with only healthcare-related criteria. However, COVID-19
created a very fast-changing environment in all areas. Therefore, re-evaluating the
performance of the countries and showing the differences between the ongoing
situation and the early stages even for a short period of time still adds to the efforts
that aim to decrease its negative effects. It is also known that the struggle against the
COVID-19 does not only depend on healthcare system, but also the characteristics of
vulnerable population. These criteria must be added into country comparisons. Unlike
Yiğit (2020), we consider population-related criteria such as the prevalence of chronic
diseases and the percentage of elderly population. Besides, we focus on a different and
more homogenous group of countries. It is natural to draw different conclusions from
this comparison relative to a more heterogeneous country group. Last, we use a
different combination of MCDM techniques. As a result, we believe that benchmarking
the countries in the first year of the pandemic is still important to suggest policy
changes for the worst-performing ones.
Bedford et al. (2020) indicate that the struggle against the COVID-19 pandemic is
harder for the countries in the low and middle-income groups than the higher-income
group countries. The literature, however, mostly concentrates on the USA and
European countries where the pandemic hit hard initially. The burden of the pandemic
on the healthcare workers and units is devastating even for the better prepared
higher-income countries. De Nardo et al. (2020) state that the situation is, even more,
overwhelming for low- and middle-income countries where the healthcare facilities
are limited and mobility restrictions are difficult to be applied. In this study, we
consider the middle-high income group countries, a less examined sample that also
includes the origin of the pandemic, i.e. mainland China. We believe that such a
comparison will help more disadvantaged countries to be prepared for future
infection spreads, to decrease the economic and social effects of pandemics, and to
allocate healthcare resources efficiently.
Türkoğlu and Tuzcu/Oper. Res. Eng. Sci. Theor. Appl. 4 (3) (2021) 59-81
62
Our findings reveal that Asian countries like Thailand, mainland China and
Malaysia produce better rankings in this comparison. These countries are more
experienced in the management of infectious diseases due to previous epidemics, i.e.
SARS. However, Bosnia and Herzegovina, Bulgaria, and Montenegro have more
disadvantaged rankings. Their relative low performance can be attributed to the
higher levels of the elderly population and their related co-morbidities as well as the
inefficient management of the existing healthcare resources.
We clearly show that countries that are successful in pandemic management have
stronger healthcare systems and they have been more prepared for such a disaster.
Countries that are ranked in the lower positions either have limited capacity or cannot
manage their resources efficiently. Delaying to take necessary social distancing
precautions is likely to increase the number of COVID-19 deaths and lowers the
country's performance.
The success in pandemic management also depends on population characteristics.
Having an elderly population increases the overall burden in the healthcare system
because it increases the requirement for long term attention. In distressing times, like
the COVID-19 pandemic, along with the co-morbidities, countries experience many
difficulties to manage the crisis, no matter how high the level of the current
expenditure is. In countries, where poverty is higher, implementing the social
distancing rules becomes almost impossible. With that limited access to clean water
and other hygiene products, the number of cases and deaths rapidly escalates.
Authorities must adopt policies regarding elderly care and people living below the
poverty line.
This paper continues with the literature review section that summarizes the
related lines of research. The data and methodology section explains the details of the
data and the selected MCDM method. Section 4 presents the findings and the policy
discussions based on these results. We compare the robustness of our results in
Section 5. The last section concludes.
2. Related Literature
Since the beginning of the COVID-19 pandemic, there has been vast literature
discussing its effects in every aspect of our lives. Besides discussing the impact of the
pandemic on our social and professional lives, many studies examine the effectiveness
of different precautions to deal with the COVID-19. In this paper, we aim to compare
the performances of countries in the battle against the current pandemic by using a
combination of two MCDM techniques. Therefore, in this literature review we
specifically focus on the two lines of research: First, we look at the literature that
investigates how well countries all over the world cope with the pandemic. In this
group, Jamison et al. (2020); Aydin and Yurdakul (2020); Adabavazeh et al. (2020) are
highlighted. Second, we examine how different decision-making techniques are used
in the COVID-19 studies. In the second group, the studies by De Nardo et al. (2020);
Shirazi, et al. (2020); Yiğit (2020), and Kayapinar Kaya (2020) are stand out among
the others.
The study by Jamison et al. (2020) is one of the first ones that investigate the county
performances against COVID-19. They compare the performance of 35 countries with
different characteristics in the early times of the pandemic depending on the doubling
Assessing country performances during the COVID-19 pandemic: a standard deviation based
range of value method
63
times of new cases and COVID-19 related deaths per 1,000,000 population. They aim
to show which policy choices of different governments provide better health and
economic results. Their results indicate that in the less populated countries, the
pandemic started with more severity, but in time, the more populated countries were
affected more. Brazil and India have always shown a bad performance in terms of
controlling the pandemic. The cases in Iran and Indonesia doubled more quickly in the
later comparisons than the initial times. Turkey showed a good performance and its
ranking is above the average of 35 countries in terms of both new cases and deaths.
Aydin and Yurdakul (2020) and Adabavazeh et al (2020) employ the DEA method
in the analysis of country efficiency. More specifically, Aydin and Yurdakul (2020)
develop a novel three-stage DEA model called WSIDEA to identify country
performances based on the number of COVID-19 cases and deaths and other
population and economic characteristics. They cluster 142 countries into 3 groups and
obtain their efficiencies according to the WSIDEA method. The factors that affect
country efficiency levels are examined with decision trees and random forest
algorithms. Adabavazeh, et al. (2020), on the other hand, employ the traditional DEA
method to analyze the efficiency of healthcare units in 71 different countries where
population, GDP per capita, day of infection, and the total number of cases are inputs
and the number of total recovered patients and total deaths are outputs. They employ
a BCC type output-oriented DEA model. Among them, 16 healthcare units including
those located in mainland China and Iran are found efficient.
The second line of research applies different decision-making tools in COVID-19
studies, such as MCDM techniques, fuzzy sets, and artificial intelligence. These
applications include many different fields, ranging from the prediction of future risks
and the effectiveness of the non-healthcare precautions to medicine selection and
hospital admissions.
Among the examples of fuzzy sets and artificial intelligence applications in the
struggle against COVID-19, the following studies can be examined: Pal et al. (2020)
benefit from artificial intelligence to predict long term country-specific risks and to
group them as high-risk, low-risk, and recovering countries. Majumder et al. (2020)
develop a decision-making tool based on TOPSIS. They try to identify the possible
COVID-19 patients depending on their linguistic information in order to help early
detection of newly infected people and to send them self-isolation or home quarantine.
They replace the Euclidian distance computation of proximity to ideal solutions with
supremum distance. Next, they apply an artificial neural network approach to provide
real-time monitoring for death values. Si et al. (2021) propose a decision-making
approach for appropriate medicine selection by employing fuzzy sets and grey
relational analysis. Mishra et al. (2021), following Si et al. (2021), rank alternative
medical treatments to apply for the mild cases of Covid-19. They employ an ARAS
framework with hesitant fuzzy information. The attribute weights are obtained
through the HF-divergence measure. Khatua et al. (2020) employ granular
differentiability based fuzzy SEIAHRD model to control the current pandemic in India.
Their results indicate that an optimal pandemic control requires more testing to detect
the infections, hospital quarantine, and long-term partial lockdowns in the country.
The following group of studies constitutes examples for the application of MCDM
techniques to the COVID-19 studies. Sayan et al. (2020) employ two different MCDM
Türkoğlu and Tuzcu/Oper. Res. Eng. Sci. Theor. Appl. 4 (3) (2021) 59-81
64
techniques, namely fuzzy PROMETHEE and fuzzy TOPSIS to rank seven diagnostic
alternatives of COVID-19 based on different criteria such as test sensitivity, cost,
usability, false result rates, accessibility, equipment.
Samanlioglu and Kaya (2020) assess the non-healthcare precautions, including
mobility and border restrictions, lockdowns, school closures, declaration of a state of
emergency in the struggle against the current pandemic by employing a hesitant fuzzy
AHP method. De Nardo et al. (2020) use PAPRIKA as an MCDM technique to detect and
prioritize the hospital admissions of COVID-19 patients with the potential of quick
clinical deterioration in Italy. The criteria weights are based on a survey applied to the
experts who deal actively with COVID-19 patients. The PAPRIKA method compares all
combinations of criteria pairs. Shirazi, et al. (2020) examine patient satisfaction under
normal terms and under times of health crisis, such as the current pandemic. To do so,
they rate the hospitals in Iran and determine the factors of patient satisfaction by using
a combination of the Fuzzy AHP-PROMETHEE approach. Kayapinar Kaya (2020)
assesses the sustainable development performance of OECD countries before and
during the COVID-19 pandemic by using MAIRCA as the main analysis method. She
compared the rankings of MAIRCA with two other MCDM techniques namely WASPAS
and MABAC. She showed that although the pandemic affects negatively all countries’
sustainable development levels, developing countries are more negatively impacted.
Sangiorgio and Parisi (2020) develop an index to forecast the contagion risk under
different mobility scenarios in urban areas to support the decision-making process of
local authorities. To do so, they collect data from 257 urban areas in Italy and they
design the problem as AHP based multi-criteria approach. Next, they calibrate the
model by using the GRG-optimization method and compared the results with an
analysis based on the Artificial Neural Networks.
Yiğit (2020) compares the performance of 36 OECD countries by using the TOPSIS
method in their struggle against the current disease. She considers the countries as
alternatives to rank and employs healthcare indicators as equally weighted criteria.
These criteria include the number of COVID-19 patients and deaths, healthcare system
expenditures by governments, and current healthcare capacities, such as hospital beds
and the number of physicians. She states that although one may observe a high
correlation between healthcare expenditures and life expectancy, some countries do
not comply with this anticipation. For example, Portugal, Spain, and Israel are among
the countries with high life expectancy but their healthcare expenditures are lower
than the OECD average. As a result, as indicated by Yiğit (2020), in a worldwide
healthcare crisis, this situation leads to different performance rankings. The findings
of Yiğit (2020) suggest that Asian countries in the OECD sample react more proactively
to the ongoing pandemic relative to European countries and the USA.
The studies applying several decision-making tools on COVID-19 are summarized
below in Table 1.
Our study aims to combine these two lines of research mentioned above. In
particular, we aim to rank the countries in the middle-high income group based on
their performances against the current pandemic while employing MCDM techniques.
Although these techniques provide transparent and consistent comparisons as noted
by De Nardo et al. (2020), the studies applying MCDM to the country performances are
rather scarce. In this sense, our aim coincides most with the study by Yiğit (2020).
Assessing country performances during the COVID-19 pandemic: a standard deviation based
range of value method
65
Table 1. The Studies Applied Decision-Making Tools in the Struggle Against COVID-19
Authors The Aim of the Paper The Decision-Making Tool
Yiğit (2020)
To compare the country
performances based on the
healthcare criteria in the battle
against the current pandemic
TOPSIS
Sayan et al. (2020)
To compare the COVID-19 diagnostic
tool alternatives
Fuzzy PROMETHEE and Fuzzy
TOPSIS
Samanlioglu and Kaya
(2020)
To rank the precautions other than
the healthcare system measures
against COVID-19
Hesitant Fuzzy AHP
De Nardo et al. (2020)
To prioritize the hospital admissions
of not currently but potentially
urgent patients.
PAPRIKA
Kayapinar Kaya (2020)
To rank the sustainable development
levels of countries before and during
the COVID-19 pandemic
MAIRCA results compared
with WASPAS and MABAC
Shirazi, et al. (2020)
To rate the hospitals in Iran
according to the patient satisfaction
under normal and pandemic
conditions
A combination of Fuzzy AHP-
PROMETHEE
Sangiorgio and Parisi
(2020)
To predict the contagion risk of
COVID-19 based on different mobility
restriction scenarios
AHP based multi-criteria
approach and GRG-
optimization method. The
results are compared with
those obtained from ANN.
Khatua et al. (2020)
To find the requirements for an
optimal pandemic control in India
Granular differentiability
based fuzzy SEIAHRD
Majumder et al. (2020)
To detect the potential COVID-19
patients based on their verbal
information
Supremum distance TOPSIS
method combined with ANN
Pal et al. (2020)
To determine the long term COVID-
19 risks of a country
Artificial Intelligence
Si et al. (2021)
To choose appropriate medicine to
treat COVID-19 patients
Fuzzy sets and grey relational
analysis
Mishra et al. (2021)
To rank medical treatment
alternatives of mild COVID-19
patients
ARAS framework with hesitant
fuzzy information
However, there are important differences between these two papers. We consider
a more homogenous and less investigated sample of countries than Yiğit (2020).
Benchmarking of countries is more intuitive when the sample becomes less
heterogeneous. In addition to the healthcare indicators employed in Yiğit (2020), we
also consider population-based criteria, such as the elderly ratio, extreme poverty, and
the existence of co-morbidities in our analysis. These criteria are also known as
important determinants of the number of COVID-19 deaths and cases. Therefore, the
performance comparison of countries in this battle must consider these criteria as
well. The criteria ranking is done based on an objective weighting method called the
SDV approach. The country comparisons are realized based on the ROV method. We
believe that our selection of criteria weighting and alternative ranking methods
overlap the black-box nature of the ongoing pandemic. Last, we employ a later time
period for the ongoing pandemic to compare the performances. Such a comparison in
this fast-changing environment is very valuable.
Türkoğlu and Tuzcu/Oper. Res. Eng. Sci. Theor. Appl. 4 (3) (2021) 59-81
66
3. Data and Methodology
This study collects data for 22 countries that are classified as middle-high income
group by the World Bank2. These countries are Albania, Argentina, Armenia, Bosnia
and Herzegovina, Brazil, Bulgaria, mainland China, Columbia, Costa Rica, Dominican
Republic, Ecuador, Georgia, Indonesia, Iran, Kazakhstan, Malaysia, Mexico,
Montenegro, Paraguay, Russia, Thailand, and Turkey. As of the date of the analysis, this
sample covers 25% of the total cases and 32% of the COVID-19 related deaths
worldwide. These countries create a rather homogeneous group to provide a ranking
for their performances and also contain mainland China, which is the origin of the
pandemic and the most discussed country for its crisis management methods. The
study begins with the identification of performance criteria and application of the SDV
method to weight them. Next, the ROV method will be applied as the main analysis
method, where the results are compared TOPSIS and EDAS approaches. For clarity, we
show the general framework of our work in Figure 1.
Figure 1. General working diagram of the study
2 The World Bank, https://datahelpdesk.worldbank.org/knowledgebase/articles/906519-world-bank-
country-and-lending-groups, Access Date: 24.02.2020
https://datahelpdesk.worldbank.org/knowledgebase/articles/906519-world-bank-country-and-lending-groups
https://datahelpdesk.worldbank.org/knowledgebase/articles/906519-world-bank-country-and-lending-groups
Assessing country performances during the COVID-19 pandemic: a standard deviation based
range of value method
67
For the comparisons, 12 evaluation criteria have been selected based on the
studies by Adabavazeh et al. (2020), Aydin and Yurdakul (2020), George et al. (2020),
and Khafaie and Rahim (2020). Those criteria, their codes, and the sources from which
the data is obtained are provided in Table 2.
Table 2. The Criteria used in the Analyses, Their Codes and Data Sources
Criteria Codes Definition Data Source
Total Cases C1
The number of patients
that have positive PCR
tests per 100.000 people
https://www.worldometers.info/coronavirus
(Access Date: 20.01.2021)
Total Deaths C2
Total number of deaths
due to COVID-19 per
100.000 people
https://www.worldometers.info/coronavirus
(Access Date: 20.01.2021)
Extreme
Poverty
C3
The percentage of the
population living less than
daily $1.90
https://ourworldindata.org
(Access Date: 20.01.2021)
Cardiovascular
Death Rate
C4
The annual number of
deaths due to
cardiovascular diseases
per 100.000 people in a
given country.
https://ourworldindata.org
(Access Date: 20.01.2021)
Diabetes
Prevalence
C5
The rate of people aged
between 20 and 79 with
type 1 and type 2 diabetes
https://ourworldindata.org
(Access Date: 20.01.2021)
Female
Smokers
C6
The number of women
who smoke in a given
country.
https://ourworldindata.org
(Access Date: 20.01.2021)
Population
Aged 65 +
C7
The rate of people aged 65
and above to the total
population in a given
country.
The World Bank Database (2019)
Male Smokers C8
The number of men who
smoke in a given country.
https://ourworldindata.org
(Access Date: 20.01.2021)
Current Health
Expenditure
C9
The amount of health
expenditures as a
percentage of the GDP of a
given country.
The World Bank Database (2018)
Total
Recovered
C10
The number of patients
recovered from COVID-19
infection per 100.000
people.
https://www.worldometers.info/coronavirus
(Access Date: 20.01.2021)
Hospital Beds
Per Thousand
C11
The number of hospital
beds per 1000 people in a
given country.
https://ourworldindata.org
(Access Date: 20.01.2021)
Total Tests C12
The number of total PCR
tests to diagnose COVID-
19 infections per 100.000
people.
https://www.worldometers.info/coronavirus
(Access Date: 20.01.2021)
3.1. Standard Deviation Method
The standard deviation method is used to weight the criteria in this analysis. This
method is developed by Diakoulaki et al. (1995) and the weights are determined
according to the standard deviations of criteria. This is an objective weighting method
in which decision-makers do not influence establishing the relative importance of
criteria. This method gives lower weights to an attribute as long as the attribute has
similar values in different alternatives. As a result, the contrasting attribute in the
https://ourworldindata.org/
https://ourworldindata.org/
https://ourworldindata.org/
https://ourworldindata.org/
https://ourworldindata.org/
Türkoğlu and Tuzcu/Oper. Res. Eng. Sci. Theor. Appl. 4 (3) (2021) 59-81
68
alternatives becomes much highlighted (Diakoulaki et al., 1995; Hassan et al., 2015).
Diakoulaki et al. (1995) indicate that when the criteria are interdependent, the results
might be misleading. However, removing some of these interdependent criteria might
cause a loss of important information as well. In this case, using the standard deviation
method in the weight determination might be a solution. This feature of the standard
deviation method becomes more important when the nature of the COVID-19
pandemic is considered. As an example, the criteria in Table 2 include total confirmed
cases, total tests, and deaths. Naturally, as more testing is performed, more cases will
be confirmed and more COVID-19 deaths will be detected. However, the omission of
any of these criteria will result in a loss of information.
While obtaining the weights into the standard deviation method, first, a
normalization process is applied through Eq. (1) and Eq. (2) to provide a common and
measurable basis for criteria that differ in scale and units (Diakoulaki et al., 1995; El-
Santawy & Ahmed, 2012):
𝑥𝑖𝑗
′ =
𝑥𝑖𝑗−min (𝑥𝑖𝑗)𝑖=1
𝑚
max (𝑥𝑖𝑗)𝑖=1
𝑚 −min (𝑥𝑖𝑗)𝑖=1
𝑚 i= 1,2,…,m; j = 1,2,… ,n (for beneficial) (1)
𝑥𝑖𝑗
′ =
max (𝑥𝑖𝑗)𝑖=1
𝑚 −𝑥𝑖𝑗
max (𝑥𝑖𝑗)𝑖=1
𝑚 −min (𝑥𝑖𝑗)𝑖=1
𝑚 i=1,2,…,m; j=1,2,…,n (for non-beneficial) (2)
There are m alternatives and n criteria. Here, xij is the raw score of ith alternative
for criterion j. (x')mxn is the matrix after the normalization process, while max xij and
min xij are the maximum and minimum values of xij respectively. The standard
deviation for each criterion is computed with the aid of Eq. (3).
𝑆𝐷𝑉𝑗 = √
1
𝑚
∑ (𝑥𝑖𝑗
′ − 𝑥𝑗
′̅)2𝑚𝑖=1 (3)
𝑥𝑗
′̅ is the average of the values of the jth criterion after the normalization process,
and j=1,2,…,n.
After obtaining standard deviation values, the weightings of criteria are computed
by using Eq. (4).
𝑤𝑗 =
𝑆𝐷𝑉𝑗
∑ 𝑆𝐷𝑉𝑗
𝑛
𝑗=1
j = 1,2,…, n (4)
3.2. Range of Value (ROV) Method
ROV approach is developed by Yakowitz and Szidarovxzky (1993). Hajkowicz and
Higgins (2008) state that this method is especially beneficial when it is not possible or
meaningful to provide quantitative weights. After one year of the pandemic, COVID-19
still protects its black box nature in many aspects. Therefore, instead of assigning
quantitative weights, we find the ROV method more suitable to assess the country's
performances during the pandemic. To the extent of our knowledge, this is to first
paper that combines the ROV approach with the standard deviation method in both
MCDM and COVID-19 literature.
ROV method basically computes the best and worst utility values for each
alternative (Hajkowicz & Higgins, 2008). To calculate those values, the utility function
is maximized and minimized, respectively. As a result, the performance rankings of
alternatives are obtained.
Assessing country performances during the COVID-19 pandemic: a standard deviation based
range of value method
69
The procedure for the application of the ROV method is simple which constitutes
another strong point for this approach. The application steps are as follows (Madić et
al., 2016):
Step 1: The relevant criteria to assess the existing alternatives are set.
Step 2: The decision matrix is formed. In this matrix, each row represents an
alternative whereas each column reflects a criterion.
Step 3: The decision matrix is normalized at this stage. For beneficial criteria,
where maximization is applied, the normalization is done by using Eq. (5) below:
𝑥𝑖𝑗 =
𝑥𝑖𝑗−min (𝑥𝑖𝑗)𝑖=1
𝑚
max (𝑥𝑖𝑗)𝑖=1
𝑚 −min (𝑥𝑖𝑗)𝑖=1
𝑚 (5)
For the non-beneficial criteria, however, where minimization is applied, the
normalization is done by using Eq. (6) below:
𝑥𝑖𝑗 =
max (𝑥𝑖𝑗)𝑖=1
𝑚 −𝑥𝑖𝑗
max (𝑥𝑖𝑗)𝑖=1
𝑚 −min (𝑥𝑖𝑗)𝑖=1
𝑚 (6)
In both Eq (5) and (6), xij is the raw score of ith alternative for criterion j. There are
m alternatives. max xij and min xij are the maximum and minimum values of xij, and the
�̅�𝑖𝑗 is the normalized values.
Step 4: For each alternative, the best and worst utility values are computed as in
Eq. (7) and (8).
Max: 𝑢𝑖
+ = ∑ 𝑥𝑖𝑗
𝑛
𝑗=1 ∗ 𝑤𝑗 (7)
Min: 𝑢𝑖
− = ∑ 𝑥𝑖𝑗
𝑛
𝑗=1 ∗ 𝑤𝑗 (8)
Where ui+ and ui- represent utility values and wj reflects the criterion weight. The
summation of the criterion weights must be equal to 1, and wj≥0.
When ui-≥ui’+, the ith alternative shows a better performance than the alternative i’
without looking at the quantitative weights. If this basis is not sufficient to distinguish
the alternatives, a midpoint scoring (ui) can be used to allow the following ranking as
such:
𝑢𝑖 =
𝑢𝑖
−+ 𝑢𝑖
+
2
(9)
Step 5: In this last step, the alternatives are ordinally ranked based on their ui
values. The best alternative has the highest ui, whereas the worst one has the lowest
ui.
4. Findings
Following the above procedure, the criteria that are presented in Table 2 are first
weighted by using the standard deviation method. Next, the decision matrix is formed.
Here, C1,…., C8 represent non-beneficial criteria to be minimized, while C9, C10, C11,
and C12 are beneficial criteria, to be maximized. The decision matrix can be seen in
Table 3.
Türkoğlu and Tuzcu/Oper. Res. Eng. Sci. Theor. Appl. 4 (3) (2021) 59-81
70
Table 3. The Initial Decision Matrix
Countries C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12
Albania 2381.8 44.8 1.1 304.2 10.1 7.1 14.2 51.2 5.3 1470.4 2.9 11105.5
Argentina 4021.9 101.9 0.6 191.0 5.5 16.2 11.2 27.7 9.6 3591.0 5.0 12677.7
Armenia 5567.6 101.3 1.8 341.0 7.1 1.5 11.5 52.1 10.0 5201.8 4.2 21246.1
Bosnia and
Herzegovina
3577.1 135.8 0.2 329.6 10.1 30.2 17.2 47.7 8.9 2698.4 3.5 17304.2
Brazil 4033.0 99.6 3.4 178.0 8.1 10.1 9.3 17.9 9.5 3584.2 2.2 13551.3
Bulgaria 3044.5 122.7 1.5 424.7 5.8 30.1 21.3 44.4 7.3 2510.0 7.5 18543.8
Mainland
China
7.0 0.3 0.7 261.9 9.7 1.9 11.5 48.4 5.4 5.8 4.3 11447.2
Colombia 3820.3 97.3 4.5 124.2 7.4 4.7 8.8 13.5 7.6 3548.2 1.7 18555.9
Costa Rica 3685.2 48.6 1.3 138.0 8.8 6.4 9.9 17.4 7.6 2870.2 1.1 11012.0
Dominican
Republic
1815.5 22.7 1.6 266.7 8.2 8.5 7.3 19.1 5.7 1356.5 1.6 9065.9
Ecuador 1333.3 82.4 3.6 140.4 5.6 2.0 7.4 12.3 8.1 1147.3 1.5 4742.1
Georgia 6663.6 79.5 4.2 496.2 7.1 5.3 15.1 55.5 7.1 6368.2 2.6 54798.1
Indonesia 338.8 9.7 5.7 342.9 6.3 2.8 6.1 76.1 2.9 282.1 1.0 3138.1
Iran 1611.5 68.6 0.2 270.3 9.6 0.8 6.4 21.1 8.7 1372.2 1.5 10413.8
Kazakhstan 1171.8 15.6 0.1 466.8 7.1 7.0 7.7 43.1 2.9 839.3 6.7 32097.7
Malaysia 506.2 1.8 0.1 260.9 16.7 1.0 6.9 42.4 3.8 399.5 1.9 13036.4
Mexico 1292.9 110.7 2.5 152.8 13.1 6.9 7.4 21.4 5.4 981.2 1.4 3284.4
Montenegro 8969.5 119.9 1.0 387.3 10.1 44.0 15.4 47.9 8.4 7592.0 3.9 34205.6
Paraguay 1740.1 35.7 1.7 199.1 8.3 5.0 6.6 21.6 6.7 1420.7 1.3 8769.1
Russia 2460.8 45.0 0.1 431.3 6.2 23.4 15.1 58.3 5.3 2096.8 8.1 67602.4
Thailand 18.0 0.1 0.1 109.9 7.0 1.9 12.4 38.8 3.8 13.8 2.1 1749.1
Turkey 2868.2 28.9 0.2 171.3 12.1 14.1 8.7 41.1 4.1 2737.5 2.8 33402.3
The next step is to normalize each element in the decision matrix. This process is
done according to Eq. (1) for beneficial criteria and according to Eq. (2) for non-
beneficial criteria. The normalized decision matrix is presented in Table 4.
Table 4. Normalized Decision Matrix
Countries
Criteria
C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12
Albania 0.74 0.67 0.82 0.50 0.59 0.85 0.46 0.39 0.33 0.19 0.26 0.14
Argentina 0.55 0.25 0.91 0.79 1.00 0.64 0.66 0.76 0.94 0.47 0.56 0.17
Armenia 0.38 0.25 0.70 0.40 0.86 0.98 0.64 0.38 1.00 0.68 0.45 0.30
Bosnia and Herzegovina 0.60 0.00 0.98 0.43 0.59 0.32 0.27 0.45 0.84 0.35 0.35 0.24
Brazil 0.55 0.27 0.41 0.82 0.77 0.78 0.79 0.91 0.93 0.47 0.17 0.18
Bulgaria 0.66 0.10 0.75 0.19 0.97 0.32 0.00 0.50 0.63 0.33 0.91 0.26
Mainland China 1.00 1.00 0.89 0.61 0.62 0.97 0.64 0.43 0.35 0.00 0.47 0.15
Colombia 0.57 0.28 0.21 0.96 0.83 0.91 0.82 0.98 0.67 0.47 0.10 0.26
Costa Rica 0.59 0.64 0.79 0.93 0.71 0.87 0.75 0.92 0.66 0.38 0.01 0.14
Dominican Republic 0.80 0.83 0.73 0.59 0.76 0.82 0.92 0.89 0.40 0.18 0.08 0.11
Ecuador 0.85 0.39 0.38 0.92 1.00 0.97 0.91 1.00 0.74 0.15 0.07 0.05
Georgia 0.26 0.42 0.27 0.00 0.86 0.90 0.41 0.32 0.59 0.84 0.22 0.81
Indonesia 0.96 0.93 0.00 0.40 0.93 0.95 1.00 0.00 0.00 0.04 0.00 0.02
Iran 0.82 0.50 0.98 0.58 0.64 1.00 0.98 0.86 0.81 0.18 0.07 0.13
Kazakhstan 0.87 0.89 1.00 0.08 0.86 0.86 0.89 0.52 0.01 0.11 0.81 0.46
Malaysia 0.94 0.99 1.00 0.61 0.00 1.00 0.94 0.53 0.12 0.05 0.12 0.17
Mexico 0.86 0.19 0.57 0.89 0.33 0.86 0.91 0.86 0.35 0.13 0.05 0.02
Montenegro 0.00 0.12 0.84 0.28 0.59 0.00 0.39 0.44 0.78 1.00 0.40 0.49
Paraguay 0.81 0.74 0.71 0.77 0.75 0.90 0.96 0.85 0.53 0.19 0.04 0.11
Russia 0.73 0.67 1.00 0.17 0.94 0.48 0.41 0.28 0.34 0.28 1.00 1.00
Thailand 1.00 1.00 1.00 1.00 0.86 0.97 0.58 0.58 0.13 0.00 0.15 0.00
Turkey 0.68 0.79 0.98 0.84 0.41 0.69 0.82 0.55 0.18 0.36 0.25 0.48
Assessing country performances during the COVID-19 pandemic: a standard deviation based
range of value method
71
In the last step of this method, the standard deviation of each performance
criterion and its relevant weight is computed by using Eq. (3) and Eq. (4). The results
are shown in Table 5.
Table 5. Standard Deviations and Criterion Weights
SDVj is the calculated standard deviation for each criterion.
Wj is the weight for each criterion
It is observed from the criteria weights in Table 5 that the most important criteria
determining the country's performances are Total Deaths (C2), Current Health
Expenditures (C9), and Cardiovascular Death Rate (C4). The least important criteria
are, however, Diabetes Prevalence (C5), Total Cases (C1), and Total Tests (C12).
Table 6. Criteria Weights Comparison Matrix
T
o
ta
l
C
a
se
s
(C
1
)
T
o
ta
l
D
e
a
th
s
(C
2
)
E
x
tr
e
m
e
P
o
v
e
rt
y
(C
3
)
C
a
rd
io
v
a
sc
u
la
r
D
e
a
th
R
a
te
(
C
4
)
D
ia
b
e
te
s
P
re
v
a
le
n
ce
(
C
5
)
F
e
m
a
le
S
m
o
k
e
rs
(C
6
)
A
g
e
d
6
5
+
(
C
7
)
M
a
le
S
m
o
k
e
rs
(
C
8
)
C
u
rr
e
n
t
H
e
a
lt
h
E
x
p
e
n
d
it
u
re
s
(C
9
)
T
o
ta
l
R
e
co
v
e
re
d
(C
1
0
)
H
o
sp
it
a
l
B
e
d
s
P
e
r
1
0
0
0
(
C
1
1
)
T
o
ta
l
T
e
st
s
(C
1
2
)
wj 7
.4
0
%
9
.7
8
%
8
.8
0
%
9
.1
1
%
7
.2
3
%
7
.9
7
%
8
.0
6
%
8
.1
0
%
9
.2
8
%
7
.9
1
%
8
.8
4
%
7
.5
4
%
Total Cases (C1) 7.40% 0.76 0.84 0.81 1.02 0.93 0.92 0.91 0.80 0.93 0.83 0.98
Total Deaths (C2) 9.78% 1.32 1.11 1.07 1.35 1.23 1.21 1.21 1.05 1.24 1.11 1.30
Extreme Poverty
(C3) 8.80% 1.19 0.90 0.97 1.22 1.10 1.09 1.09 0.95 1.11 1.00 1.17
Cardio. Death Rate
(C4) 9.11% 1.23 0.93 1.04 1.26 1.14 1.13 1.13 0.98 1.15 1.03 1.21
Diabetes
Prevalence (C5) 7.23% 0.98 0.74 0.82 0.79 0.91 0.90 0.89 0.78 0.91 0.82 0.96
Female Smokers
(C6) 7.97% 1.08 0.81 0.91 0.87 1.10 0.99 0.98 0.86 1.01 0.90 1.06
Aged 65 + (C7) 8.06% 1.09 0.82 0.92 0.88 1.11 1.01 1.00 0.87 1.02 0.91 1.07
Male Smokers (C8) 8.10% 1.09 0.83 0.92 0.89 1.12 1.02 1.00 0.87 1.02 0.92 1.07
Current Health
Exp. (C9) 9.28% 1.25 0.95 1.05 1.02 1.28 1.16 1.15 1.15 1.17 1.05 1.23
Total Recovered
(C10) 7.91% 1.07 0.81 0.90 0.87 1.09 0.99 0.98 0.98 0.85 0.89 1.05
Hospital Beds Per
1000 (C11) 8.84% 1.20 0.90 1.00 0.97 1.22 1.11 1.10 1.09 0.95 1.12 1.17
Total Tests (C12) 7.54% 1.02 0.77 0.86 0.83 1.04 0.95 0.94 0.93 0.81 0.95 0.85
Wj is the weight for each criterion
T
o
ta
l
C
a
se
s
(C
1
)
T
o
ta
l
D
e
a
th
s
(C
2
)
E
x
tr
e
m
e
P
o
v
e
rt
y
(
C
3
)
C
a
rd
io
v
a
sc
u
la
r
D
e
a
th
R
a
te
(C
4
)
D
ia
b
e
te
s
P
re
v
a
le
n
ce
(
C
5
)
F
e
m
a
le
S
m
o
k
e
rs
(
C
6
)
A
g
e
d
6
5
+
(
C
7
)
M
a
le
S
m
o
k
e
rs
(
C
8
)
C
u
rr
e
n
t
H
e
a
lt
h
E
x
p
e
n
d
it
u
re
s
(C
9
)
T
o
ta
l
R
e
co
v
e
re
d
(
C
1
0
)
H
o
sp
it
a
l
B
e
d
s
P
e
r
1
0
0
0
(C
1
1
)
T
o
ta
l
T
e
st
s
(C
1
2
)
SDVj 0.25 0.33 0.30 0.31 0.24 0.27 0.27 0.27 0.31 0.27 0.30 0.25
wj 0.07 0.10 0.09 0.09 0.07 0.08 0.08 0.08 0.09 0.08 0.09 0.08
Türkoğlu and Tuzcu/Oper. Res. Eng. Sci. Theor. Appl. 4 (3) (2021) 59-81
72
As noted before, the standard deviation method determines the criteria weights
based on the contrasting attributes. In other words, this method puts less importance
if the attribute distributes more evenly among the alternatives, but places more
emphasis if it differs significantly. From Table 5, it is seen that the number of cases and
number of tests per one hundred thousand people are similar for the countries in the
sample. However, the differences in the number of deaths per one hundred thousand
and current health expenditures become prominent across countries.
Based on the weights in Table 5, we also construct the matrix in Table 6 to show
the relative importance for any pair of criteria following De Nardo et al. (2020). This
matrix is interpreted from left to right and it is not symmetric. For example, Table 6
indicates that the number of COVID-19 deaths is slightly more important than the
deaths due to cardiovascular diseases (1.07) and the current health expenditures
(1.05), but 1.32 times more important than the number of confirmed cases. The
extreme poverty level is as important as the number of hospital beds. In fact, current
health expenditures and the extreme poverty level are two of the criteria that show
the biggest discrepancies among the sample countries. The diabetes level in a
population, however, is the least important criterion in this analysis.
Table 7. The Results of the ROV Method
Countries
Results
ui- ui+ ui Rankings
Thailand 0.59 0.03 0.60 1
Iran 0.52 0.11 0.58 2
Paraguay 0.54 0.08 0.58 3
Ecuador 0.52 0.09 0.57 4
Costa Rica 0.52 0.10 0.57 5
Dominican Republic 0.53 0.07 0.56 6
Mainland China 0.52 0.08 0.56 7
Argentina 0.46 0.19 0.55 8
Kazakhstan 0.49 0.12 0.55 9
Turkey 0.49 0.10 0.54 10
Malaysia 0.51 0.04 0.53 11
Colombia 0.45 0.13 0.52 12
Brazil 0.43 0.15 0.51 13
Russia 0.38 0.22 0.49 14
Armenia 0.37 0.21 0.48 15
Mexico 0.45 0.05 0.47 16
Albania 0.42 0.08 0.46 17
Indonesia 0.42 0.00 0.42 18
Georgia 0.28 0.20 0.38 19
Bosnia and Herzegovina 0.30 0.16 0.37 20
Bulgaria 0.28 0.18 0.37 21
Montenegro 0.22 0.22 0.33 22
Average 0.50
After obtaining the criterion weights, we find the rankings of country performances
in the struggle against COVID-19 by employing the ROV approach. Since the first three
steps in the ROV approach are the same as the weighting procedure of the standard
deviation model, the normalized decision matrix shown in Table 4 is used for the ROV
method as well. The best and worst utility values are computed according to Eq. (7)
Assessing country performances during the COVID-19 pandemic: a standard deviation based
range of value method
73
and Eq. (8). Next, for each alternative, the mid values (ui) are calculated by using Eq.
(9). According to these values, countries, which constitute the alternatives, are ranked.
The overall results of the ROV method are given in Table 7.
The results in Table 7 demonstrate that Thailand, Iran, and Paraguay perform the
best in the fight against the COVID-19 pandemic, while Bosnia and Herzegovina,
Bulgaria, and Montenegro are the worst-performing countries. The average
performance score is 0.4984. The starting point of the pandemic, mainland China has
the 7th ranking.
From Table 7, it is seen that the high-performing countries have lower COVID-19
deaths, higher healthcare expenditures, lower poverty rates, and much lower rates of
cardiovascular diseases. Bosnia and Herzegovina and Bulgaria, the countries with the
lowest performance, have higher COVID-19 deaths and the highest elderly population
in the sample. Montenegro is one of the countries with the lowest COVID-19 deaths.
Since the beginning of the pandemic, Montenegro has applied strict physical
distancing rules. However, when the population characteristics are closely observed,
it is seen that smoking and diabetes are very common. The cardiovascular death rates
are elevated. Together with the large elderly population, the struggle against the
current pandemic becomes tougher.
Interestingly, the countries with the lowest performance have relatively high levels
of current health expenditures and hospital bed capacity. In the case of Bulgaria, the
lack of the medical workforce, especially in terms of nursing staff created difficulties
for the treatment of COVID-19 disease. Particularly during the wave of October 2020,
the number of infected medical professionals increased heavily and generated a
shortage in workforce capacity3. The Bulgarian government was also criticized for not
putting into force necessary non-healthcare precautions timely. The physical
distancing rules were started to be applied late. Bosnia and Herzegovina, on the other
hand, caught relatively short of the COVID-19 pandemic. There was no initial
emergency action plan in the country for such a disaster. Although the healthcare
resources are higher than the sample average in terms of expenditures and hospital
beds, the resources seem to be inefficiently managed during the pandemic4.
These findings indicate that each country struggles with this global health crisis
with its own resources. Healthcare capacity and the amount of healthcare expenditure
matter, albeit, are not enough. The efficient management of available resources and
the existence of an emergency action plan are also necessary. In addition, these three
countries have the highest elderly percentage in the sample. An older population
increases the demand for long term healthcare attention (Yiğit, 2020). This situation
raises the burden on the overall system particularly in times of crisis and limits the
success of the “flattening the curve” efforts.
3COVID-19 Health System Response Monitor (HSRM)
https://www.covid19healthsystem.org/countries/bulgaria/livinghit.aspx?Section=2.2%20Wo
rkforce&Type=Chapter#7Physicalinfrastructure Access Date: 09.07.2021
4 COVID-19 Health System Response Monitor (HSRM)
https://www.covid19healthsystem.org/countries/bosniaandherzegovina/livinghit.aspx?Secti
on=2.1%20Physical%20infrastructure&Type=Section Access Date: 09.07.2021
https://www.covid19healthsystem.org/countries/bulgaria/livinghit.aspx?Section=2.2%20Workforce&Type=Chapter#7Physicalinfrastructure
https://www.covid19healthsystem.org/countries/bulgaria/livinghit.aspx?Section=2.2%20Workforce&Type=Chapter#7Physicalinfrastructure
https://www.covid19healthsystem.org/countries/bosniaandherzegovina/livinghit.aspx?Section=2.1%20Physical%20infrastructure&Type=Section
https://www.covid19healthsystem.org/countries/bosniaandherzegovina/livinghit.aspx?Section=2.1%20Physical%20infrastructure&Type=Section
Türkoğlu and Tuzcu/Oper. Res. Eng. Sci. Theor. Appl. 4 (3) (2021) 59-81
74
When we turn our attention to the Asian countries, we observe that Thailand,
Malaysia, and mainland China are among the least affected countries by the COVID-19
pandemic in the world (Olivia et al., 2020). Thailand has a very low rate of COVID-19
deaths which brings the country to the first ranking. In fact, it is the only middle-high
income country that has a position in the top ten health systems in the Global Health
Security index ratings5. Together with mainland China, Thailand is among the
countries that apply very strict measures since the beginning of the pandemic. The
country had an influenza pandemic plan and people have been used to wear face
masks because of previous epidemics and air pollution (Issac et al., 2021). Issac et al.
(2021) indicate that the success behind Thailand’s COVID-19 management is due to
the primary healthcare workers that also surveil individual mobility and high usage of
technological tools.
Mainland China has the 7th ranking in this comparison. China is known for its very
restrictive measures since the beginning of the pandemic. On the global level, it has a
low amount of total deaths due to effective coordination between its local and central
administrations and the applied social control levels (Jiang, 2020). Comparing to the
country figures in our sample, mainland China has COVID-19 deaths less than the
sample average as well. It is position is mostly due to its averaged current healthcare
expenditure levels and relatively higher cardiovascular death rates in society.
Malaysia has the average performance ranking according to our analysis (11th
place). The country has lower COVID-19 deaths. Its population is relatively young and
the cardiovascular deaths rates are close to the sample average. Hamzah et al. (2021)
investigate the performance of Malaysia against COVID-19 very thoroughly with a
network DEA approach. They examine the performance into three sub-areas, namely
community surveillance, non-critical and critical medical care needs. They show that
the highest inefficiency is originated from the medical care that must be applied to
patients with critical conditions. The country demonstrates efficient contact tracking
and surveillance, on the other hand. Hamzah et al. (2021) point out that this is mostly
because of the existing emergency plans and the country’s elevated preparedness
levels. The inference of Hamzah et al. (2021) is also confirmed by our analysis. We
observe that Malaysia has much lower current health expenditures than the sample
average which lowers the position of the country and causes a middle ranking despite
its high preparedness level.
In contrast to most Asian countries, Indonesia produces a much lower performance
in the struggle against the pandemic. De Nardo et al. (2020) indicate that in low and
middle-income countries, the poverty rates are higher, so implementing the social
distancing rules is much difficult as well as accessing clean water. This is the case with
Indonesia. It is mostly due to very high rates of extreme poverty, cardiovascular
disease prevalence, low amount of hospital beds per capita (Olivia et al., 2020). Setiati
and Azwar, (2020) point out that the number of beds in intensive care units per capita
is among the lowest in Asian countries. They are mostly located in specific areas, so
5 This global index compares 195 countries according to 6 categories, including disease
prevention, detection and reporting, and health system capabilities. According to this index,
Thailand has an 73.2 index score and had 6th place out of 195 countries in 2019.
https://www.ghsindex.org/about/ and GHS Homepage https://www.ghsindex.org/ Access
Date: 06.08.2021
https://www.ghsindex.org/about/
https://www.ghsindex.org/
Assessing country performances during the COVID-19 pandemic: a standard deviation based
range of value method
75
not evenly distributed. Healthcare workers cannot reach enough protective suits while
dealing with COVID-19 patients. The country has also been criticized for reacting late
to the initial cases while its neighbors, Malaysia and Singapore, had already started
mass testing (Olivia et al., 2020).
Turkey has 10th place in the performance ranking. The country has lower health
expenditures and hospital bed capacity than the sample average. It owes its relatively
high position to the low number of COVID-19 deaths, much lower cardiovascular
disease rates, and younger population. It had an influenza pandemic plan before the
current pandemic which facilitated being prepared for the spread of this disease.
As noted by Jamison et al. (2020), the efforts of “flattening the curve” play a
significant role in those rankings. The delay in the implementation of such measures
can also be a determinant. Brazil, for instance, does not have a high rate of chronic
diseases or deaths, has a relatively younger population and expenditures for the
healthcare system as a percentage of GDP are relatively higher. However, the Brazilian
government has minimized the necessary precautions. The social distancing
implementations are very weak and the extreme poverty in the country is high. This
explains the very high rates of deaths related to COVID-19.
In contrast to Brazil, the countries located in central and south America mostly
demonstrated good performance against COVID-19. Paraguay, Ecuador, Costa Rica,
and the Dominican Republic have positions from 3rd to 6th in the sample. Among these
countries, Ecuador and Costa Rica have advanced Global Health Index rankings 6. The
population percentage of 65 years and over is relatively small in Paraguay. The
country also experiences a small number of COVID-19 deaths. UNDP Paraguay report
(2020) reveals that the country was fighting the Dengue epidemic when the COVID-19
pandemic started. Because of the previous experience from the Dengue outbreak, the
Paraguay government responded quickly to this new disease and applied physical
distance measures and movement restrictions early.
We observe that a bigger healthcare capacity and lower poverty levels maintain a
strong stance against health system crises even with higher levels of disease spread.
As the healthcare system becomes larger and more evenly distributed, this battle gets
easier. However, as we see from the Bosna and Herzegovina example, the efficient
usage of the existing capacity is crucial to be successful. The initial preparedness of the
countries and their willingness to apply social distancing measures are also important
to limit the number of deaths. Brazil, for example, has a relatively greater overall score
in the rankings of the Global Health Security Index. However, the government’s
reluctance to put in force necessary measures caused a significant number of deaths.
Poverty levels, on the other hand, determine the applicability of non-healthcare
precautions. In the countries where most of the population lives below the poverty
line, it is almost impossible to achieve physical distancing and personal hygiene. This
environment accelerates the new numbers of COVID-19 cases. Taken together with
the limited healthcare facilities, it is natural to observe a higher number of deaths.
6https://www.ghsindex.org/country/ecuador/ and
https://www.ghsindex.org/country/costarica/ Access Date: 06.08.2021.
https://www.ghsindex.org/country/ecuador/
https://www.ghsindex.org/country/costarica/
Türkoğlu and Tuzcu/Oper. Res. Eng. Sci. Theor. Appl. 4 (3) (2021) 59-81
76
5. Sensitivity Analysis
To show that our findings from the SDV based ROV approach are robust, we also
conduct TOPSIS and EDAS analyses and compare the results. As discussed before, the
ROV method depends on the comparison of best and worst utility functions for each
alternative. Based on these utility functions, we obtain the rankings of alternatives in
an easy manner. TOPSIS method, which is widely used in different areas (for example,
Mimović et al. 2021), computes distances from positive ideal and negative ideal
solutions. The alternative is ranked based on its proximity to the positive ideal and its
remoteness from the negative ideal. The EDAS method, however, calculates the
distances from average solutions. The positive and negative ideals are not required to
be estimated (Ghorabaee et al., 2015).
We apply both TOPSIS and EDAS methods by employing the criteria weights
obtained from SDV analysis. The comparisons of the rankings from these three
methods are demonstrated in Figure 2.
Figure 2: The rankings from ROV, EDAS, and TOPSIS methods respectively
The comparisons show that the rankings from these three methods are quite
parallel to each other. Particularly, the rankings of the low-performing countries stay
very similar. In addition to the visual analysis, we also look at the Spearman rank
correlations and Wilcoxon rank tests. The former one is applied to observe the
direction and strength of the relation between the ranks obtained from these three
methods. The latter test shows whether the mean ranks are statistically different. Both
are non-parametric tests since MCDM analysis provides results that are not normally
distributed. The findings are presented in Table 8.
0
5
10
15
20
25
A
lb
a
n
ia
A
rg
e
n
ti
n
a
A
rm
e
n
ia
B
o
sn
ia
a
n
d
H
e
rz
e
g
o
v
in
a
B
ra
zi
l
B
u
lg
a
ri
a
C
o
lo
m
b
ia
C
o
st
a
R
ic
a
D
o
m
in
ic
a
n
R
e
p
u
b
li
c
E
cu
a
d
o
r
G
e
o
rg
ia
In
d
o
n
e
si
a
Ir
a
n
K
a
za
k
h
st
a
n
M
a
in
la
n
d
C
h
in
a
M
a
la
y
si
a
M
e
xi
co
M
o
n
te
n
e
g
ro
P
a
ra
g
u
a
y
R
u
ss
ia
T
h
a
il
a
n
d
T
u
rk
e
y
ROV EDAS TOPSIS
Assessing country performances during the COVID-19 pandemic: a standard deviation based
range of value method
77
Table 8. The Results for Spearman Rank Correlations and Wilcoxon Rank Tests
Spearman Rank Correlations
ROV EDAS TOPSIS
ROV 1
EDAS 0.69 1
TOPSIS 0.61 0.97 1
Wilcoxon Rank Tests
H0 ROV=EDAS ROV=TOPSIS
Z 0.488 0.261
p-value 0.626 0.806
The findings from Spearman rank correlations indicate that the ROV method
produces positive and fairly high association with EDAS and TOPSIS methods (0.69
and 0.61 respectively). In the second part of Table 8, the mean ranking of ROV is tested
against EDAS and TOPSIS. The null hypotheses here are the mean ranking of ROV is
not different than the mean ranking of EDAS and TOPSIS respectively (first and second
columns). The p-values for both tests are much higher than any acceptable significance
levels, so the null hypotheses cannot be rejected.
The overall results suggest that the ROV method has a computational advantage
over more popular methods such as TOPSIS and EDAS and offers similar rankings.
6. Conclusion
This study aims to shed light on the black-box nature of the COVID-19 pandemic by
comparing the country's performances in this global struggle. To do so, the
performance of 22 countries, which all belong to high–middle income group according
to the World Bank classifications, are assessed. Since several and conflicting criteria
are used, the performance ranking against managing the current pandemic is
considered a multicriteria decision-making problem. The benchmarking is made with
the standard deviation-based ROV approach. To the extent of our knowledge, this is
the first study that combines these two methods and applies them to the COVID-19
pandemic.
In the struggle against COVID-19, there are some important parameters such as the
properties of the healthcare system, application of physical distancing rules and
mobility restrictions, and the population characteristics. Among them, particularly the
healthcare capacity, proxied by the number of hospital beds and the current
healthcare expenditures, becomes prominent. Our analysis confirms this expectation
and demonstrates that Thailand has the highest ranking in our sample. The country’s
very high position in the Global Health Security Index and the very low number of
COVID-19 deaths are no surprise, in this sense. However, the findings from the lowest
ranking countries reveal that the efficient usage of healthcare resources matters as
much as its amount. These countries have a relatively high amount of healthcare
resources, but they have still experienced elevated numbers of COVID-19 deaths. As
seen from the Bosnia and Herzegovina case, the lack of an emergency healthcare plan
makes much more difficult to obtain efficiency in the usage of resources.
We also observe that the high-ranked countries are mostly praised for their quick
response to take necessary non-healthcare restrictive precautions and the low-ranked
Türkoğlu and Tuzcu/Oper. Res. Eng. Sci. Theor. Appl. 4 (3) (2021) 59-81
78
ones are highly criticized for the same reason. It is seen that many Asian countries, for
instance, Thailand, mainland China, and Malaysia, applied strict social distancing rules,
whereas Brazil, Bulgaria, and Indonesia are the ones where these rules are weak or
sometimes non-existent.
Our analysis also confirms that population characteristics are other determinants
of country performances against COVID-19. The low-ranking countries mostly have an
elder population. Age brings co-morbidities with itself which increases the likelihood
of COVID-19 deaths. It also boosts the requirement of long-term healthcare attention,
which creates another burden on the already overwhelmed healthcare system.
Another population characteristic that affects country rankings is the level of poverty.
In countries, where the poverty level is higher, implementing the social distancing
rules and distant working becomes almost impossible. In fact, we show that extreme
poverty is as crucial as the number of hospital beds in this battle. Combining the fact
that these areas have diminished access to clean water, the spread of the disease is
inevitable. This is one of the social consequences of COVID-19: It contributes to the
gap between rich and poor. Policymakers should consider policies that decrease the
level of poverty to control future pandemics as well. The vaccination policies against
COVID-19 will also have importance. Since the delivery of vaccines requires more time
in the middle-high income countries in comparison to their developed counterparts,
the public must comply with the social distancing rules to avoid future deaths.
For future pandemics, countries that manage their healthcare resources more
efficiently and that are quick to apply social distancing rules will be more successful
to eliminate the negative impacts. However, policymakers must also focus on elderly
care and poverty levels in their countries.
There are some limitations of this study that must be mentioned: First, the very
quick changes in the pandemic landscape restrict the generalizability of the findings
to some extent. This is why the COVID-19 studies that compare the beginning of the
pandemic to the ongoing situation are valuable. Second, we have only used
quantitative data in this analysis. Further researches may consider qualitative data
and appropriate techniques such as fuzzy MDCM to compare the performance of
countries. Last, at the beginning of the pandemic, the definitions of COVID-19 deaths
and patients were not clear for all countries7. In time, there has been a convergence in
these definitions. However, this situation may create a limitation for this study in
terms of comparability.
Acknowledgments: A preliminary version of this paper was presented at the
International Covid-19 and Current Issues Congress, March 12-14, 2021. The authors
would like to thank the participants and reviewers of the conference for their valuable
comments and suggestions.
7Some countries have made a distinction between “deaths from COVID-19” and “deaths with
COVID-19”. The same difference was valid at the beginning of the pandemic for the positive
cases. Some counted a person as COVID-19 patient depending on the clinical diagnosis, others
required a positive test result.
Source:https://eurohealthobservatory.who.int/monitors/hsrm/analyses/hsrm/how-
comparable-is-covid-19-mortality-across-countries Access Date: 05.08.2021
https://eurohealthobservatory.who.int/monitors/hsrm/analyses/hsrm/how-comparable-is-covid-19-mortality-across-countries
https://eurohealthobservatory.who.int/monitors/hsrm/analyses/hsrm/how-comparable-is-covid-19-mortality-across-countries
Assessing country performances during the COVID-19 pandemic: a standard deviation based
range of value method
79
Author Contributions: All authors contributed equally to this paper and have
approved this version.
Conflicts of Interest: The authors declare no conflict of interest.
References
Adabavazeh, N., Nikbakht, M., & Amirteimoori, A. (2020). Envelopment analysis for
global response to novel 2019 coronavirus-SARS-COV-2 (COVID-19). Journal of
Industrial Engineering and Management Studies, 7(2), 1–35.
https://doi.org/10.22116/JIEMS.2020.226564.1354
Aydin, N., & Yurdakul, G. (2020). Assessing countries’ performances against COVID-19
via WSIDEA and machine learning algorithms. Applied Soft Computing Journal, 97,
106792. https://doi.org/10.1016/j.asoc.2020.106792
Bedford, J., Enria, D., Giesecke, J., Heymann, D. L., Ihekweazu, C., Kobinger, G., Lane, H.
C., Memish, Z., Oh, M. don, Sall, A. A., Schuchat, A., Ungchusak, K., & Wieler, L. H. (2020).
COVID-19: towards controlling of a pandemic. The Lancet, 395(10229), 1015–1018.
https://doi.org/10.1016/S0140-6736(20)30673-5
De Nardo, P., Gentilotti, E., Mazzaferri, F., Cremonini, E., Hansen, P., Goossens, H., &
Tacconelli, E. (2020). Multi-Criteria Decision Analysis to prioritize hospital admission
of patients affected by COVID-19 in low-resource settings with hospital-bed shortage.
International Journal of Infectious Diseases, 98, 494–500.
https://doi.org/10.1016/j.ijid.2020.06.082
Diakoulaki, D., Mavrotas, G., & Papayannakis, L. (1995). Determining objective weights
in multiple criteria problems: The critic method. Computers and Operations Research,
22(7), 763–770. https://doi.org/10.1016/0305-0548(94)00059-H
El-Santawy, M. F., & Ahmed, A. N. (2012). A SDV-MOORA Technique for Solving Multi-
Criteria Decision Making Problems with No Preference. Life Science Journal, 9(4),
5881–5883.
George, B., Verschuere, B., Wayenberg, E., & Zaki, B. L. (2020). A Guide to
Benchmarking COVID-19 Performance Data. Public Administration Review, 80(4), 696–
700. https://doi.org/10.1111/puar.13255
Ghorabaee, M. K., Zavadskas, E. K., Olfat, L., & Turskis, Z. (2015). Multi-Criteria
Inventory Classification Using a New Method of Evaluation Based on Distance from
Average Solution (EDAS). Informatica (Netherlands), 26(3), 435–451.
https://doi.org/10.15388/Informatica.2015.57
Hajkowicz, S., & Higgins, A. (2008). A comparison of multiple criteria analysis
techniques for water resource management. European Journal of Operational
Research, 184(1), 255–265. https://doi.org/10.1016/j.ejor.2006.10.045
Hassan, N., Kamal, Z., Moniruzzaman, A. S., Zulkifli, S., & Yusop, B. (2015). Weighting
Methods and their Effects on Multi- Criteria Decision Making Model Outcomes in Water
Resources Management.
Issac, A., Radhakrishnan, R. V., Vijay, V., Stephen, S., Krishnan, N., Jacob, J., Jose, S.,
Azhar, S., & Nair, A. S. (2021). An examination of Thailand’s health care system and
Türkoğlu and Tuzcu/Oper. Res. Eng. Sci. Theor. Appl. 4 (3) (2021) 59-81
80
strategies during the management of the COVID-19 pandemic. Journal of Global Health,
11(03002), 1–4. https://doi.org/10.7189/jogh.11.03002
Jamison, D. T., Lau, L. J., Wu, K. B., & Xiong, Y. (2020). Country performance against
COVID-19: Rankings for 35 countries. BMJ Global Health, 5(12).
https://doi.org/10.1136/bmjgh-2020-003047
Jiang, H. (2020). China’s Anti-epidemic Experience and Its World Significance: An
Introduction to China’s Fight against the COVID-19 Epidemic: Its Contribution and
Implications to the World in the Eyes of Foreigners. International Critical Thought,
10(4), 655–658.
Kayapinar Kaya, S. (2020). Evaluation of the Effect of COVID-19 on Countries’
Sustainable Development Level: A comparative MCDM framework. Operational
Research in Engineering Sciences: Theory and Applications, 3(3), 101.
https://doi.org/10.31181/oresta20303101k
Khafaie, M. A., & Rahim, F. (2020). Cross-Country Comparison of Case Fatality Rates of
Covid19/Sars-Cov-2. Osong Public Health Research Perspective, 11(2), 74–80.
Khatua, D., De, A., Kar, S., Samanta, E., Sekh, A. A., & Guha, D. (2020). A Fuzzy Dynamic
Optimal Model for COVID-19 Epidemic in India Based on Granular Differentiability. In
SSRN Electronic Journal. https://doi.org/10.2139/ssrn.3621640
Madić, M., Radovanović, M., & Manić, M. (2016). Application of the ROV method for the
selection of cutting fluids. Decision Science Letters, 5(2), 245–254.
https://doi.org/10.5267/j.dsl.2015.12.001
Majumder, S., Kar, S., & Samanta, E. (2020). A fuzzy rough hybrid decision making
technique for identifying the infected population of COVID-19. Soft Computing.
https://doi.org/10.1007/s00500-020-05451-0
Md Hamzah, N., Yu, M. M., & See, K. F. (2021). Assessing the efficiency of Malaysia
health system in COVID-19 prevention and treatment response. Health Care
Management Science, 24(2), 273–285. https://doi.org/10.1007/s10729-020-09539-9
Mimovic, P., Tadic, D., Borota-Tisma, A., Nestic, S., & Lafuente, J. G. (2021). Evaluation
and ranking of insurance companies by combining TOPSIS and the interval fuzzy
rough sets. Serbian Journal of Management, 16(2).
Mishra, A. R., Rani, P., Krishankumar, R., Ravichandran, K. S., & Kar, S. (2021). An
extended fuzzy decision-making framework using hesitant fuzzy sets for the drug
selection to treat the mild symptoms of Coronavirus Disease 2019 (COVID-19). Applied
Soft Computing, 103, 107155. https://doi.org/10.1016/j.asoc.2021.107155
Olivia, S., Gibson, J., & Nasrudin, R. (2020). Indonesia in the Time of Covid-19. Bulletin
of Indonesian Economic Studies, 56(2), 143–174.
https://doi.org/10.1080/00074918.2020.1798581
Pal, R., Sekh, A. A., Kar, S., & Prasad, D. K. (2020). Neural network based country wise
risk prediction of COVID-19. Applied Sciences (Switzerland), 10(18).
https://doi.org/10.3390/APP10186448
Samanlioglu, F., & Kaya, B. E. (2020). Evaluation of the COVID-19 Pandemic
Intervention Strategies with Hesitant F-AHP. Journal of Healthcare Engineering, 2020.
https://doi.org/10.1155/2020/8835258
Sayan, M., Sarigul Yildirim, F., Sanlidag, T., Uzun, B., Uzun Ozsahin, D., & Ozsahin, I.
(2020). Capacity Evaluation of Diagnostic Tests for COVID-19 Using Multicriteria
Assessing country performances during the COVID-19 pandemic: a standard deviation based
range of value method
81
Decision-Making Techniques. Computational and Mathematical Methods in Medicine,
2020. https://doi.org/10.1155/2020/1560250
Setiati, S., & Azwar, M. K. (2020). COVID-19 and Indonesia. April.
Shirazi, H., Kia, R., & Ghasemi, P. (2020). Ranking of hospitals in the case of COVID-19
outbreak: A new integrated approach using patient satisfaction criteria. International
Journal of Healthcare Management, 13(4), 312–324.
https://doi.org/10.1080/20479700.2020.1803622
Si, A., Das, S., & Kar, S. (2021). Picture fuzzy set-based decision-making approach using
Dempster–Shafer theory of evidence and grey relation analysis and its application in
COVID-19 medicine selection. Soft Computing, 9(2014).
https://doi.org/10.1007/s00500-021-05909-9
Yakowitz, D. S., & Szidarovxzky, F. (1993). Multi-attribute Decision Making:
Dominance with Respect to an Importance Order of Attributes. Applied Mathematics
and Computation, 54, 167–181.
Yiğit, A. (2020). The Performance of OECD Countries in Combating with Covid 19
Pandemics: A Cross-Sectional Study. Journal of Current Researches on Social Sciences,
10(10 (2)), 399–416. https://doi.org/10.26579/jocress.372
© 2021 by the authors. Submitted for possible open access publication under the
terms and conditions of the Creative Commons Attribution (CC BY)
license (http://creativecommons.org/licenses/by/4.0/).
ASSESSING COUNTRY PERFORMANCES DURING THE COVID-19 PANDEMIC: A Standard devıatıon BASED range of value METHOD
Serap Pelin Türkoğlu1, Sevgi Eda Tuzcu2*
1. Introduction
2. Related Literature
3. Data and Methodology
3.1. Standard Deviation Method
3.2. Range of Value (ROV) Method
4. Findings
5. Sensitivity Analysis
6. Conclusion
References