04_Zorica Mitrovic:tipska.qxd 39 Zorica Mitrović1, Mirko Vujošević2, Gordana Savić3 1,2,3 Univetsity of Belgrade, Faculty of Organizational Sciences Management 2016/75 Data Envelopment Analysis for Evaluating Serbia’s Health Care System UDC: 005.336.1:614.2(497.11) DOI: 10.7595/management.fon.2015.0012 1. Introduction A key policy challenge in both developing and developed countries is to improve outcomes of the health care system while containing cost pressures. In Serbia the current economic and financial crisis weights heavily on fiscal positions, with gross government debt projected to exceed 69.7% of GDP in Serbia by 2016 (Goverment of the Republic of Serbia, 2013), and has reinforced the need to improve public spending effi- ciency. The burden on the health care sector efficiency is even heavier due to the fact that public spending on health care is one of the largest government spending items, representing on average 10.4% of GDP (World Health Organisation, 2014). Furthermore, health care costs are rising rapidly, driven by population ageing and rising relative prices. To improve public health care spending projections and meet a rapidly growing health care demand, without putting even more the public finances on an unsustainable path where they already are, efficiency gains in health care will be crucial. Therefore, this paper the use data envelop- ment analysis, which is one of the most important techniques currently available to model and measure the efficiency of national health care systems across the world . The application of the DEA to derive cross-country comparisons of health care system efficiency at a sys- tem level is challenging.The analysis and measurement of efficiency is complicated due to conceptual chal- lenges, multiple objectives and a great scope for measurement error (Jacobs et al., 2009). Besides, in the health care field, the data envelopment analysis has been used largely for micro-level service efficiency, par- ticularly hospitals, and it could be complex for undertaking at a system level. This paper is an attempt to measure and evaluate the efficiency of the health care system of Serbia in com- parison with countries in the European region using the data envelopment analysis (DEA), one of the most im- portant methods to model and measure the efficiency of health care systems. The research included the sample of 42 countries representing decision-making units (DMU), divided into two groups of countries with similar levels of development. To assess the performance of those DMUs we used three outputs representing mortality rates and three inputs representing health care expenditure and health care human resources. In the presence of undesirable outputs, the proposed model was Bad Output Model with simple modifications. The analysis showed that 19 out of 42 countries are relatively efficient in providing health services and a close examination of these efficient countries shows that most countries are countries with low input values. On the other hand, Serbia’s health system is ranked the 15th out of 21 analyzed systems within its group, with a gap of 37% in comparison with the first ones ranked. The usual explanation is that the health system in Serbia has been affected, during time, by the lack of reforms, by poor funding as well as by the lack of interest of the au- thorities in establishing a long-term strategy; further analysis, however, are needed to show the reasons for this state and what steps should be done to improve the current state. We find that this paper could be useful to both public health practitioners and researchers especially because due to its complexity in undertaking, the measuring efficiency at the system level has recently been applied only in few studies and this research will be a significant contribution. Keywords: data envelopment analysis, health system efficiency, Serbian Health Care System, health policy, European region health care systems 2. Efficiency Measurements and Data Envelopment Analysis Data envelopment analysis (DEA) is a popular method for ranking various kinds of decision-making units (DMUs) by their relative efficiencies. This popularity comes from non-parametric approach that enables DEA efficiency scores to be free of restrictive parametric assumptions. Efficiency scores are generated by comparing all DMUs to a best practice frontier, observations with the highest levels of outputs for the lowest levels of inputs. All DMUs with less efficient combinations of outputs and inputs lie within the frontier. Analysts can then identify these less efficient DMUs and determine what combinations of inputs might move each toward the best practice frontier, thus increasing their efficiency. The DEA methodology is considered to have several advantages over other methods : • it handles multiple inputs and outputs; • it does not require prior weights (as in index numbers); • it does not require any specific assumptions about the functional form between inputs and outputs; • it emphasises individual observations rather than statistical estimates (as in regression analysis); • it is a dynamic analytical decision-making tool that indicates possibilities for improving relative efficiency; • it uses the benchmarking approach to measure DMU efficiency relative to others in their group; • it can assist in identifying best-practice or efficient DMUs and inefficient DMUs within the group. Since Charneset al. (1978) introduced DEA as a mathematical programming based approach for measur- ing relative efficiency it has been extended and widely employed to assess performance of organisations in a vast number of fields including the health care as one of the earliest application domains. It has been used largely for analysing and measuring a micro-level service efficiency, particularly hospitals and some of the recent studies were: measuring efficiency of nursing homes in Italy (Garavaglia, Lettieri, Agasisti, & Lopez, 2011) and USA , measuring productivity of hospitals in Holland (Blank &Valdmanis, 2010), local health de- partments in US (Mukherjee, Santerre, & Zhang, 2010), assessment of hospital efficiency in Turkey and Greece , assessment of efficiency of primary health care centres operating in the San Joaquin Valley, Cali- fornia , evaluating health centre groups in Portugal , performance analysis of Brazilian hospitals , technical efficiency and scale of efficiency of district hospitals in Tamil Nadu, a state in India . This short list from sev- eral past years suggests that the application of DEA is all over the map for measuring efficiency on the micro- level. On the other hand, due to its complexity in undertaking, the measuring efficiency at the system level has recently been applied only in few studies and those were for the EU 27 , the WHO region , the OECD countries , Cyprus, Greece, Iceland, Ireland, Malta, Romania, Slovak Republic, Slovenia and Turkey and India . To implement a model for assessing a health sector efficiency and comparison of the Serbian health care system with those of the countries in the European region, we will consider the listed analysis. 3. Model and Data The starting point of model definition is the choice of the decision-making units (DMUs) . It is obviously nec- essary to assess the efficiency of a country’s health system relative to those of countries with similar levels of development . In this paper we choose to divide the available sample of 42 countries into two groups of countries with similar levels of development, measured by GPD in PPP($) per capita and life expectancy. Group one consisted of countries with GDP PPP($) per capita of more than $25,000 per annum and life ex- pectancy longer than 78 years: Portugal, Greece, Czech Republic, Slovenia, Malta, Israel, Spain, Italy, France, United Kingdom, Iceland, Finland, Belgium, Germany, Ireland, Denmark, Austria, the Netherlands, Switzerland, Norway and Luxembourg. The other group consisted of countries with GDP PPP($) per capita of less than $25,000 per annum and less than 78 years of life expectancy: Kyrgyzstan, the Republic of Moldova, Georgia, Ukraine, Albania, Bosnia and Herzegovina, TFYR Macedonia, Serbia, Kazakhstan, Mon- tenegro, Belarus, Bulgaria, Romania, Latvia, Croatia, Poland, Lithuania, Hungary, Russian Federation, Es- tonia and Slovakia. The next step is the choice of outputs and inputs to be used to characterize the performance of those DMUs . Many factors combine with health expenditures to produce health within a pop- ulation and although higher levels of health expenditure may be associated with more health, economists 40 2016/75Management 41 are uncertain as to the importance of health expenditure in the production of health, relative to other signif- icant factors such as education, nutrition, and social inclusion . Furthermore, studies undertaken on the sys- tem level, as shown in Table 1 differ in using the internal and external factors as inputs and outputs or as explanatory variables. Table 1: Overview of inputs and outputs used in evaluation of health system efficiency No Inputs Outputs Authors Region 1 (Popescu et al., 2014) EU 27 region • Non Immunized Rate • Health expenditures • Adult Survival Rate • Tuberculosis Rate. 2 (K.R., 2012) WHO region Health status: • Per capita total expenditure on health (PPP int. $) • Number of nursing and midwifery personnel • Number of physicians Health service coverage: • Per capita total expenditure on health (PPP int. $) • Number of nursing and midwifery personnel • Number of physicians Health status: • Adult mortality rate (probability of dying between 15 to 60 years per 1,000 population) • Infant mortality rate (probability of dying between birth and age 1 per 1,000 live births) • Neonatal mortality rate (per 1,000 live births) • Under-five mortality rate (probability of dying by age 5 per 1,000 live births) Health service coverage: • Diphtheria tetanus toxoid and pertussis (DTP3) immunization coverage (per cent) • Measles (MCV) immunization coverage among one-year-olds (per cent) 3 (Afonso & St. Aubyn, 2011) OECD countries • Number of practicing physicians • Number of nurses • Number of acute care beds per thousand habitants • Magnetic Resonance Imagers (MRI) • Life expectancy • Infant survival rate • Potential years of life not lost 4 (Alexander et al., 2003) Developing countries: Bangladesh, Bhutan Lao (PDR), Madagascar, Tanzania, China, Indonesia, Jamaica, Sri Lanka • Health expenditure per capita PPP • GDP per capita PPP • Male disability adjusted life expectancy • Female disability adjusted life expectancy • Infant mortality rate subtracted from 1,000. 5 (Pelone et al., 2013) Europe: Cyprus, Greece, Iceland, Ireland, Malta, Romania, the Slovak Republic, Slovenia and Turkey • Governance (12 indicators) • Economic dimensions (11 indicators) • Workforce development (16 indicators) • Access to PC services (12 indicators) • Comprehensiveness of PC (10 indicators) • Continuity of care (9 indicators) • Coordination of care (7 indicators) 6 (Varabyova & Schreyögg, 2013) OECD region • Beds • Employment • Physicians • Nurses • Discharges • Mortality 7 (Retzlaff-Roberts, Chang, & Rubin, 2004) OECD region Health inputs: • Beds (per 1,000 population) • MRI (units per million population) • Physicians (per 1,000 population) • Expenditure (percentage of GDP) Social environment inputs: • School expectancy (years of education expected) 20.0 • Gini (gini coefficient of income distribution) 30.5 • Tobacco (maximum percentage of male or female smokers) • Infant mortality (deaths per 1000 live births) • Life expectancy (years from birth) 8 (Shetty & Pakkala, 2010) India states • Per capita health expenditure (PCHE) • Health Centre Per Million Populations (HCPMP) • Percentage of Population Below Poverty Line (BPL) • Literacy Rate (LR) • Infant Mortality Rate (IMR) • Average life expectancy at birth of men and women (LEB) The other concern is an appropriate number of DMUs, given the number of inputs and outputs chosen. suggests that large numbers of inputs and outputs compared to the number of DMUs may diminish the dis- criminatory power of DEA and that the number of DMUs should be at least twice the number of inputs and outputs combined. Taking all above in consideration together with the data available, selected inputs were physicians per 100,000,nurses and midwives (PP) per 100,000 and total health expenditure, PPP($) per capita, WHO esti- mates. On the other hand, the outputs were infant deaths per 1,000 live births, neonatal deaths per 1,000 live births, SDR all causes, 0-64, per 100,000. Data on the chosen inputs and outputs were obtained from the World Health Statistics (World Health Organization, 2014). The DEA usually assumes that producing more outputs relative to less input resources is a criterion of efficiency. In the presence of undesirable out- puts, as it is in our case, systems with less bad (undesirable) outputs relative to less input resources should be recognized as efficient. In the DEA literature, several authors have proposed methods for this purpose. Our model is modified so as to account for undesirable outputs: BadOutput model. In BadOutput Model output matrix is decomposed into ) where and denote good (de- sirable) and bad (undesirable) output matrices, respectively. For a DMU , the decomposition is de- noted as . The production possibility set is defined by (1) where λ is the intensity vector, and L and U are the lower and upper bounds of the intensity vector, respec- tively. The efficiency status was defined in this framework as follows: Definition of efficient DMU: A DMU is efficient in the presence of bad outputs, if there is no vector such that with at least one strict inequality. In accordance with this definition, the slacks-based measure of efficiency was modified as follows: (2) Subject to: The vectors and correspond to excesses in inputs and bad outputs, while expresses shortages in good outputs. s denotes the number of elements. The optimal solution of the above is ( ). Then the DMU is efficient in the presence of undesirable outputs if and only if , i.e., If the DMU is inefficient. 42 2016/75Management The described model was used on the available dataset to generate information on how efficient is a rela- tive health care system. Hence, this information is obtained by analyzing a relative efficiency of health care systems, singling out the most efficient one and ones that happen to perform below average and compar- ing efficiency of health care systems within the group generating rankings of health care systems. The in- dividual DEA output efficiency measures distinguish those countries that are doing best in their groups, given the level of per capita health expenditures number of nurses and midwives and number of physicians. The results are presented in following chapter. 4. Results Table 2 shows the descriptive statistics for both groups. It is interesting to notice that even though groups are determined by the level of development, measured by GPD in PPP($) per capita and life expectancy, they show a partial overlapping when the inputs are concerned. The number of physicians ranges from 249.5 (Slovenia) to 614.47 (Greece) in group one, and from 114.54 (Albania) to 431.04 (Russia) in group two. The standard deviation within the groups is almost the same but the groups’ average differs just in 15%. Simi- larly, the number of nurses and midwives varies from 377.19 (Greece) to 1767.73 (Switzerland) within group one and from 342.66 (Georgia) to 1114.2 (Belarus) within group two but in this case average number sig- nificantly differs between groups, as it is case with other variables. The third input, total health expenditure significantly differs between groups, what is easy to explain with GDP difference between groups, only Slo- vakia (2087.9) from group two and the Czech Republic (1922.86) from group one are outliers that blur an otherwise clear line between groups. The situation with outputs is completely different. The difference be- tween groups is obvious, for example, the average value of every variable for group two is at least twice higher in comparison with those in group one (Table 2). The partial overlapping in inputs such as physi- cians and nurses and a significant difference in total health expenditure and outputs between groups sig- nals the need for additional analysis that could explain this phenomenon. Table 2: Statistics on Input/Output Data Table 3 shows the results of the DEA with two additional explanatory variables, previously used to divide the countries in two groups of countries with similar development level. Rank 1 has those countries that were found to be efficient by the DEA in each group (Table 3). To understand the nature of efficiency in each group it would be useful to compare information on difference in health outcomes and expenditures for most efficient and selected countries within groups as shown in table 4. 43 Management 2016/75 Physicians per 100,000 Nurses and Midwives (PP) per 100,000 Total health expenditure, PPP($) per capita Infant deaths per 1000 live births Neonatal deaths per 1,000 live births SDR all causes, 0-64, per 100,000 Group 1 Max 614.47 1767.73 6876.1 6.3 5.6 228.45 Min 249.5 377.19 1922.86 1.79 0.99 130.89 Average 353.52 1033.11 3756.18 3.36 2.48 166.43 SD 80.68 392.02 1276.84 0.88 0.89 24.70 Group 2 Max 431.04 1114.22 2087.9 22.3 16.01 587.64 Min 114.54 342.66 160.7 2.38 1.5 221.3 Average 297.19 658.18 1026.68 7.86 5.16 372.23 SD 82.48 161.84 468.57 4.39 3.03 108.29 Table 3: Countries with most efficient health sectors Table 4: Health outcomes and expenditures of selected countries 44 2016/75Management Group 1 Group 2 Explanatory variables Explanatory variables R a n k Country Score GDP, PPP($) per capita LE at birth R a n k Country Score GDP, PPP($) per capita LE at birth 1 Spain 1 32086.5 82.45 1 TFYR Macedonia 1 11560.8 75.10 1 Slovenia 1 26943.0 79.96 1 Albania 1 8866.2 76.24 1 Luxembourg 1 88796.9 81.65 1 Belarus 1 14938.0 70.62 1 Czech Republic 1 26332.3 78.06 1 Bosnia and Herzegovina 1 9076.4 76.65 1 Iceland 1 36483.0 81.93 1 Republic of Moldova 1 3369.1 71.13 1 Ireland 1 40868.1 80.80 1 Croatia 1 19486.7 77.26 1 Israel 1 28809.3 82.24 1 Estonia 1 21996.5 76.63 1 Italy 1 32672.4 82.50 1 Georgia 1 5465.1 74.69 1 Greece 1 25857.8 80.91 1 Latvia 1 18950.8 73.70 10 Netherlands 0.999 42779.3 81.15 1 Kyrgyzstan 1 2401.9 69.51 11 Finland 0.903 37455.4 80.76 11 Poland 0.999 21085.1 76.96 12 Portugal 0.839 25564.3 80.90 12 Montenegro 0.872 13431.9 75.63 13 United Kingdom 0.785 35598.4 80.78 13 Lithuania 0.698 21479.9 73.57 14 Norway 0.745 60392.1 81.53 14 Hungary 0.696 21661.1 75.15 15 Malta 0.719 27503.5 81.04 15 Serbia 0.626 11886.8 74.64 16 France 0.681 35247.1 81.76 16 Slovakia 0.611 24094.7 75.66 17 Switzerland 0.602 51227.1 82.83 17 Bulgaria 0.604 15082.7 74.27 18 Germany 0.602 39456.4 80.64 18 Ukraine 0.601 7208.1 71.12 19 Belgium 0.601 38722.8 80.29 19 Romania 0.590 15983.3 73.83 20 Austria 0.598 42172.5 81.30 20 Kazakhstan 0.521 13099.3 68.59 21 Denmark 0.565 40933.0 80.09 21 Russian Federation 0.377 21920.9 69.03 Countries Group Rank Score Total health expenditure, PPP($) per capita SDR all causes, 0-64, per 100,000 Slovenia 1 1 1 2,518.88 198.79 Switzerland 1 17 0.602 5,564.26 134.27 Macedonia 2 1 1 789.4 264.39 Serbia 2 15 0.626 1,195.22 307.35 Russia 2 21 0.377 1,316.32 587.64 Disscusion and Conclusion First, the analysis showed that 9 out of 21 countries are relatively efficient in providing health care services in group one and 10 out of 21 in group two. A close examination of these efficient countries shows that most countries are countries with low input values (e.g., expenditure on health, density of doctors and nurses per 100,000 population). Furthermore, the re- sult shows that most of poor countries from group two are working with increasing returns to scale (IRS). This shows that, for these countries, the output changes more than in proportion by which input changes. The case with developed coun- tries from the first group is opposite. Second, it is important to note that the health care sector in countries from group two is mainly under the control of government, thereby most of the governments give free primary health care to their citizens. The third important point is that the remaining countries are not utilizing their resources effectively, i.e., waste of resources REFERENCES [1] Alexander, C., Busch, G., & Stringer, K. (2003). Implementing and interpreting a data envelopment analy- sis model to assess the efficiency of health systems in developing countries, 14, 49–63. [2] Androutsou, L., Geitona, M., & Yfantopoulos, J. (2011). Measuring Efficiency and Productivity Across Hospitals in the Regional Health Authority of Thessaly, in Greece. Journal of Health Management, 13(2), 121–140. doi:10.1177/097206341101300201 [3] Cook, W. D., Tone, K., & Zhu, J. (2014). Data envelopment analysis: Prior to choosing a model. Omega, 44, 1–4. doi:10.1016/j.omega.2013.09.004 [4] Dash, U., Vaishnavi, S. D., & Muraleedharan, V. R. (2010). Technical Efficiency and Scale Efficiency of District Hospitals: A Case Study. Journal of Health Management, 12(3), 231–248. doi:10.1177/097206341001200302 [5] Ferreira, C., Marques, R. C., & Nicola, P. (2013). On evaluating health centers groups in Lisbon and Tagus Valley: efficiency, equity and quality. BMC Health Services Research, 13(1), 529. doi:10.1186/1472-6963-13-529 [6] Goverment of the Republic of Serbia. Fiscal stretgy for 2014th and projections for 2015-2016. Strategy, Belgarde: Goverment of the Republic of Serbia, 2013. [7] Guerra, M., de Souza, A. A., & Moreira, D. R. (2012). Performance analysis: a study using data envel- opment analysis in 26 Brazilian hospitals. Journal of Health Care Finance, 38(4), 19–35. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/22894019 [8] K.R., S. (2012). Evaluation of the efficiency of national health systems of the members of World Health Organization. Leadership in Health Services, 25(2), 139–150. doi:10.1108/17511871211221055 [9] Pelone, F., Kringos, D. S., Spreeuwenberg, P., De Belvis, A. G., & Groenewegen, P. P. (2013). How to achieve optimal organization of primary care service delivery at system level: lessons from Europe. In- ternational Journal for Quality in Health Care : Journal of the International Society for Quality in Health Care / ISQua, 25(4), 381–93. doi:10.1093/intqhc/mzt020 [10] Popescu, C., Asandului, L., & Fatulescu, P. (2014). A Data Envelopment Analysis for Evaluating Roma- nia’s Health System. Procedia - Social and Behavioral Sciences, 109(1994), 1185–1189. doi:10.1016/j.sbspro.2013.12.609 [11] Rahman, M. A., & Capitman, J. A. (2012). Care Health Practitioners Increase from California ’ s San Joaquin Valley, 38(3), 78–92. [12] Retzlaff-Roberts, D., Chang, C. F., & Rubin, R. M. (2004). Technical efficiency in the use of health care resources: a comparison of OECD countries. Health Policy, 69(1), 55–72. doi:10.1016/j.health- pol.2003.12.002 [13] Sahin, I., Ozcan, Y. a., & Ozgen, H. (2009). Assessment of hospital efficiency under health transforma- tion program in Turkey. Central European Journal of Operations Research, 19(1), 19–37. doi:10.1007/s10100-009-0121-3 [14] Shetty, U., & Pakkala, T. P. M. (2010). Technical Efficiencies of Healthcare System in Major States of India: An Application of NP-RDM of DEA Formulation. Journal of Health Management, 12(4), 501–518. doi:10.1177/097206341001200406 45 Management 2016/75 is considerably large. Policy makers should consider this a serious issue. Some countries have been found to be operat- ing at decreasing returns to scale, as their outputs do not seem to be proportional to their inputs. These countries should undertake a thorough examination of the use of the resources and improve their efficiency. Finally, as shown in the analy- sis, Serbia’s health care system ranks the 15th out of 21 analyzed systems, with a gap of 37% compared to the first ones ranked. The health care system in Serbia has been affected, during time, by the lack of reforms, the poor funding and by the lack of interest of the authorities in establishing a long-term strategy. As a result, after 1990 the population health sta- tus has been continuously deteriorating. Further analysis will show the reasons for this state and what steps should be taken to improve the current state. ACKNOWLEDGEMENT This paper is a result of a Strategic Project founded by the Ministry of Education and Science of the Republic of Serbia: Exploring modern trends of strategic management of the application of specialized management disciplines in the func- tion of the competitiveness of Serbian economy, No 179081. 46 2016/75Management About the Author [15] Shimshak, D. G., Lenard, M. L., & Klimberg, R. K. (2009). Incorporating Quality into Data Envelopment Analysis of Nursing Home Performance: A Case Study. Omega, 37(3), 672–685. doi:10.1016/j.omega.2008.05.004 [16] Varabyova, Y., & Schreyögg, J. (2013). International comparisons of the technical efficiency of the hos- pital sector: panel data analysis of OECD countries using parametric and non-parametric approaches. Health Policy (Amsterdam, Netherlands), 112(1-2), 70–9. doi:10.1016/j.healthpol.2013.03.003 [17] World Health Organisation. Global Health Observatory Data Repository. February 15, 2014. http://apps.who.int/gho/data/node.main.75 (accessed February 15, 2014). Receieved: October 2014. Accepted: May 2015. Zorica Mitrović University of Belgrade, Faculty of Organizational Sciences zorica.mitrovic@fon.bg.ac.rs Zorica Mitrović is a teaching assistant at the University of Belgrade, Faculty of Organizational Sciences. She is a member of the Serbian Project Managers Association. Her primary research areas are performance management, strategic management and project management. She has published over 30 papers in peer-reviewed conferences and journals and taken part in several national and international research projects. Mirko Vujošević University of Belgrade, Faculty of Organizational Sciences vujosevic.mirko@fon.bg.ac.rs Mirko Vujošević graduated in electrical engineering at the Belgrade University where he completed his postgraduate studies and earned his doctorate. From 1976 to 1995 he was with Mihailo Pupin Institute, Belgrade, and now he is full professor at the Faculty of Organizational Sciences, Belgrade University. He published more than 180 professional papers on different topics of operational research, reliability, maintenance, inventory control and applied mathematics. He is the author and co-author of two monographs, six textbooks, and several chapters in various monographs. Gordana Savić Univetsity of Belgrade, Faculty of Organizational Sciences, goca@fon.bg.ac.rs Gordana Savić is an assistant professor at the Faculty of Organizational Sciences. She took her PhD in May 2012 in the field of Operational Research. She has published more than 50 research and conference papers. The areas of her research interest include mathematical modelling, real-life problem optimization and efficiency evaluation by Data envelopment analysis.