Plane Thermoelastic Waves in Infinite Half-Space Caused Decision Making: Applications in Management and Engineering Vol. 4, Issue 1, 2021, pp. 153-173. ISSN: 2560-6018 eISSN: 2620-0104 DOI: https://doi.org/10.31181/dmame2104153m * Corresponding author. E-mail addresses: zizovic@gmail.com (M. Zizovic), dragan.pamucar@va.mod.gov.rs (D. Pamucar), bole@ravangrad.net (B, Miljkovic), karan972a@gmail.com (A, Karan). MULTIPLE-CRITERIA EVALUATION MODEL FOR MEDICAL PROFESSIONALS ASSIGNED TO TEMPORARY SARS-COV-2 HOSPITALS Mališa Zizovic 1, Dragan Pamucar 2*, Boža D. Miljković 3 and Aleksandra R. Karan 4 1 Faculty of Technical Sciences in Cacak, University of Kragujevac, Cacak, Serbia 2 Department of Logistics, Military academy, University of Defence in Belgrade, Belgrade, Serbia 3 Faculty of Education Sombor, University of Novi Sad, Sombor, Serbia 4 General Hospital "Dr Radivoj Simonović", Sombor, Serbia Received: 18 December 2020; Accepted: 27 February 2021; Available online: 13 March 2021. Original scientific paper Abstract: Hospitals around the world, as health institutions with a key role in the health system, face problems while providing health services to patients with various types of diseases. Currently, those problems are intensified due to the pandemic caused by SARS-CoV-2 virus. This pandemic has caused an extreme spread of the disease with constantly changing needs of patients which impacts the capacities and overall functioning of hospitals. In order to meet the challenge of the COVID-19 (COronaVIrus Disease- 2019) pandemic, health systems must adjust to new circumstances and establish separate hospitals exclusive for patients infected with SARS-CoV-2 virus. In the process of creating COVID-19 hospitals, health systems face a shortage of medical professionals trained for work in COVID-19 hospitals. Using this as a starting point, this study puts forward a two-phase model for the evaluation and selection of nurses for COVID-19 hospitals. Each phase of the model features a separate multiple-criteria model. In the first phase, a multiple-criteria model with a dominant criterion is formed and candidates who meet the defined requirements are evaluated. In the second phase, a modified multiple-criteria model is formed and used to evaluate medical professionals who do not meet the requirements of the dominant criterion. By applying this model, two groups of medical professionals are defined: 1) medical professionals who completely meet the requirements for working in COVID-19 hospitals and 2) medical professionals who require additional training. The criteria for evaluation of medical professionals in this multiple-criteria model are defined based on research conducted on medical professionals assigned to the COVID- mailto:zizovic@gmail.com mailto:zizovic@gmail.com mailto:dragan.pamucar@va.mod.gov.rs mailto:bole@ravangrad.net mailto:karan972a@gmail.com Zizovic et al./Decis. Mak. Appl. Manag. Eng. 4 (1) (2021) 153-173 154 19 Crisis Response Team during the COVID-19 pandemic in the Republic of Serbia. The model was tested on a real example of evaluating medical professionals assigned to the COVID-19 hospital in Sombor. The model for evaluating medical professionals presented in this paper can help decision makers in hospitals and national policy makers to determine the readiness level of hospitals for working in the conditions of the COVID-19 pandemic, as well as underline the areas in which hospitals are not ready to meet the challenges of the pandemic. Key words: COVID-19 pandemic; health care service; multicriteria decision making. 1. Introduction COVID-19 (COronaVIrus Disease- 2019) is a disease caused by a novel virus from the group of coronaviruses, first isolated in 1962. Since then, it is known that some viruses from the coronavirus group, infect only certain animals, some humans, while some can breach the barrier between species, causing states ranging from a mild cold to severe acute respiratory syndrome (SARS). A new, so far unknown coronavirus, SARS-CoV-2, the cause if COVID-19 disease, belongs to the same subgroup as MERS- CoV and SARS-CoV, and was first detected in the Chinese city of Wuhan, in the province of Huubei, the ground zero of the epidemic. WHO (World Health Organisation) declared a global pandemic in March 2020 and started a research mission into it. The COVID-19 pandemic came as a surprise and brought with it many challenges to health systems all over the world, including our own. The immediate demand for staff training, repurposing structures, equipment acquisition, communication management, continuous supervision, healthcare availability to special groups of population under new conditions etc. has created numerous operational, logistical, organizational and ethical tasks for managers, medical professionals and associates. Success and weaknesses of the health systems during the pandemic, is graded by the global health safety index, used for evaluating readiness in prevention, detection, fast response to high-risk environments, and following international protocols in new conditions. Due to the specifics of the COVID-19 pandemic, continuous analysis of all advantages and weaknesses of health systems and planning based on adaptive models of operation and providing services are necessary because the immediate demands and the unpredictability of the COVID-19 infection requires agility and flexibility of all services in regards to timely responses, especially in the case of medical professionals. Previous research on evaluating the readiness of hospitals and medical professionals for performance in crises and disasters is very limited. This applies especially to the application of multiple-criteria decision models in this field, which limited the analysis of already available literature. Most authors consider the application of two concepts for solving this problem: 1) application of multiple-criteria decision making for evaluating readiness of hospitals in crises and 2) application of research based on surveys and statistical analysis. Nekoie-Moghadam et al. (2016) present a comprehensive review of literature with different methodologies used for evaluating hospitals for work in crises. In their review, they considered the most important topics that relate to evaluating health systems, such as logistics, planning, human resources, communication, management and control, training, evacuation, disaster recovery, coordination, transport, safety (Fallah-Aliabadi et al. 2020; Verheul and Duckers, 2020; Nekoie-Moghadam et al., 2016). Of the 15 papers considered in Multiple-criteria Evaluation Model for Medical Professionals Assigned to Temporary… 155 total by Nekoie-Moghadam et al. (2016), all consider the application of research based on statistical analysis of data collected in surveys. In some situations, authors used statistical methodologies, such as the Delphi technique and similar tools (Rezaei and Mohebbi-Dehnavi, 2019). A comprehensive review of literature that considers the application of statistical tools for hospital readiness analysis in crises is presented in Fallah-Aliabadi et al. (2020), Verheul and Duckers (2020), Alruvaili et al. (2019) and Nekoie-Moghadam et al. (2016). In their study Tabatabaei and Abbasi (2016) carried out risk assessment during crises based on the Hospital Safety Index. The Safety Index was defined based on comprehensive research with statistical data processing. Samsuddin et al. (2018) identified key factors to determine the readiness level of hospitals for work in crises. The results have demonstrated that human resources, their training and ability to adapt in a timely fashion, are the crucial factor. Marzaleh et al. (2019.) put forward an approach where the Delphi technique is used to evaluate the readiness of emergency services in hospitals for working in crises. Their study identified 31 criteria grouped in 3 clusters. The results demonstrate that training of medical professionals has the highest priority. A similar approach for analyzing hospital capacities using the Delphi technique was demonstrated by Shabanikiya et al. (2019.). However, aside from statistical processing of factor weight used for evaluating hospitals in crises, previous research also contains a number of papers based on multiple-criteria techniques. For example, Mulyasari et al. (2013) have applied multiple-criteria techniques for ranking eight hospitals in Iran. The ranking was carried out using factors grouped in four clusters, based on which hospital structural and functional readiness in crises was evaluated. Following this study, Hosseini et al. (2019.) developed a model for ranking hospitals based on the TOPSIS (Technique for Order of Preference by Similarity to Ideal Solution) multiple-criteria technique. The study identifies 21 factors grouped in 4 clusters. However, this study used direct evaluation of survey participants instead of subjective/objective methods for determining weight coefficients of criteria. Also, Ortiz-Barrios et al. (2017) demonstrate the possibility of applying analytical multiple-criteria approach for analyzing the readiness of particular wards/clinics in hospitals during crises. The approach is a hybrid model that consists of: 1) applying the AHP (Analytic Hierarchy Process) and the DEMATEL (Decision Making Trial and Evaluation Laboratory) methods for determining weight coefficients of criteria and their mutual relationships and 2) applying the TOPSIS method for evaluating hospital capacities. Unlike previously listed studies, Ortiz-Barrios et al. (2017), in addition to defining weight coefficients of criteria, also put forward a methodology for defining mutual relationships and impact of factors used for evaluation. Following Ortiz-Barrios et al. (2017), Roy et al. (2018) identified key factors for evaluating hospital capacities using the DEMATEL method. However, unlike Ortiz-Barrios et al. (2017), Roy et al. (2018) used rough numbers to exploit the lack of certainty and precision found in expert preferences. Considering that the crisis caused by the COVID-19 pandemic is still ongoing, there is no research that considers the problem of evaluating the training of medical professionals for working in COVID-19 hospitals. There is a limited number of papers that consider the application of multiple-criteria tools for solving problems caused by the COVID-19 pandemic. Sarkar (2020) maps the areas susceptible to COVID-19 infection in Bangladesh. The AHP method, in conjunction with GIS (Geographic Information System) spatial analysis were used for area mapping. Sangiorgio and Parisi (2020) used Artificial Neural Network and GIS for mapping COVID-19 infection risks in urban zones in Italy. Their study showed spatial analyses of a total of 257 city Zizovic et al./Decis. Mak. Appl. Manag. Eng. 4 (1) (2021) 153-173 156 districts. Nardo et al. (2020) demonstrated the application of multi-criteria decision analysis for determining weights for eleven criteria in order to prioritize COVID-19 non-critical patients for admission to hospitals in healthcare settings with limited resources. Yildirim et al. (2020) evaluated the available COVID-19 treatment options in hospitals. For their evaluation, they used modified PROMETHEE (Preference Ranking Organization METHod for Enrichment of Evaluations) and VIKOR (VIseKriterijumska Opitimizacija i Kompromisno Resenje in Serbian) technique by applying fuzzy numbers. As we can see from the above-mentioned studies, there are numerous approaches relating to evaluating readiness of hospitals in crises. Most papers contribute by putting forward methodological frames that require application of research featuring statistical processing of data collected via surveys. On the other hand, based on the above-mentioned literature, we can see the significance of multiple-criteria techniques for researching topics pertaining to evaluating readiness of hospital capacities in crises. We can note a wide spectrum of multiple-criteria techniques used in literature and applied in various fields. However, the number of papers that apply these methods for evaluating hospital capacities is limited. Furthermore, the number of papers that consider the application of multiple-criteria methods for evaluating hospitals and medical professionals in the conditions of the COVID-19 pandemic is especially limited. Therefore, it is the aim of this paper to develop a multiple-criteria model for selection and evaluation of medical professionals for working in COVID-19 hospitals in the conditions of a pandemic. The suggested model deals with assessing the training of medical professionals for working in the conditions of the COVID-19 pandemic. Furthermore, the suggested model has the following advantages that improve the literature relating to the application of multiple-criteria techniques in the field of healthcare: 1. An original, multiple-criteria methodology for evaluating propriety of medical professionals for working in COVID-19 hospitals has been developed. The demonstrated methodology is conducted in two phases. A separate multiple- criteria model has been developed for each phase. The criteria and criteria value scales have been defined following months of research with the participation of medical professionals from the COVID-19 Crisis Response Team of the Republic of Serbia. 2. The demonstrated methodology is not limited to application in the healthcare field and is applicable in other fields due to its adaptability 3. The suggested methodology provides a new, clear and concise frame for resource management. In order to illustrate the effectiveness of the suggested methodology, an empirical study of the application of this multiple-criteria methodology is presented in the paper. 4. The approach presented in this paper can solve the problem of selection of medical professionals in the conditions of the COVID-19 pandemic in a systematic and analytical way. The developed model was implemented and tested in a case study of the COVID-19 hospital in Sombor (Serbia). 5. The model for evaluating medical professionals presented in this paper can help decision makers in hospitals and national policy makers to determine the readiness level of hospitals for working in the conditions of the COVID-19 pandemic, as well as underline the areas in which hospitals are not ready to meet the challenges of the pandemic. The paper is structured into four sections. After the introduction that presents the problem and analyzes the existing literature, the second section mathematically formulates the multiple-criteria model for evaluating medical professionals in the Multiple-criteria Evaluation Model for Medical Professionals Assigned to Temporary… 157 conditions of the COVID-19 pandemic. The third section presents the implementation of the multiple-criteria model on a real example of evaluating medical professionals in the COVID-19 hospital in Sombor, the Republic of Serbia. The fourth section of the paper presents the conclusion and directions for future research. 2. Multiple-criteria Evaluation Model for Medical Professionals Assigned to COVID Hospitals Let us assume a multiple-criteria model with defined criteria  1 2, ,...,j nC C C C where n stands for the number of criteria used in the multiple-criteria model. Also let us assume a set of alternatives  1 2, ,...,i nA A A A where m stands for the number of alternatives to be ranked in the model. We can define the decision matrix mxn whose elements ij a stand for value of j criterion for i alternative. Table 1. Decision matrix 1 C 2 C n C 1 A 11 a 12 a 1n a 2 A 21 a 22 a 2n a m A 1m a 2m a mn a All criteria of the set  1 2, ,...,j nC C C C were assigned weight coefficients  1 2, ,...,j nw w w w that meet the requirement of 1 1 n j j w   . Let sC stand for the dominant criterion of the set  1 2, ,...,j nC C C C . In case of the dominant criterion not being met, then the alternative that does not meet it cannot be considered a solution to the problem. Ranking of alternatives in the conditions of meeting or partially meeting the dominant criterion s C was considered by Žižović et al. (2019). In multiple- criteria models with a dominant criterion, there is a problem where it is necessary to choose more than one alternative, for example p alternative (p  yV A , we select the candidate  xV A . 3. Results This chapter considers the case study of organizing hospitals specially prepared for the admission of patients suffering from a SARS-CoV-2 infection. Such hospitals in the Republic of Serbia were organized as separate parts of existing hospital capacities or were repurposed as entire hospitals to admit only patients diagnosed with COVID- 19 during the crisis. Simultaneously, there was a need for medical professionals trained for working in the newly formed medical institutions. There was a need for priority treatment of patients suffering from a largely unknown, high-risk disease, with too few qualified medical professionals in the field. Additionally, there was a need for selection of qualified medical professionals to carry out the listed duties, as well as simultaneously train other medical professionals that, at the moment, were not trained for assignment to COVID-19 hospitals. Zizovic et al./Decis. Mak. Appl. Manag. Eng. 4 (1) (2021) 153-173 162 This study presents the application of the model for evaluating medical professionals for working in COVID-19 hospitals on the example of the COVID-19 hospital in Sombor, Republic of Serbia. The study features part of the general hospital in Sombor repurposed for treatment of COVID-19 patients. Eight criteria were identified that were used for evaluating nurses in two phases of the model. Criteria and criteria value scales for evaluation of medical professionals were defined based on a survey with members of the COVID-19 Crisis Response Team and with medical doctors who participated in the operation of the COVID-19 hospital in Sombor. All candidates for assignment in COVID-19 hospitals were psychologically tested and interviewed by teams from the COVID-19 Crisis Response Team (teams of medical doctors). Data obtained in this way served to form evaluations for criteria. Below, we present eight criteria with their value scales. The presented criteria are applied for evaluation in the first phase of the model. C1 - Experience in working with infectious and pulmonary diseases in hospital treatment (C1). Value scale:  Nurse with work experience at an infectious or pulmonary diseases ward in hospital treatment - 5 points;  Nurse with work experience at internal medicine ward in hospital treatment - 4 points;  Nurse with work experience at a different ward in hospital treatment - 2 points;  Nurse with no work experience in hospital treatment - 1 point. C2 - Professional training of the candidate for working with COVID-19 diagnosed (Covid+) patients in “COVID-19” zone:  Professional training for immediate work with COVID-19+ patients, use of personal protective equipment and materials in COVID-19 conditions, knowledge of work organization in COVID-19 conditions, training for movement through safety zones - 5 points  Professional training for immediate work with COVID-19+ patients, use of personal protective equipment and materials in COVID-19 conditions, knowledge of work organization in COVID-19 conditions - 4 points;  Professional training for immediate work with COVID-19+ patients, use of personal protective equipment and materials in COVID-19 conditions - 3 points;  Professional training for immediate work with COVID-19+ patients - 2 points;  No professional training - 0 points. C3 - Health risk of the candidate:  No health risk - 10 points;  Diseases and injuries that do not significantly affect ability - 8 points;  Physical injuries that partially affect mobility - 7 points;  Lower risk chronic diseases in COVID-19 conditions - 6 points;  Single parent of a child up to 12 years old - 3 points;  Parent of a child up to 3 years old - 2 points;  Age (over 60) - 1 point;  Chronic diseases like: diabetes, psychosomatic diseases, autoimmune diseases, malignancy, diseases or treatment with a negative influence on the immune system, pregnancy - 0 points. C4 - Physical evaluation of candidates for work in difficult working conditions and in shifts: Multiple-criteria Evaluation Model for Medical Professionals Assigned to Temporary… 163  Very capable - 5 points;  Capable - 4 points;  Partially capable - 3 points;  Barely capable - 1 point. C5 - Motivation of candidates for working in COVID-19 conditions:  Very motivated - 5 points;  Motivated - 4 points;  Somewhat motivated - 2 points;  Barely motivated - 1 point;  Not motivated - 0 points. C6 - Availability of candidate to the workplace:  Easily available - 5 points;  Available - 4 points;  Poorly available - 3 points;  Very poorly available - 1 point. C7 - Candidate reliability:  Very reliable - 5 points;  Reliable - 4 points;  Somewhat reliable - 2 points;  Unreliable - 1 point. C8 - Candidate’s vaccination history:  Mandatory vaccination (with BCG+POLIO) - 5 points;  Mandatory vaccination (without BCG/POLIO) - 4 points;  Basic vaccination – 2 points;  None - 1 point. After defining the criteria, below we present the application of the model on the selection of medical professionals. First phase: Defining an additional criterion from the set of criteria and forming a multiple-criteria model with a dominant criterion. Step 1: From the defined set of criteria  1 2 8, ,...,jC C C C for evaluating medical professionals, criterion 1 C was defined as the dominant criterion. All candidates who do not meet the conditions defined by criterion 1 C move on to the second phase of evaluation, while ranking of candidates who meet criterion 1 C is done in the first phase. Step 2: Determining criteria weight coefficients jw ( 1, 2,...,8j  ). As noted above, for determining criteria weight coefficients we used the NDSL model (Žižović et al., 2020). Determining criteria weight coefficients using the NDSL model is presented below in steps 2.1 - 2.5. Step 2.1: Determining the most important criterion from the criteria set  1 2 8, ,...,jC C C C and ranking criteria. As C1 was defined as the dominant criterion, C1 is also the most important criterion in the set jC . Based on evaluations of experts, the criteria from the set jC were ranked as follows: C1>C7>C3>C4>C5>C6>C2>C8. Step 2.2: Grouping criteria into significance levels. The criteria were grouped into four levels, as follows: Level 1 L : 1C , Level 2 L : 3, 4, 5, 7C C C C , Zizovic et al./Decis. Mak. Appl. Manag. Eng. 4 (1) (2021) 153-173 164 Level 3 L : 2, 6, 8C C C . Step 2.3: Based on relations for defining border values of the criteria ( ) the values i  were defined for criteria relating to significance levels: 5 1 2 1 7 3 4 6 2 83 0 17; 19; 20; 24; 26; 28; 29. : : : Level L Level L Level L                 The values of border values of criteria significance ( i  ) were defined for the value 50N  . Step 2.4: Criteria significance functions ( ) i f C , 1, 2,...,8i  , were defined based on relation    ( ) /i i if C N N   : 1 3 7 4 5 6 2 8 ( ) 1.000; ( ) 0.493; ( ) 0.449; ( ) 0.429; ( ) 0.351; ( ) 0.316; ( ) 0.282; ( ) 0.266. f C f C f C f C f C f C f C f C         Step 2.5: Based on defined values ( ) i f C , 1, 2,...,8i  we arrive at values of criteria weight coefficients, Table 2. Table 2. Criteria weight coefficients Criterion wj C1 0.279 C2 0.079 C3 0.137 C4 0.119 C5 0.098 C6 0.088 C7 0.125 C8 0.074 Step 3: Forming the preliminary decision matrix. The Crisis Response Team carried out the evaluation of medical professionals from the pulmonology and infectious wards of the general hospital in Sombor. These two wards counted in total 43 nurses who were candidates for working in the COVID-19 hospital, i.e. for working in the “red” and “orange” zones. A selection of 29 nurses was considered of the available 43 candidates. Based on the defined evaluation criteria and available number of medical professionals, the preliminary decision matrix was formed 43 8 ij X x      , Table 3. Table 3. Preliminary decision matrix – first phase Alt C1 C2 C3 C4 C5 C6 C7 C8 A1 5 5 10 5 4 4 5 5 A2 2 5 10 5 5 4 5 5 A3 5 5 10 5 5 4 4 5 A4 1 2 0 1 1 5 1 5 A5 5 5 10 4 5 5 5 4 A6 5 5 10 3 5 5 5 5 A7 5 5 7 3 5 3 5 5 Multiple-criteria Evaluation Model for Medical Professionals Assigned to Temporary… 165 Alt C1 C2 C3 C4 C5 C6 C7 C8 A8 2 5 7 5 2 4 4 5 A9 5 5 6 4 4 5 5 2 A10 5 5 8 4 5 5 5 5 A11 5 5 10 5 2 4 2 5 A12 5 5 2 5 4 4 5 5 A13 1 5 10 5 5 5 5 5 A14 5 5 6 5 0 3 2 5 A15 5 5 10 4 5 3 5 5 A16 1 0 0 1 0 5 5 5 A17 5 5 2 5 2 5 2 5 A18 1 5 3 5 2 1 4 5 A19 5 5 10 5 5 5 5 2 A20 5 5 6 3 2 3 4 5 A21 5 5 10 5 5 5 5 5 A22 1 4 8 4 2 5 5 5 A23 2 4 7 3 0 5 5 4 A24 2 3 7 3 1 5 5 5 A25 2 3 7 3 1 5 5 5 A26 5 5 6 4 5 3 4 5 A27 5 3 1 1 1 5 5 5 A28 5 2 1 3 1 5 5 2 A29 2 3 1 1 1 5 5 2 A30 5 5 2 3 2 1 5 5 A31 2 5 10 5 5 5 5 5 A32 5 5 7 3 5 5 1 4 A33 5 5 10 5 5 5 5 4 A34 2 5 8 5 5 1 4 4 A35 5 5 10 5 4 5 2 5 A36 5 5 10 5 5 5 5 5 A37 5 5 10 5 5 4 4 5 A38 2 5 6 4 4 3 4 5 A39 1 5 10 5 5 5 5 5 A40 2 5 10 5 5 4 5 5 A41 2 5 6 4 2 5 4 5 A42 5 5 10 5 5 1 5 5 A43 5 5 10 5 5 5 5 5 Step 4: Normalization of elements of the preliminary decision matrix. Since all the considered criteria fall under the max type (higher value is better), for the normalization of values we used the expression (1). The elements of the normalized matrix are shown in Table 4. Table 4. Normalized decision matrix – first phase Alt. C1 C2 C3 C4 C5 C6 C7 C8 A1 1.0 1.0 1.0 1.0 0.8 0.8 1.0 1.0 A2 0.4 1.0 1.0 1.0 1.0 0.8 1.0 1.0 A3 1.0 1.0 1.0 1.0 1.0 0.8 0.8 1.0 A4 0.2 0.4 0.0 0.2 0.2 1.0 0.2 1.0 A5 1.0 1.0 1.0 0.8 1.0 1.0 1.0 0.8 A6 1.0 1.0 1.0 0.6 1.0 1.0 1.0 1.0 Zizovic et al./Decis. Mak. Appl. Manag. Eng. 4 (1) (2021) 153-173 166 Alt. C1 C2 C3 C4 C5 C6 C7 C8 A7 1.0 1.0 0.7 0.6 1.0 0.6 1.0 1.0 A8 0.4 1.0 0.7 1.0 0.4 0.8 0.8 1.0 A9 1.0 1.0 0.6 0.8 0.8 1.0 1.0 0.4 A10 1.0 1.0 0.8 0.8 1.0 1.0 1.0 1.0 A11 1.0 1.0 1.0 1.0 0.4 0.8 0.4 1.0 A12 1.0 1.0 0.2 1.0 0.8 0.8 1.0 1.0 A13 0.2 1.0 1.0 1.0 1.0 1.0 1.0 1.0 A14 1.0 1.0 0.6 1.0 0.0 0.6 0.4 1.0 A15 1.0 1.0 1.0 0.8 1.0 0.6 1.0 1.0 A16 0.2 0.0 0.0 0.2 0.0 1.0 1.0 1.0 A17 1.0 1.0 0.2 1.0 0.4 1.0 0.4 1.0 A18 0.2 1.0 0.3 1.0 0.4 0.2 0.8 1.0 A19 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.4 A20 1.0 1.0 0.6 0.6 0.4 0.6 0.8 1.0 A21 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 A22 0.2 0.8 0.8 0.8 0.4 1.0 1.0 1.0 A23 0.4 0.8 0.7 0.6 0.0 1.0 1.0 0.8 A24 0.4 0.6 0.7 0.6 0.2 1.0 1.0 1.0 A25 0.4 0.6 0.7 0.6 0.2 1.0 1.0 1.0 A26 1.0 1.0 0.6 0.8 1.0 0.6 0.8 1.0 A27 1.0 0.6 0.1 0.2 0.2 1.0 1.0 1.0 A28 1.0 0.4 0.1 0.6 0.2 1.0 1.0 0.4 A29 0.4 0.6 0.1 0.2 0.2 1.0 1.0 0.4 A30 1.0 1.0 0.2 0.6 0.4 0.2 1.0 1.0 A31 0.4 1.0 1.0 1.0 1.0 1.0 1.0 1.0 A32 1.0 1.0 0.7 0.6 1.0 1.0 0.2 0.8 A33 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.8 A34 0.4 1.0 0.8 1.0 1.0 0.2 0.8 0.8 A35 1.0 1.0 1.0 1.0 0.8 1.0 0.4 1.0 A36 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 A37 1.0 1.0 1.0 1.0 1.0 0.8 0.8 1.0 A38 0.4 1.0 0.6 0.8 0.8 0.6 0.8 1.0 A39 0.2 1.0 1.0 1.0 1.0 1.0 1.0 1.0 A40 0.4 1.0 1.0 1.0 1.0 0.8 1.0 1.0 A41 0.4 1.0 0.6 0.8 0.4 1.0 0.8 1.0 A42 1.0 1.0 1.0 1.0 1.0 0.2 1.0 1.0 A43 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 Step 5: Evaluation and selection of medical professionals who meet the dominant criterion 1 C . Medical professionals who have the value of the dominant criterion 1 4C  enter the evaluation process in the first phase of selection. Border value for the dominant criterion is defined based on the evaluation of experts. The remaining candidates who do not meet the requirement move on to the second phase of selection. Based on the analysis of the preliminary decision matrix (Table 3) we see that 26 candidates meet the requirement of 1 4C  . By applying the expression (3) we evaluate and rank the candidates, Table 5. Multiple-criteria Evaluation Model for Medical Professionals Assigned to Temporary… 167 Table 5. Grades of candidates who meet the requirement of the dominant criterion Evaluation function V(Ai) Rank V(A21)=0.279+λ 0.72 0.3690 1 V(A36)=0.279+λ 0.72 0.3690 1 V(A43)=0.279+λ 0.72 0.3690 1 V(A33)=0.279+λ 0.705 0.3672 4 V(A1)=0.279+λ 0.683 0.3644 5 V(A5)=0.279+λ 0.681 0.3642 6 V(A3)=0.279+λ 0.677 0.3637 7 V(A37)=0.279+λ 0.677 0.3637 7 V(A19)=0.279+λ 0.676 0.3635 9 V(A6)=0.279+λ 0.672 0.3631 10 V(A10)=0.279+λ 0.669 0.3626 11 V(A15)=0.279+λ 0.661 0.3616 12 V(A42)=0.279+λ 0.65 0.3602 13 V(A35)=0.279+λ 0.625 0.3572 14 V(A7)=0.279+λ 0.596 0.3535 15 V(A26)=0.279+λ 0.581 0.3517 16 V(A9)=0.279+λ 0.577 0.3512 17 V(A12)=0.279+λ 0.573 0.3507 18 V(A11)=0.279+λ 0.569 0.3501 19 V(A32)=0.279+λ 0.517 0.3436 20 V(A20)=0.279+λ 0.499 0.3413 21 V(A17)=0.279+λ 0.477 0.3386 22 V(A14)=0.279+λ 0.457 0.3361 23 V(A30)=0.279+λ 0.434 0.3332 24 V(A27)=0.279+λ 0.392 0.3279 25 V(A28)=0.279+λ 0.379 0.3264 26 V(A31)=0.112+λ 0.288 0.1476 27 V(A2)=0.112+λ 0.281 0.1467 28 V(A40)=0.112+λ 0.281 0.1467 28 V(A34)=0.112+λ 0.233 0.1407 30 V(A8)=0.112+λ 0.231 0.1405 31 V(A38)=0.112+λ 0.225 0.1397 32 V(A41)=0.112+λ 0.223 0.1395 33 V(A24)=0.112+λ 0.209 0.1377 34 V(A25)=0.112+λ 0.209 0.1377 34 V(A23)=0.112+λ 0.201 0.1367 36 V(A29)=0.112+λ 0.139 0.1290 37 V(A13)=0.056+λ 0.144 0.0738 38 V(A39)=0.056+λ 0.144 0.0738 38 V(A22)=0.056+λ 0.119 0.0707 40 V(A18)=0.056+λ 0.094 0.0675 41 V(A16)=0.056+λ 0.062 0.0636 42 V(A4)=0.056+λ 0.052 0.0624 43 Zizovic et al./Decis. Mak. Appl. Manag. Eng. 4 (1) (2021) 153-173 168 An example of defining a candidate grade  1V A , on the condition that 0.125  (expression (3)), is given below:   1.0 0.279 1.0 0.125(1.0 0.079 1.0 0.137 ... 1.0 0.074) 0.364iV A            In a similar way, the remaining candidates’ grades were attained, as shown in Table 5. The mandatory criteria were met by 26 candidates, who were ranked in Table 5. Since 29 candidates were needed for the COVID-19 hospital, the remaining 17 candidates were ranked in the second phase of the model with the aim of selecting 3 additional, most appropriate candidates. Second phase: Forming a modified multiple-criteria model. In the modified multiple-criteria model, we adjust the starting set of criteria. Criteria C1 and C2 are eliminated and in their place, we insert new ones: * 1 C - Evaluation of speed of acquiring knowledge and skills for working in COVID-19 institutions:  Very quick and safe start of work activities – 5 points;  Sufficiently quick and safe start of work activities – 4 points;  Satisfactorily quick and safe start of work activities – 3 points;  Slow but safe start of work activities – 2 points;  Slow and barely safe start of work activities – 1 point. * 2 C - School grades in subjects close to the needs of the position. For this criterion, an average grade was taken, from the interval [2, 5]. The remaining criteria were unchanged, same as in the first phase, i.e. * 1 C , * 2 C , * 3 3 C C , * 4 4 C C , * 5 5 C C , * 6 6 C C , * 7 7 C C , * 8 8 C C . This forms the final set of criteria used for the evaluation of medical professionals in the second phase  * * * *1 2 8, ,...,jC C C C . The modified multiple-criteria model with a newly defined criteria set * j C ( 1, 2,...,8j  ) is conducted in four steps presented below. Step 1: Defining the criteria set  * * * *1 2 8, ,...,jC C C C and calculation of newly formed set of criteria weight coefficients * j w ( 1, 2,...,8j  ). Similar to the first phase (Step 2), weight coefficients are defined using the NDSL model (Žižović et al., 2020). Step 1.1: Based on the evaluation of experts, the criteria from the set  * * * *1 2 8, ,...,jC C C C were ranked as follows: C1*>C7*>C4*>C6*>C3*>C8*>C5*>C2*. Step 1.2: The criteria were grouped into sets of four levels, as follows: Level 1 L : * * *1 7 4, ,C C C , Level 2 L : * *6 3,C C , Level 3 L : * *8 5,C C , Level 4 L : *2C . Step 1.3: Based on relations for defining border values of criteria significance ( i  ) we define the values i  for criteria in significance levels: 2 1 7 4 6 3 8 5 2 1 3 4 0; 5; 10 18; 20; 26; 28; 32. : : : : Level L Level L Level L Level L                 Multiple-criteria Evaluation Model for Medical Professionals Assigned to Temporary… 169 The values of border values of criteria significance ( i  ) were defined for the value 50N  . Steps 1.4 and 1.5: Based on the defined values * ( ) i f C , 1, 2,...,8i  we arrive at the values of criteria weight coefficients, Table 6. Table 6. Criteria weight coefficients Criterion * j w C1* 0.238 C2* 0.052 C3* 0.102 C4* 0.158 C5* 0.067 C6* 0.112 C7* 0.194 C8* 0.077 Step 2: Forming the preliminary decision matrix. Since the minimum criteria were not met by 17 of 43 candidates, we form the preliminary decision matrix of rank , Table 7. Table 7. Preliminary decision matrix - second phase Alt. C1* C2* C3* C4* C5* C6* C7* C8* A2 5 5 10 5 5 4 5 5 A4 5 2 0 1 1 5 1 5 A8 2 5 7 5 2 4 4 5 A13 2 5 10 5 5 5 5 5 A16 4 0 0 1 0 5 5 5 A18 5 5 3 5 2 1 4 5 A22 4 4 8 4 2 5 5 5 A23 4 4 7 3 0 5 5 4 A24 2 3 7 3 1 5 5 5 A25 5 3 7 3 1 5 5 5 A29 4 3 1 1 1 5 5 2 A31 4 5 10 5 5 5 5 5 A34 2 5 8 5 5 1 4 4 A38 4 5 6 4 4 3 4 5 A39 4 5 10 5 5 5 5 5 A40 4 5 10 5 5 4 5 5 A41 5 5 6 4 2 5 4 5 Steps 3 and 4: Normalization of elements of the preliminary decision matrix is done by applying the expression (4). After the normalization of the decision matrix elements, by applying expression (6) we define the grade for each candidate. The final grades and candidate ranking are shown in Table 8. Zizovic et al./Decis. Mak. Appl. Manag. Eng. 4 (1) (2021) 153-173 170 Table 8. Candidate grades after second evaluation phase Alt. V(Ai) Rank V(A2) 0.9776 1 V(A31) 0.9524 2 V(A39) 0.9524 2 V(A40) 0.9300 4 V(A13) 0.8572 5 V(A22) 0.8498 6 V(A41) 0.8486 7 V(A25) 0.8318 8 V(A38) 0.7830 9 V(A23) 0.7658 10 V(A18) 0.7600 11 V(A8) 0.7252 12 V(A34) 0.6930 13 V(A24) 0.6890 14 V(A29) 0.6136 15 V(A16) 0.6050 16 V(A4) 0.5316 17 After the evaluation of candidates (Table 8), three best ranked candidates were selected, and after completing a training cycle, were assigned to a COVID-19 hospital. The training program is defined by the Crisis Response Team. The remaining 14 candidates also completed the training program but are currently not assigned to the COVID-19 hospital. They are available for assignment in case of assigned staff being removed from the team for self-isolation. Self-isolation may be a consequence of accidental exposure (human error, breakdown of equipment, etc.) or disease. 5. Conclusions Management of human resources is a key segment that affects the efficacy of the health system of every country. This is especially obvious in crises, like the COVID-19 pandemic. This is why it is necessary to efficiently manage human resources in hospitals, to reduce as much as possible the dangers caused by the COVID-19 pandemic. As far as the authors are aware, there are no current models for considering the evaluation of medical professionals’ training for working in crises, so the motivation for a study such as this is logical. In this paper, we put forward a multiple-criteria model that allows decision makers in medical institutions and national crisis response teams to evaluate the training of medical professionals for working in the conditions of the COVID-19 pandemic. For the needs of this multiple-criteria model, we defined criteria based on which we evaluate medical professionals. The criteria and criteria evaluation scales were defined after months of research with participation from health institution managers and members of the Crisis Response Team of Serbia. The developed multiple-criteria model is conducted in two phases. The first phase evaluates medical professionals according to one or more dominant criteria. Medical professionals who meet the conditions defined in the first phase, meet the conditions necessary for working in a COVID-19 hospital. Medical professionals who do not meet the conditions defined in Multiple-criteria Evaluation Model for Medical Professionals Assigned to Temporary… 171 the first phase, move on to the second phase of evaluation. After completing the second phase of evaluation, the staff who partially meet the conditions are identified and they undergo training for working in COVID-19 hospitals. This methodology was applied to the example of the COVID-19 hospital in Sombor. The suggested methodology can be used for other decision problems, by adapting the criteria according to the nature of the decision problem. The basic advantage of this study is application, i.e. testing of the suggested methodology on objective data in the conditions of the COVID-19 pandemic. This demonstrates, on a real example, all advantages of this methodology. Future research should be directed towards implementing the suggested methodology in the conditions of uncertain input model parameters (Ecer and Pamucar, 2020). Uncertainty in future research can be exploited by applying various uncertainty theories such as fuzzy theory or rough theory. Author Contributions: Each author has participated and contributed sufficiently to take public responsibility for appropriate portions of the content. Funding: This research received no external funding. Conflicts of Interest: The authors declare that there is no conflict of interest. References Behzad, M., Hashemkhani Zolfani, S., Pamucar, D., & Behzad, M. (2020). A Comparative Assessment of Solid Waste Performance Management in the Nordic Countries based on BWM-EDAS. Journal of Cleaner Production, 266 122008, https://doi.org/10.1016/j.jclepro.2020.122008. Ecer, F., & Pamucar, D. (2020). Sustainable supplier selection: A novel integrated fuzzy best worst method (F-BWM) and fuzzy CoCoSo with Bonferroni (CoCoSo'B) multi- criteria model. Journal of Cleaner Production, 266, 121981. https://doi.org/10.1016/j.jclepro.2020.121981. Fallah-Aliabadi, S., Ostadtaghizadeh, A., Ardalan, A., Fatemi, F., Khazai, B., & Mirjalili, M. R. (2020). Towards developing a model for the evaluation of hospital disaster resilience: a systematic review. BMC health services research, 20(1), 64-73. Hosseini, S. M., Bahadori, M., Raadabadi, M., & Ravangard, R. (2019). Ranking hospitals based on the disasters preparedness using the TOPSIS technique in western Iran. Hospital topics, 97(1), 23-31. Marzaleh, M. A., Rezaee, R., Rezaianzadeh, A., Rakhshan, M., Hadadi, G., & Peyravi, M. (2019). Emergency Department Preparedness of Hospitals for Radiation, Nuclear Accidents, and Nuclear Terrorism: A Qualitative Study in Iran. Iranian Red Crescent Medical Journal, 21(5), 300–306. Mulyasari, F., Inoue, S., Prashar, S., Isayama, K., Basu, M., Srivastava, N., & Shaw, R. (2013). Disaster preparedness: Looking through the lens of hospitals in Japan. International Journal of Disaster Risk Science, 4(2), 89-100. Nardo, P.D., Gentilotti, E., Mazzaferri, F., Cremonini, E., Hansen, P., Goossens, H., & Tacconelli, E. (2020). Multi-Criteria Decision Analysis to prioritize hospital admission Zizovic et al./Decis. Mak. Appl. Manag. Eng. 4 (1) (2021) 153-173 172 of patients affected by COVID-19 in low-resource settings with hospital-bed shortage. International Journal of Infectious Diseases, 98, 494-500. Nekoie-Moghadam, M., Kurland, L., Moosazadeh, M., Ingrassia, P. L., Della Corte, F., & Djalali, A. (2016). Tools and checklists used for the evaluation of hospital disaster preparedness: a systematic review. Disaster medicine and public health preparedness, 10(5), 781-788. Ortiz‐Barrios, M. A., Aleman‐Romero, B. A., Rebolledo‐Rudas, J., Maldonado‐Mestre, H., Montes‐Villa, L., De Felice, F., & Petrillo, A. (2017). The analytic decision‐making preference model to evaluate the disaster readiness in emergency departments: The ADT model. Journal of Multi‐Criteria Decision Analysis, 24(5-6), 204-226. Pamucar, D., Behzad, M., Bozanic, D., & Behzad, M. (2020c). Decision making to support sustainable energy policies corresponding to agriculture sector: Case study in Iran’s Caspian Sea coastline. Journal of Cleaner Production, 292, 125302, https://doi.org/10.1016/j.jclepro.2020.125302. Pamucar, D., Cirovic, G., Zizovic, M.M., & Miljkovic, B. (2020a). A Model for Determining Weight Coefficients by Forming a Non-Decreasing Series at Criteria Significance Levels (NDSL). Mathematics, 8(5), 745. Pamucar, D., Yazdani, M., Obradovic, R., Kumar, A., & Torres-Jiménez, M. (2020b). A Novel fuzzy hybrid Neutrosophic decision-making approach for the resilient supplier selection problem. International Journal of Intelligent Systems, 35(12), 1934-1986. Rezaei, F., & Mohebbi-Dehnavi, Z. (2019). Evaluation of the readiness of hospitals affiliated to Isfahan University of Medical Sciences in unexpected events in 2017. Journal of education and health promotion, 8. Roy, J., Adhikary, K., Kar, S., & Pamucar, D. (2018). A rough strength relational DEMATEL model for analysing the key success factors of hospital service quality. Decision Making: Applications in Management and Engineering, 1(1), 121-142. Samsuddin, N. M., Takim, R., Nawawi, A. H., & Alwee, S. N. A. S. (2018). Disaster preparedness attributes and hospital’s resilience in Malaysia. Procedia engineering, 212, 371-378. Sangiorgio, V., & Parisi, P. (2020). A multicriteria approach for risk assessment of Covid-19 in urban district lockdown. Safety Science, 130, 104862, https://doi.org/10.1016/j.ssci.2020.104862. Sarkar, S. (2020). COVID-19 Susceptibility Mapping Using Multicriteria Evaluation. Disaster Medicine and Public Health Preparedness, 14(4), 1-17. Shabanikiya, H., Jafari, M., Gorgi, H. A., Seyedin, H., & Rahimi, A. (2019). Developing a practical toolkit for evaluating hospital preparedness for surge capacity in disasters. International journal of disaster risk reduction, 34, 423-428. Tabatabaei, S. A. N., & Abbasi, S. (2016). Risk assessment in social security hospitals of Isfahan Province in case of disasters based on the hospital safety index. International Journal of Health System and Disaster Management, 4(3), 82. Verheul, M. L., & Dückers, M. L. (2020). Defining and operationalizing disaster preparedness in hospitals: a systematic literature review. Prehospital and disaster medicine, 35(1), 61-68. Multiple-criteria Evaluation Model for Medical Professionals Assigned to Temporary… 173 Yildirim, F.S., Sayan, M., Sanlidag, T., Uzun, B., Uzun, D., & Ozsahin, I. (2020). A Clinical Decision Support System for the Treatment of COVID-19 with Multi-Criteria Decision- Making Techniques. JMIR Medical Informatics, 22, 88-106. Zizovic, M.M, Albijanic, M., Jovanović, V., & Zizovic, M (2019). A New Method of Multi- Criteria Analysis for Evaluation and Decision Making by Dominant Criterion. Informatica, 30(4), 819-832. © 2018 by the authors. Submitted for possible open access publication under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).