WSC' 16 Preparing Manuscripts IJAHP Article: Dehghanimohammadabadi, Kabadayi/A two-stage AHP multi-objective simulation optimization approach in healthcare International Journal of the Analytic Hierarchy Process 117 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 A TWO-STAGE AHP MULTI-OBJECTIVE SIMULATION OPTIMIZATION APPROACH IN HEALTHCARE Mohammad Dehghanimohammadabadi Mechanical and Industrial Engineering Department Northeastern University Boston, MA, USA m.dehghani@northeastern.edu Nihan Kabadayi Department of Production Management Istanbul University School of Business Avcilar, Istanbul, Turkey nihank@istanbul.edu.tr. ABSTRACT Quality of care is crucial for patients' satisfaction and safety in healthcare centers. The majority of hospitals attempt to implement facility-wide improvements to ensure high- quality care delivery. This study proposes a combined simulation-optimization (SO) and multi-criteria decision making (MCDM) approach to accurately assess the impact of quality improvement initiatives on different facets of the healthcare system. In this framework, first, the importance (weights) of the different healthcare criteria is determined by health providers using the Analytic Hierarchy Process (AHP) approach. Then, the weights provided by the AHP are applied in a simulation-optimization environment to determine the most efficient action that leads to the most desirable quality of care. Simulation provides a platform to examine the effectiveness of different improvement efforts and calculate their impact on the system performance measures. The proposed model is generic enough to be applied to similar problems in different domains. Keywords: Analytic Hierarchy Process; multi-criteria decision making; simulation- optimization; healthcare operations; simheuristic; simio 1. Introduction Simulation is a promising technique to study complex, stochastic, and non-linear systems. Within simulation, the discrete-event simulation (DES) is a popular approach with the ability to mimic the dynamics of real systems (Moon & Phatak, 2005). DES provides a well-established mechanism for many types of modeling processes (Alt & Lieberman, 2010) and is an effective decision-making tool for tactical and operational level decisions (Dehghanimohammadabadi, 2016). Using DES, a decision-maker can compare different solutions (scenarios) and evaluate their impact on the system’s performance. However, determining the most efficient and practical solution requires a great deal of effort with careful analysis to ensure reliable results. This becomes even more challenging in problems with multiple responses (criteria), in which the system needs to be optimized with respect to multiple objectives. mailto:m.dehghani@northeastern.edu mailto:nihank@istanbul.edu.tr IJAHP Article: Dehghanimohammadabadi, Kabadayi/A two-stage AHP multi-objective simulation optimization approach in healthcare International Journal of the Analytic Hierarchy Process 118 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 The existing DES software packages are usually equipped with a built-in optimizer tool, such as OptQuest. This feature enables users to optimize the expected performance of the simulated model based on pre-determined objectives or criteria. In the case of multi- objective problems, these optimizers can use a weighted sum method to transform all of the objectives into a single-objective by varying weights in [0, 1]. Therefore, determining the weights of objectives (criteria) is a critical and somewhat challenging step in deploying multi-objective simulation-optimization (MSO). To address this challenge, this study proposes a general conceptual framework for integrating one of the most extensively used MCDM methods, the Analytic Hierarchy Process (AHP), with a DES module. Saaty’s AHP (Saaty, 1980) is a pairwise comparison method designed to capture relative judgments in a manner that ensures consistency (Chen, 2006). This algorithm provides an effective procedure to deal with complex decision-making and can assist in identifying and weighing criteria (Pun et al., 2017). As depicted in Figure 1, the development of this model has two stages. The first stage takes advantage of AHP to include several decision-makers’ preferences to determine the weight of the objectives. In the second stage, the SO model takes the weights calculated from the AHP to form a weighted sum of objective functions and combines them into a scalar fitness function. Using the new single-objective model, SO explores the solution space in order to obtain the best configuration for the simulation model. Therefore, the main contributions of this paper are listed below:  Developing a hybrid MCDM-SO model considering multiple objectives,  Providing implementation aspects of the proposed model by integrating three modules, and  Promoting the advantage of the new proposed model in a healthcare setting. The rest of this paper is organized as follows. Section 2 reviews some related literature and discusses the novelty of this work. In section 3, an effort is made to explain the implementation details of the proposed model and interactions between its components. To illustrate the applicability of this hybrid model, a healthcare case study is discussed in Section 4 followed by a sensitivity analysis. This paper is concluded in Section 5 by giving some remarks and future speculations. Figure 1 General structure of the integrated Simulation-Optimization and MCDM framework IJAHP Article: Dehghanimohammadabadi, Kabadayi/A two-stage AHP multi-objective simulation optimization approach in healthcare International Journal of the Analytic Hierarchy Process 119 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 2. Literature review The combination of simulation and MCDM methods has been used in different areas of research. Hsu and Pan (2009) proposed an integrated model to rank dental quality attributes. In this model, Monte Carlo simulation and the AHP method are combined to determine the service quality dimensions of dental services. Their application results show that the quality perceptions of patients and service providers are different. Eskandari et al. (2011) offered an integrated model to reduce waiting times of patients in an emergency department (ED) at a governmental hospital in Tehran, Iran. In the proposed model, the simulation was used to create different scenarios of patient flow processes, while the AHP and the technique for order preference by similarity to ideal solution (TOPSIS) methods were applied to evaluate the performance of alternative scenarios. Baccouche et al. (2011) developed a two-stage model to solve a supply chain design problem. In the first stage of the model, they used the crowding clustering genetic algorithm (CCGA) to measure the performance of alternative designs using simulation. After they obtained enough alternatives with acceptable performance in the first step, they aimed to determine a collective design that could satisfy the expectations of all of the members of the decision-making team in the second step of the problem. A multiplicative variant of the popular AHP method was applied for the second stage of the problem. In a study conducted by Meng (2015), an integrated DES and AHP method to develop a model to improve the process of the design of a grafting operation was proposed. Gul et al. (2016) combined a computer simulation method with MCDM methods for interval type-2 fuzzy AHP and ELECTRE to evaluate the performance of an ED in a university hospital. They suggested that the integrated method was a suitable method to assess the performance of the ED. In addition, the method helps to determine the optimal number of nurses and doctors for three shifts by trying different scenarios. Baležentis & Streimikiene (2017) conducted a study to develop a multi-criteria ranking model to determine the most suitable energy planning for the European Union (EU). Their model was a combination of three methods, weighted aggregated sum/product assessment (WASPS), the additive ratio assessment (ARAS) method, and TOPSIS, to evaluate alternative energy planning scenarios. In the proposed model, Monte Carlo simulation was used to generate the egalitarian weights to supplement MCDM methods. In another study, Bamakan and Dehghanimohammadabadi (2015) introduced a new quantitative risk analysis and assessment methodology by integrating AHP and Monte Carlo simulation. In this article, the AHP is applied to create favorable weights for security characteristic criteria. Then, a Monte Carlo simulation is utilized to handle the stochastic nature of risk assessment. None of the existing models address the applicability of MCDM approaches to perform a multi-objective simulation-optimization. This study aims to encourage the advancement of MCDM models such as AHP, within the simulation optimization environment. The proposed framework takes advantage of both worlds for the first time to create a decision support system (DSS) based on a simulation model. 3. The two-stage AHP-multi-objective SO framework structure The proposed model deploys simulation-optimization to solve stochastic problems with multiple objectives. This hybrid model consists of two main modules (i) an MCDM IJAHP Article: Dehghanimohammadabadi, Kabadayi/A two-stage AHP multi-objective simulation optimization approach in healthcare International Journal of the Analytic Hierarchy Process 120 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 module that deploys AHP, and (ii) a simulation-optimization module that optimizes the performance of the simulated system. This framework starts with a decision-making module (AHP), where experts’ opinions are taken into account to attain the relative importance of performance criteria and their weights. By summing the objective functions multiplied by weighting coefficients, the multi-objective model is transformed into a single-objective function. Then, in an iterative manner, SO leverages this single-objective function to find an optimal or close- to-optimal configuration of the simulated system. In the following sections, an effort is made to explain the importance of each of these components and their detailed procedure, followed by a discussion about their interaction. 3.1 Decision making module: Analytic Hierarchy Process (AHP) The AHP is one of the popular MCDM methods, which was developed by Saaty (1980). Since then, it has been used to solve different kinds of decision-making problems (supplier selection, facility location analysis, forecasting, choice of technology, risk modeling, performance evaluation, etc.) in the literature. It is an appropriate method for analyzing complex real-life problems as it allows experts to incorporate their knowledge and experience to generate a solution. One of the main advantages of this method is its relative ease in handling multiple criteria. In addition, the AHP allows both qualitative and quantitative data to be evaluated effectively. Therefore, it is a suitable method to solve healthcare performance evaluation problems, which include both tangible and intangible criteria. The AHP application process can be summarized in six steps as listed below: Step 1: Define the problem and construct the hierarchy of the problem; Step 2: Compute weights of the criteria with the help of pairwise comparisons of experts’ judgment. Construct a pairwise comparison matrix (size 𝑛 × 𝑛) which is composed of the values that describes the relative importance between two alternatives. In this step, pairwise comparisons are performed using Saaty’s (1980) fundamental 9-point scale; Step 3: Normalize pairwise comparison matrix; Step 4: Calculate the weights of alternatives using the pairwise comparison matrix. The weight vector (𝑤) is calculated by computing the average of each row of the decision matrix developed in the first step; Step 5: Measure the consistency within the pairwise comparison matrix. The consistency index (𝐶𝐼) is computed using Equation 1, where 𝜆𝑚𝑎𝑥 is the unique largest eigenvalue and 𝑛 is the matrix size. 𝐶𝐼 = 𝜆𝑚𝑎𝑥−𝑛 𝑛 − 1 (1) Step 6: Finally, the acceptance of the consistency of each judgment matrix is tested. The consistency ratio (𝐶𝑅) is defined using Equation 2, where an average random consistency index (𝑅𝐼) is used according to the size of each comparison matrix. The decision is accepted when 𝐶𝑅 < 0.1. IJAHP Article: Dehghanimohammadabadi, Kabadayi/A two-stage AHP multi-objective simulation optimization approach in healthcare International Journal of the Analytic Hierarchy Process 121 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 𝐶𝑅 = 𝐶𝐼 𝑅𝐼 (2) It needs to be noted that, in any AHP analysis, the number of experts depends on many factors, namely their availability, the level of their heterogeneity, experience, and domain knowledge (Karczmarek et al., 2017). Obviously, having a large number of experts’ opinions helps the aggregation process to be more effective and makes the differences between the preferences distinct. As a result, the weights provided by the AHP would be more reliable to form a robust single-objective function to be used by the optimization module. The following sections describe how the AHP results are utilized in an SO model to solve the problem. 3.2 Optimization module: Genetic Algorithm (GA) Many studies could successfully employ metaheuristics to develop simulation- optimization models and solve large-scale complex stochastic problems with reasonable computer resource consumption. These models are called simheuristics, where a metaheuristic algorithm is used in conjunction with a simulation model to find the optimal or near-to-optimal configuration of the simulation model (Dehghanimohammadabadi et al., 2017; Juan et al., 2015). Genetic algorithm (GA) is one of the popular metaheuristic algorithms, which mimics biological evolution (Dehghanimohammadabadi & Keyser, 2017). GA is based on the assumption that the potential solution of any problem can be represented by a set of parameters that are referred to as genes of a chromosome. The degree of goodness of the chromosome for the problem is reflected by a positive fitness value related to the objective value of the problem (Man, Tang & Kwong, 1999). A random population of 𝑁 individuals, which is composed of potential solutions to the problem, is created at the beginning of the GA search. Then, these individuals are evaluated for their so-called fitness, i.e. in this case, the weighted-sum of the model’s criteria. Then, individuals with the higher fitness scores are selected to create a mating pool of size 𝑁 . This created population evolves in successive generations steps until a predetermined termination condition is satisfied (Marseguerra, Zio & Podofillini, 2002). Genetic operators, such as crossover and mutation are applied in a probabilistic manner to some individuals from the mating pool to produce offspring for the next generation (Chambers, 2019). These operators aim to create a new generation that contains better offspring (solution) to the problem The crossover operator is based on the exchange of subtrees while the mutation is based on the random change in the tree (Kokol et al., 2012). GA performance could be affected by the setting values for various parameters, such as crossover rate, population size, and mutation rate. Therefore, parameter optimization is one of the critical steps of GA. This algorithm is designed to keep a delicate balance between the exploration of the feasible domain and the exploitation of good solutions (Carson & Maria, 1997). Due to its wide applicability and efficiency, GA has been successfully used in several SO studies such as facility layout optimization (Azadivar & Wang, 2000), risk management (Yin, Win & Hsu, 2017), scheduling (Dahghanimohammadabadi, 2016; Al-Dhaheri, Jebali & Diabat, 2016), and supply chain management (Göçken, 2017, 2015). As a result, GA is selected to complete this hybrid model. IJAHP Article: Dehghanimohammadabadi, Kabadayi/A two-stage AHP multi-objective simulation optimization approach in healthcare International Journal of the Analytic Hierarchy Process 122 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 The GA process can be summarized in four steps as listed below: Step 1: Selection of a pair of individuals as parents that are going to transmit their genetic material to the next generation; Step 2: Crossover of the parents, with generation of two children; Step 3: Genetic mutation is applied to maintain the diversity of the population; Step 4: Replacement in the population, so as to maintain the population number N constant. Since this model leverages a combination of simulation and optimization, each generated solution in GA needs to be tested in a simulated environment. Details of the simulation model approach and its interaction with the optimization module are described in the following sections. 3.3 Simulation module: Discrete Event Simulation (DES) Discrete event simulation is a great tool to model complex systems and the interactions between individuals and their environments (Alshaebi et al., 2017). This approach helps decision-makers take advantage of running the model under different settings and configurations and evaluate their impact to obtain the most desirable settings. The following section describes the implementation aspects of the proposed framework and how the simulation module is integrated with the optimization module to perform SO through specialized software. 3.4 Framework implementation To implement the proposed framework, three software packages including MS Excel, MATLAB, and Simio are linked together to optimize a model’s performance with multiple criteria. The holistic view of this framework structure and its components are shown in Figure 2. In the first stage of the model, AHP analysis is performed to calculate the criteria weights. This calculation can be performed relatively quickly using Microsoft Excel. Then, the attained weights are transferred to the second stage to execute the SO operation. This stage includes optimization (GA) and simulation (DES) modules which are iteratively linked together to optimize the model. Due to its computational power and wide set of functions, MATLAB is used as the main platform to deploy GA. MATLAB has a large community of committed users who are Figure 2 Holistic view of the proposed two-stage AHP-multi-objective simulation optimization framework IJAHP Article: Dehghanimohammadabadi, Kabadayi/A two-stage AHP multi-objective simulation optimization approach in healthcare International Journal of the Analytic Hierarchy Process 123 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 developing and sharing algorithms (Belevich et al., 2016; Ozgur et al., 2017), which also increases the chance of this generic model being used by other users. More importantly, the interaction between MATLAB and the applied simulation model is seamless due to the advancement conducted by Dehghanimohammadabadi and Keyser (Dehghanimohammadabadi & Keyser, 2017). Finally, the DES package that is being used in this study is Simio, which is developed in C# (Vieira et al., 2016) and enables a user to perform customized operations. The integration of Simio and MATLAB proposed by Dehghanimohammadabdi and Keyser (2015) makes the development of the decision support system feasible and easy-to- implement. The pseudocode presented in Figure 3, clearly shows the important steps of the stages and their interactions. After performing AHP analysis in the first stage, the SO model is initiated. In this stage, in every iteration of GA, a number of solutions are generated through cross-over and mutation operators. Each solution provides a new configuration of the simulation model, which basically determines a specific value for all of the simulation model controls. By generating each solution in GA, the simulation model is triggered to obtain the expected performance of the given solution and its fitness. In other words, the simulation model acts as a cost function (evaluation function) of the GA, in which solutions will be tested. After stopping criteria are met, the algorithm ends its run and releases the best-obtained solution up to that point. Figure 3 Pseudocode of the proposed two-stage AHP-multi-objective simulation optimization framework IJAHP Article: Dehghanimohammadabadi, Kabadayi/A two-stage AHP multi-objective simulation optimization approach in healthcare International Journal of the Analytic Hierarchy Process 124 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 4. Case study: using the AHP-SO model in a healthcare setting A healthcare case study is discussed in this section to demonstrate the applicability of the proposed model. It needs to be noted that the main goal of this section is to discuss implementation aspects of the work, and its general capacity to be replicated by researchers to solve simulation-optimization problems with multiple objectives. Therefore, a typical example from the Simio library is used to validate the model’s efficiency and illustrate its great potential and prospective applications. The selected simulation model is “HospitalEmergencyDepartment.spfx” which represents a small emergency department. This simulation model is available to all Simio users in the “\Documents\Simio\Examples” directory and is accompanied by a very detailed documentation. Interested readers are referred to the model details from the provided documentation by Simio in the same directory “HospitalEmergencyDepartment.pdf”. This ED model includes a waiting area, a registration desk, a triage room, a radiology station, a billing area, 6 beds and 6 rooms that are used for patients that are admitted into the hospital. 4.1 AHP analysis In this model, the AHP algorithm is used to determine the weights (importance) of the objectives that are applied in the multi-objective simulation model. The input of the AHP algorithm is the pairwise comparison results provided by the members of the decision- making team including the financial director of the hospital, physicians and nurses who work in the ED. MS Excel is used to deploy AHP because of its popularity, ubiquity, and more importantly, its compatibility with MATLAB. After compiling judgments, the obtained weights for all of the criteria are transferred to MATLAB to execute the simulation-optimization process. The aim of this model is to determine the best combination of controls, number of nurses, number of doctors, and number of registration desk staff to improve the ED performance measures. Multiple criteria are considered to evaluate the performance of the system. A hierarchical structure is created to provide decision makers with a visual presentation to assist with creating the pairwise comparisons. As illustrated in Figure 4, the hierarchy Figure 4 AHP structure and criteria of ED IJAHP Article: Dehghanimohammadabadi, Kabadayi/A two-stage AHP multi-objective simulation optimization approach in healthcare International Journal of the Analytic Hierarchy Process 125 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 of this model consists of three levels: the first level comprises the main goal, the second level comprises the performance measures, and the last level includes three sub-criteria for each of these criterion. The main goal of the problem is defined as the best configuration of the emergency department and performance measurement criteria are determined as patient satisfaction, resource utilization, and system performance. Each of these criteria consists of three sub-criteria, which are described as follows.  Average Length of Stay: the average time between a patients admission and discharge  Average Wait Time for a Bed: the average amount of time that a patient waits for a bed to become available  Average Wait Time for a Room\Physician: the average time a patient waits for a room to become available and be visited by the physician  Physician Utilization: the percentage of time the physician spends with a patient  Nurse Utilization: the percentage of time a nurse spends with a patient and provides care  Room/Bed Occupancy Rate: the percentage of a time a room/bed is occupied  Cost of Staff: the total cost of healthcare providers based on Usage Cost Rate and Idle Cost Rate  Number of Diversions: the number of times that a new patient fails to enter the ED due to the lack of an available room  Total Leave Without Being Seen (LWBS): the number of patients who arrive in the main entrance but leave because of either a long waiting time or a full waiting area To understand the importance of these factors and their influence on the model, experts’ judgments are collected using the AHP to obtain the weights of each criterion and sub- criteria. The responses of five experts are used to perform the pairwise comparison and get local weights of the main criteria and the sub-criteria. The calculated CR value for all of the analysis is less than 0.1, which guarantees the quality of the results. Interested readers can find the experts’ responses and the details of the AHP analysis at this link. Table 1 lists the AHP results and global weights of the ER sub-criteria. These weights are then used to form the weighted sum multiple-objective function in the SO section. https://www.dropbox.com/s/tfa41lh551vnwmv/AHPFinalResults.xlsx?dl=0 IJAHP Article: Dehghanimohammadabadi, Kabadayi/A two-stage AHP multi-objective simulation optimization approach in healthcare International Journal of the Analytic Hierarchy Process 126 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 Table 1 AHP results for global weights of the ER criteria and sub-criteria Main Criteria Local Weights Sub-criteria Local Weights Global Weights Patient Satisfaction 0.274 Avg. length of stay 0.111 0.031 Avg. Wait time for a bed 0.475 0.130 Avg. Wait time to see doctor 0.414 0.114 Resource Utilization 0.155 Physician utilization 0.582 0.091 Nurse utilization 0.253 0.039 Room/bed occupation rate 0.165 0.026 System Preference 0.570 Cost of staff 0.305 0.174 Number of diversions 0.108 0.062 LWBS 0.587 0.335 Sum 1.000 - - 1.000 As can be seen from Table 1, the most important criterion related to the quality of the hospital ED is determined as LWBS with 33.5 %. The rate of patients who leave without being seen (LWBS) by a physician in EDs has critical importance in terms of the safety of human life and the quality of care in hospitals. Therefore, it is usually a major concern for healthcare providers and used as an ED performance evaluation metric in hospitals. According to the results, cost of staff is also another important criterion that determines the performance of ED. Besides, average waiting time for a bed and average waiting time to see a doctor are also significantly important metrics for ED performance due to their negative impacts on the patients’ satisfaction and health. 4.2 Simulation-optimization model The calculated weights from AHP are applied to formulate the multi-objective simulation-optimization problem. As shown in Equation (3), a weighted sum formula transforms all of the objectives (𝑓𝑖 ) into a single-objective function (𝑧) . All of the objective functions are weighted using global weights obtained from the AHP analysis (𝑤𝑖 ). Since different objectives have different units and magnitudes, all of the objective functions are divided by their max value (𝑓𝑖 𝑚𝑎𝑥 ) to be scaled between [0,1]. 𝑧 = ∑ 𝑤𝑖 𝑓𝑖 𝑚𝑎𝑥 𝑓𝑖 𝑚 𝑖=1 (3) The simheuristic model used the weighted sum objective function to perform the best combination of controls. The defined range for the model inputs (number of nurses, number of doctors, and number of registration desk staff) is defined between min=2 and max=10. Therefore, in each iteration, GA provides a new combination of inputs, and after a certain number of iterations, returns the best-ever-found results. For this case study, a max-iteration of GA is set to 20, with a population size of 10, and a Simio replication size of 100. IJAHP Article: Dehghanimohammadabadi, Kabadayi/A two-stage AHP multi-objective simulation optimization approach in healthcare International Journal of the Analytic Hierarchy Process 127 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 4.3 Results The proposed hybrid model is developed using MATLAB R2018a, and Simio 11. After 20 iterations and generating 230 solutions, GA determined the best model configuration. The solution results of the SO model suggest five (5) nurses, (4) doctors, and (2) registration desk staff for the ED model (Table 2). The simulation helps to run the model with multiple replications and estimate the expected performance level for each of the criteria. Respective descriptive statistics of 100 simulation replications are tabulated in Table 3 for all of the criteria. These results show physician utilization has the highest variability (half-width = 0.595) while cost of staff experienced less change through the replications (half-width = 0.002). The AHP analysis determined LWBS, cost of staff, and wait time to see doctor as the top three measures for the ED model. Therefore, to analyze and better understand these factors, a graphical representation of results is provided in Figure 5. As shown in Figure 5-(a), there are only 12 cases where the system observes LWBS (12%), and on the positive cases, the values are very minimal (LWBS < 3). This aligned with the AHP results where LWBS attained the highest weight which emphasizes its tendency to reduce the number of unvisited patients. The other two box-plots in Figure 5-(b, c), provide insights regarding cost of staff, and wait time to see the doctor. These results indicate that, under the optimal configuration of the model, it is most likely that a patient is visited by a doctor within 0.65 hours of his/her arrival. Table 2 Final model configuration (best GA solution) Solution Values Number of nurses, 5 Number of doctors 4 Number of registration desk staff 2 Table 3 Performance of the ED model based on the best GA solution Main Criteria Sub-criteria (Responses) Mean Min Max Half- width Patient Satisfaction Avg. length of stay 42.276 33.325 58.360 1.171 Avg. wait time for a bed 14.711 12.787 17.559 0.176 Avg. wait time to see doctor 0.580 0.534 0.697 0.005 Resource Utilization Physician utilization 57.975 50.527 65.445 0.595 Nurse utilization 47.987 41.864 54.171 0.489 Room/bed occupation rate 33.148 26.506 39.779 0.522 System Preference Cost of staff 44.166 44.138 44.196 0.002 Number of diversions 0.250 0.000 3.000 0.119 LWBS 0.163 0.000 2.823 0.104 IJAHP Article: Dehghanimohammadabadi, Kabadayi/A two-stage AHP multi-objective simulation optimization approach in healthcare International Journal of the Analytic Hierarchy Process 128 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 4.4 Sensitivity analysis To evaluate the stability of the priority ranking, a sensitivity analysis is performed by varying the main criteria weights of AHP. This shows how altering the ranking factors could affect the final results and the selection process. In this section, three scenarios are studied. As indicated in Equation 4, the weight (𝑊𝑖) of each of the main criteria (𝐹𝑖) is thought to change (one at a time), while the summation of new weights (𝑊𝑖 ′) adds up to 1 (Equation 5). Three control parameters namely 𝛼 , 𝛽 , and 𝛾 are defined to quantify changes to patient satisfaction, resource utilization, and system preference weights, respectively. 𝑧 = (1 + 𝛼)𝑊1𝐹1 + (1 + 𝛽)𝑊2𝐹2 + (1 + 𝛾)𝑊3𝐹3 (4) ∑ 𝑊𝑖 ′ = 1 3 𝑖 (5) As tabulated in Table 4, in each scenario, one of the main criteria is changed in three levels (100%, 200%, and 300%) and its impact is evaluated on the ED model configuration. Therefore, in each setting, a new set of weights is obtained for the main criteria and the sub-criteria to consequently form a new objective function. Then, the SO model is deployed based on the new objective function to find the best configuration of the simulated model. This analysis helps to figure out which of the main criteria is the most critical with the highest impact on the ED system. Figure 6 illustrates a heatmap of new weights in each scenario and provides an overview perspective of how the importance of criteria changes from one scenario to another. Details of scenarios and their corresponding results are discussed as follows: Figure 5 Simulation results of top three criteria based on the best GA solution IJAHP Article: Dehghanimohammadabadi, Kabadayi/A two-stage AHP multi-objective simulation optimization approach in healthcare International Journal of the Analytic Hierarchy Process 129 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 Figure 6 Heatmap of new weights in each scenario * Weights with higher values are highlighted with a darker color α=100% α=200% α=300% β=100% β=200% β=300% γ=100% γ=200% γ=300% Avg. length of stay 0.030 0.048 0.059 0.067 0.026 0.023 0.021 0.019 0.014 0.011 Avg. wait time for a bed 0.130 0.204 0.252 0.286 0.113 0.099 0.089 0.083 0.061 0.048 Avg. wait time to see doctor 0.113 0.178 0.220 0.249 0.098 0.087 0.077 0.072 0.053 0.042 Physician utilization 0.090 0.071 0.058 0.050 0.156 0.207 0.246 0.057 0.042 0.033 Nurse utilization 0.039 0.031 0.025 0.022 0.068 0.090 0.107 0.025 0.018 0.014 Room/bed occupation rate 0.026 0.020 0.017 0.014 0.044 0.059 0.070 0.016 0.012 0.009 Cost of staff 0.174 0.137 0.113 0.096 0.151 0.133 0.119 0.222 0.244 0.257 Number of diversions 0.062 0.048 0.040 0.034 0.053 0.047 0.042 0.079 0.086 0.091 LWBS 0.335 0.263 0.217 0.184 0.290 0.256 0.229 0.427 0.469 0.494 Sum 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 Sce nario 1 Sce nario 2 Sce nario 3 BaseSub-crite ra Table 4 Experimental setting for the AHP sensitivity analysis Main Criteria Scenarios 𝜶 𝜷 𝜸 Patient Satisfaction 1 100%, 200%, 300% 0% 0% Resource Utilization 2 0% 100%, 200%, 300% 0% System Preference 3 0% 0% 100%, 200%, 300% 4.4.1. Scenario 1: Patient satisfaction sensitivity analysis In this scenario, the first main decision factor, patient satisfaction, is subject to change. The importance of patient satisfaction is increased in three steps with three levels, 𝛼=100%, 200%, and 300%. As is evident in Table 5, by increasing change levels, the weight of patient satisfaction is increased and this increment is compromised by the decreasing weight of the other two criteria. Table 5 Scenario 1 setting - Patient satisfaction sensitivity analysis Main Criteria Base Model 𝛼 = 0% Level 1 𝛼 = 100% Level 2 𝛼 = 200% Level 3 𝛼 = 300% Patient Satisfaction 27.40% 29.34% 31.17% 32.91% Resource Utilization 15.50% 15.09% 14.69% 14.32% System Preference 57.10% 55.58% 54.13% 52.76% Sum 100.00% 100.00% 100.00% 100.00% Under each setting, the SO model is executed to understand how the model setting changes and how sensitive it is to the patient satisfaction measures. As depicted in Figure 7, by putting more weight on patient satisfaction and its sub-criteria (average length of stay, average wait time for a bed, and average wait time to see doctor), number of nurses and number of doctors in the model remain unchanged. The only decision variables that are liable to change is number of registration desk staff which changes from 2 to 5 in order to improve the quality of care considering more weight for the patient satisfaction. IJAHP Article: Dehghanimohammadabadi, Kabadayi/A two-stage AHP multi-objective simulation optimization approach in healthcare International Journal of the Analytic Hierarchy Process 130 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 This seems rational since the cost of registration desk staff is relatively lower than doctors and nurses, and could increase the efficiency of the ED process. Insight 1: Increasing the importance of the patient satisfaction criteria increases the number of registration desk staff while the other two controls remain the same. 4.4.2. Scenario 2: Resource utilization sensitivity analysis The second factor that is changed to analyze its effect is resource utilization. This main criteria includes three sub-criteria, namely physician utilization, nurse utilization, and room/bed occupation rate. Similar to the first scenario, this factor is changed in three levels (𝛽=100%, 200%, and 300%). Table 6 shows the corresponding weights for these levels and the results are shown in Figure 8. Since this scenario aims to increase resource utilization, the number of doctors is reduced in all levels by 1 (from 4 to 3) to make doctors busier and make their utilization larger. Interestingly, in contrast to number of doctors, the number of registration desk staff is increased from 2 to 6. This huge change is mainly due to the fact that utilization of these staff is not taken into account. The only utilization measures included in this study is for nurses and doctors, and therefore, the reduction in doctors staffing in the model is compensated for by increasing the registration desk staff. Insight 2: Physician utilization is one of the sub-criteria of the resource utilization factor. Therefore, by increasing 𝛽, the number of doctors is reduced (from 4 to 3) to improve all physicians utilization. Table 6 Scenario 2 setting - Resource utilization sensitivity analysis Main Criteria Base Model 𝛽 = 0% Level 1 𝛽 = 100% Level 2 𝛽 = 200% Level 3 𝛽 = 300% Patient Satisfaction 27.40% 26.98% 26.58% 26.18% Resource Utilization 15.50% 16.79% 18.04% 19.25% System Preference 57.10% 56.23% 55.38% 54.56% Sum 100.00% 100.00% 100.00% 100.00% Figure 7 Scenario 1 sensitivity analysis results IJAHP Article: Dehghanimohammadabadi, Kabadayi/A two-stage AHP multi-objective simulation optimization approach in healthcare International Journal of the Analytic Hierarchy Process 131 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 4.4.3. Scenario 3: System preference sensitivity analysis Similar to the first two scenarios, the third main factor of the AHP analysis is studied under three levels (Table 7) and the results are depicted in Figure 9. These results suggest keeping the number of doctors and nurses untouched in all three levels of 𝛾. The only observed change is for number of registration desk staff, which slightly increases as 𝛾 increases. This implies that, even though increasing number of staff implies more cost to the system, it facilitates lowering the number of diversions, and LWBS. Insight 3: The only input variable that is subject to 𝛾 changes is the number of registration desk staff. Table 7 Scenario 3 setting - System preference sensitivity analysis Main Criteria Base Model 𝛾 = 0% Level 1 𝛾 = 100% Level 2 𝛾 = 200% Level 3 𝛾 = 300% Patient Satisfaction 27.40% 25.92% 24.59% 23.39% Resource Utilization 15.50% 14.66% 13.91% 13.23% System Preference 57.10% 59.42% 61.50% 63.37% Sum 100.00% 100.00% 100.00% 100.00% Figure 8 Scenario 2 sensitivity analysis results IJAHP Article: Dehghanimohammadabadi, Kabadayi/A two-stage AHP multi-objective simulation optimization approach in healthcare International Journal of the Analytic Hierarchy Process 132 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 5. Conclusion and future works This work proposes an integrated AHP-SO based model to solve multi-objective stochastic problems. By integrating AHP with a SO model, this framework provides a realistic weight for the objective functions and makes the SO model structure more reliable. This framework proceeds in two stages. In the first stage, AHP is used to prioritize the given system performance measures and evaluate their weights. Then, in the second stage, the provided weights from the AHP are used to transform all of the objective functions into a single-objective function in the SO model. At this stage, the SO model runs the simulated model to determine the best configuration of the system using the provided single-objective function. To show the applicability of the proposed framework, this model is applied to configure an emergency department staff setting. In this case study, the SO model determines the best staffing level for nurses, doctors, and registration desk staff by considering multiple objective functions including patient satisfaction, resource utilization, and system preference. Finally, a sensitivity analysis is applied to investigate the influences of performance criteria on the ED model configuration. This paper encourages the advancement of MCDM models such as AHP, within the simulation optimization environment. Therefore, researchers and practitioners in both fields can benefit from this model and deploy it to other task domains. It is a simple structure that makes it easy to implement and generic enough to be applied in many disciplines. However, this promising approach is in its initial stage and can be extended in many directions. From the MCDM point of view, there are plenty of MCDM approaches that can be used to facilitate the SO model construction. A study can extensively combine a variety of MCDM methods with a SO model and evaluate their performance under different circumstances. Considering a fuzzy MCDM approach with SO is also desirable since both approaches address uncertainty in the model. Figure 9 Scenario 3 sensitivity analysis results IJAHP Article: Dehghanimohammadabadi, Kabadayi/A two-stage AHP multi-objective simulation optimization approach in healthcare International Journal of the Analytic Hierarchy Process 133 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 REFERENCES Al-Dhaheri, N., Jebali, A. and Diabat, A. (2016). A simulation-based Genetic Algorithm approach for the quay crane scheduling under uncertainty. Simulation Modelling Practice and Theory, 66, 122–138. doi: https://doi.org/10.1016/j.simpat.2016.01.009 Alshaebi, A., Dauod, H., Weiss, J. and Yoon, S.W. (2017). Evaluation of different forklift battery systems using statistical analysis and discrete event simulation. IIE Annual Conference. Proceedings, 1637–1642. Alt, J.K. and Lieberman, S. (2010). Representing dynamic social networks in discrete event social simulation. Proceedings of the 2010 Winter Simulation Conference, 1478–1489. doi: https://doi.org/10.1109/wsc.2010.5679046 Azadivar, F. and Wang, J. (2000). Facility layout optimization using simulation and genetic algorithms. International Journal of Production Research, 38(7), 4369–4383.doi: https://doi.org/10.1080/00207540050205154 Baccouche, A. ,Goren, S., Huyet, A.L. and Pierreval, H. (2011). An approach based on simulation optimization and AHP to support collaborative design: With an application to supply chains. 2011 IEEE Workshop On Computational Intelligence In Production And Logistics Systems (CIPLS), 1– 7. doi: https://doi.org/10.1109/cipls.2011.5953360 Baležentis, T. and Streimikiene, S. (2017). Multi-criteria ranking of energy generation scenarios with Monte Carlo simulation. Applied Energy, 185, 862–871. doi: https://doi.org/10.1109/cipls.2011.5953360 Bamakan, S.M.H. and Dehghanimohammadabadi, M. (2015). A weighted Monte Carlo simulation approach to risk assessment of information security management system. International Journal of Enterprise Information Systems (IJEIS), 11(4), 63–78. doi: https://doi.org/10.4018/ijeis.2015100103 Belevich, I., Joensuu, M., Kumar, D.,Vihinen, H. and Jokitalo, E. (2016). Microscopy image browser: a platform for segmentation and analysis of multidimensional datasets. PLoS Biology, 14(1), e1002340. doi: https://doi.org/10.1371/journal.pbio.1002340 Carson Y. and Maria, A. (1997). Simulation optimization: methods and applications. Proceedings of the 29th conference on Winter simulation, 118–126. Chambers, L.D. (2019). The practical handbook of genetic algorithms: New Frontiers, Volume II. Chen, C.F. Applying the analytical hierarchy process (AHP) approach to convention site selection. Journal of Travel Research, 45(2), 167–174. doi: https://doi.org/10.1177/0047287506291593 Dehghanimohammadabadi, M. (2016). Iterative optimization-based simulation (IOS) with predictable and unpredictable trigger events in simulated time, Western New England University. doi: https://doi.org/10.1109/wsc.2015.7408423 Dehghanimohammadabadi, M., Rezaeiahari, M. and Keyser, T.K. (2017). Simheuristic of patient scheduling using a table-experiment approach - Simio and MATLAB integration application. Proceedings of the 2017 Winter Simulation Conference. IJAHP Article: Dehghanimohammadabadi, Kabadayi/A two-stage AHP multi-objective simulation optimization approach in healthcare International Journal of the Analytic Hierarchy Process 134 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 Dehghanimohammadabadi, M. and Keyser, T.K. (2017). Intelligent simulation: Integration of Simio and MATLAB to deploy decision support systems to simulation environment. Simulation Modelling Practice and Theory, 71, 45–60. doi: https://doi.org/10.1109/wsc.2017.8248015 Dehghanimohammadabadi, M. and Keyser, T.K. (2015). Smart Simulation: Integration of Simio and Matlab | Simio. Proceedings of the 2015 Winter Simulation Conference. doi: https://doi.org/10.1109/wsc.2017.8248015 Eskandari, H., Riyahifard, M., Khosravi, S. and Geiger, C.D. (2011). Improving the emergency department performance using simulation and MCDM methods. Proceedings of the 2011 winter simulation conference (WSC), 1211–1222. doi: https://doi.org/10.1109/wsc.2011.6147843 Göçken, M., Dosdoğru, A.T., Boru, A. and Geyik, F. (2017). Characterizing continuous (s, S) policy with supplier selection using Simulation Optimization. SimulTIon, 0037549716687044. doi: https://doi.org/10.1177/0037549716687044 Göçken, M., Dosdoğru, A.T., Boru, A. and Geyik, F. (2015). (R,s,S) Inventory control policy and supplier selection in a two-echelon supply chain: An optimization via simulation approach. Proceedings of the Winter SImulation Conference, California, USA. doi: https://doi.org/10.1109/wsc.2015.7408320 Gul, M., Celik, E., Gumus, A.T. and Guneri, A.F. (2016). Emergency department performance evaluation by an integrated simulation and interval type-2 fuzzy MCDM-based scenario analysis. European Journal of Industrial Engineering, 10(2), 196–223. doi: https://doi.org/10.1504/ejie.2016.075846 Hsu, T.H. and Pan, F.F. (2009). Application of Monte Carlo AHP in ranking dental quality attributes. Expert Systems with Applications, 36(2), 2310–2316. doi: https://doi.org/10.1016/j.eswa.2007.12.023 Juan, A.A., Faulin, J., Grasman, S.E., Rabe, M. and Figueira, G. (2015). A review of simheuristics: Extending metaheuristics to deal with stochastic combinatorial optimization problems. Operations Research Perspectives, 2, 62–72. doi: https://doi.org/10.1016/j.orp.2015.03.001 Karczmarek, P., Pedrycz, W., Kiersztyn, A., and Rutka, P. (2017). A study in facial features saliency in face recognition: an analytic hierarchy process approach. Soft Computing, 21(24), 7503–7517. doi: https://doi.org/10.1007/s00500-016-2305-9 Kokol, P., Pohorec, S., Štiglic, G. and Podgorelec, V. (2012). Evolutionary design of decision trees for medical application. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 2(3), 237–254. doi: https://doi.org/10.1002/widm.1056 Man, K.F. Tang, K.S. and Kwong, S. (1999). Genetic algorithms: Concept and design. London: Springer Verlag. Marseguerra, M., Zio, E., and Podofillini, L. (2002). Condition-based maintenance optimization by means of genetic algorithms and Monte Carlo simulation. Reliability Engineering & System Safety, 77(2), 151–165. doi: https://doi.org/10.1016/s0951-8320(02)00043-1 IJAHP Article: Dehghanimohammadabadi, Kabadayi/A two-stage AHP multi-objective simulation optimization approach in healthcare International Journal of the Analytic Hierarchy Process 135 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 Meng, C. (2015). Simulation-based decision support for agricultural supply chain performance improvement. The University of Arizona. Moon, Y.B. and Phatak, D. (2005). Enhancing ERP system’s functionality with discrete event simulation. Industrial Management & Data Systems, 105(9), 1206–1224. doi: https://doi.org/10.1108/02635570510633266 Ozgur, C., Colliau, T., Rogers, G., Hughes, Z. and Myer-Tyson, B. (2017). MatLab vs. Python vs. R. Journal of Data Science, 15(3), 355–372. Pun, K.P., Tsang, Y.P., Choy, K.L., Tang, V. and Lam, H.Y. (2017). A fuzzy-AHP-based decision support system for maintenance strategy selection in facility management. 2017 Portland International Conference on Management of Engineering and Technology (PICMET), 1–7. doi: https://doi.org/10.23919/picmet.2017.8125300 Saaty, T.L. (1980) The Analytic Hierarchy Processs. New York: McGraw-Hill. Talbi, E.G. (2009). Metaheuristics: from design to implementation. John Wiley & Sons. Vieira, A.A.C., Dias, L.S., Pereira, G., Oliveira, J.A., Carvalho, M.S. and Martins (2016). Automatic simulation models generation of warehouses with milk runs and pickers. 28th European Modeling and Simulation Symposium, EMSS 2016, 231–241. Yin, P.Y., Wu, T.H. and Hsu, P.Y. (2017). Risk management of wind farm micro-siting using an enhanced genetic algorithm with simulation optimization.” Renewable Energy, 107, 508–521. doi: https://doi.org/10.1016/j.renene.2017.02.036 Author biographies Mohammad Dehghanimohammadabadi is Assistant Teaching Professor of the Mechanical and Industrial Engineering Department at Northeastern University (Boston, USA). He received his Ph.D. in Engineering Management from Western New England University (USA) in 2016. His research is mainly focused on developing and generalizing reinfrocement learning (RL) and simulation-optimization frameworks in different disciplines such as healthcare, supply chain, and manufacturing. His website address is http://www.mie.neu.edu/people/dehghani-mohammad and his e-mail address is m.dehghani@northeastern.edu. Nihan Kabadayi is Assistant Professor in the School of Business at Istanbul University (Istanbul, Turkey) where she has been a faculty member since 2005. She received her Ph.D. in Supply Chain Management from Istanbul University in 2013. Her research interest includes supply chain management, healthcare operations management, quality management, and multi-criteria decision making. Her e-mail address is nihank@istanbul.edu.tr. http://www.mie.neu.edu/people/dehghani-mohammad mailto:m.dehghani@northeastern.edu mailto:nihank@istanbul.edu.tr