IJAHP Article: Moons, Pintelon, Jorissen, De Ridder, Everaerts/Identification of multi- stakeholder value in medical decision-making International Journal of the Analytic Hierarchy Process 82 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 IDENTIFICATION OF MULTI-STAKEHOLDER VALUE IN MEDICAL DECISION-MAKING Karen Moons Ku Leuven Belgium karen.moons@kuleuven.be Liliane Pintelon liliane.pintelon@kuleuven.be Pieterjan Jorissen pieterjan.jorissen@student.kuleuven.be Dirk De Ridder dirk.deridder@uzleuven.be Wouter Everaerts wouter.everaerts@uzleuven.be ABSTRACT The recent trend is towards value-based healthcare, which is characterized by the relationship between quality of care and the associated costs. However, value in healthcare is a largely unmeasured and misunderstood concept. In particular, a definition of multi-stakeholder value in healthcare decision-making is missing. In this case study, we review the radical prostatectomy procedure because previous studies have shown discrepancies between what the patient’s value most and what the experts think the patient’s value most. The objective of this research is to address this gap by identifying a multi-stakeholder definition of value which will improve healthcare decision-making. Multi-criteria decision-making (MCDM) techniques, more specifically the Analytic Hierarchy Process (AHP), are applied to prioritize the quality of care indicators according to the following six pillars of quality: safety, effectiveness, efficiency, timeliness, patient- centeredness and equity. In addition, the average cost of a radical prostatectomy is estimated. A distinction is made between three treatment stages in order to calculate the overall quality score by assigning weights to criteria in each treatment stage. According to the AHP weights, the pillars of effective, patient-centered and equitable care contribute the most to value creation. Finally, some of the challenges of MCDM studies are addressed, such as conflicting preferences between stakeholder groups. In conclusion, this case study stresses the need to adapt operational research methods and knowledge to be applied in value-based healthcare. The AHP is a suitable approach to address the needs of healthcare decision-makers, to set priorities, and to identify value improvement opportunities while considering all of the stakeholders involved in the full care cycle. Keywords: value-based healthcare; Analytic Hierarchy Process; multi-stakeholder decision-making mailto:karen.moons@kuleuven.be mailto:liliane.pintelon@kuleuven.be mailto:pieterjan.jorissen@student.kuleuven.be mailto:dirk.deridder@uzleuven.be mailto:wouter.everaerts@uzleuven.be IJAHP Article: Moons, Pintelon, Jorissen, De Ridder, Everaerts/Identification of multi- stakeholder value in medical decision-making International Journal of the Analytic Hierarchy Process 83 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 1. Introduction Since the 1990s, the healthcare sector has changed rapidly due to the need to deliver efficient and effective healthcare services because of increased competition and the growing influence of patients. Increasing pressure to decrease costs and improve the quality of patient care is driving healthcare organizations towards adopting more efficient healthcare delivery processes. However, measuring the performance of these processes is complicated due to the complex nature of healthcare organizations and the involvement of multiple stakeholders. Performance management in both the industrial and service sectors provides a competitive advantage as it enables organizations to control their supply chain strategy, implement continuous improvement programs and improve decision-making by focusing on achieving strategic, tactical or operational goals in the most efficient way (Maestrini et al., 2017). In a healthcare context, however, slow progress in performance improvement is a result of the existence of many performance definitions among the various stakeholders. Each stakeholder has their own, possibly conflicting goal for efficient performance management, including high quality, cost containment, safety, access to services, patient-centeredness or satisfaction. According to Porter (2010), creating high value for patients should be the main goal of any healthcare organization. Value improvement reflects the shared goal that unites the interests and activities of patients, payers, care providers and suppliers. Value is defined as health outcomes achieved per dollar spent, and therefore value improvement also addresses cost containment in hospitals (Mukherjee, 2008). Value-based healthcare with the goal of achieving high-quality patient care at the lowest possible cost is a hot topic in today’s research. However, until now care decisions have often been based only on the hospital’s perspective, and therefore ignore the patient’s preferences (Forbes, Hebb & Mu 2018). This work considers the medical services that are provided to patients who are diagnosed with prostate cancer. Prostate cancer is the most common form of cancer observed among men (Richman et al., 2005; Roth, Weinberger & Nelson, 2008). This study focused on one treatment option, namely surgery that is also referred to as (robot-assisted) radical prostatectomy. A multi-stakeholder definition of value is missing for this procedure. Value measurement should encompass all of the provided services that aim to meet the desired patient outcomes. In the case of a radical prostatectomy, this involves three stages of the treatment including the pre-operative, per-operative and post-operative stage. Value creation in the post-operative stage mainly refers to longer-term costs and patient outcomes, such as the need for ongoing interventions or sustainable recovery (Institute of Medicine (IOM), 2006). Current measures in healthcare organizations cover outcomes for single departments or a whole hospital rather than patient outcomes for the full care cycle. Value measurement over the full care cycle is needed in order to realign reimbursement practices with value so that payments cover the full care cycle as a reward for value creation. In Belgium, a reform of the hospital financing system aims to allocate resources to hospitals according to the perceived quality of care (Stephani et al., 2018). However, the definition of quality of care is unclear and every hospital claims that they have the best quality of care for specific medical disciplines. This work stresses the need to rigorously measure performance using operational research tools in value-based prostate cancer care by clearly defining quality of care and determining the associated costs, while considering all of the stakeholders involved in the full radical prostatectomy care cycle. IJAHP Article: Moons, Pintelon, Jorissen, De Ridder, Everaerts/Identification of multi- stakeholder value in medical decision-making International Journal of the Analytic Hierarchy Process 84 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 High quality of care is defined collectively by multiple outcomes. The Institute of Medicine (IOM) (2001) defines six dimensions of quality of care, namely safe, effective, patient-centered, timely, efficient and equitable care. However, there may be multiple competing patient outcomes (e.g. near-term safety versus long-term functionality). Relevant quality indicators are identified to measure value creation by relating quality of care to the associated costs. The objective of this work is to map the trade-offs between the patient outcomes, set priorities and construct a ranking of the quality criteria according to their relative importance, depending on each stakeholder’s perspective. This is a problem including multiple criteria which necessitates the use of MCDM techniques. In this study, different MCDM methods are described and their healthcare applications are discussed. The AHP is selected as the most suitable for solving this ranking or prioritization problem. This research mainly contributes to the literature by incorporating multiple stakeholders’ views to determine value in healthcare, whereas other studies only focus on the views of surgeons or patients (De Bekker-Grob et al., 2013; MacLennan et al., 2015; Marsh et al., 2014a; Martin et al., 2015). The stakeholders included in this study are patients, urologists, nursing staff, hospital management, general practitioners and payers. In addition, the authors propose solutions to address typical challenges in MCDM studies, such as inconsistencies, knowledge gaps, conflicts between stakeholders or uncertainty. Finally, the ranking of the criteria is combined with the associated costs of radical prostatectomy resulting in a value ratio that provides a guideline to identify opportunities for value improvement. 2. Literature review This research investigates the application of MCDM techniques to identify a multi- stakeholder value definition by addressing the needs of healthcare decision-makers. Value-based healthcare is a hot topic in the literature and will be discussed below. A brief overview is presented of different prostate cancer treatments. In addition, the literature is reviewed to determine the most suitable MCDM method and typical MCDM applications in healthcare are given. 2.1 Value-based healthcare A shift in focus from volume to value is needed in the healthcare sector and is described in Porter and Lee (2016) as the movement from ‘fee-for-service’ to ‘value-based’ payment models. In the former model, hospitals and physicians are paid based on the number of delivered services, while payments in the latter model are based on patient health outcomes. All of the actors involved in healthcare delivery benefit from a value- based model because of lower costs, higher patient satisfaction, reduced risks, reduced spending, etc. The transformation to value-based healthcare requires restructuring how healthcare delivery is organized, measured and reimbursed (Porter & Lee, 2013). Porter (2010) advocates adopting value as the most important objective in healthcare performance management in order to obtain efficient healthcare systems while maximizing patient care quality. Value in value-based healthcare is defined as “the health outcomes achieved that matter to patients relative to the cost of achieving those outcomes” (Porter & Lee, 2013). However, value in healthcare is hard to measure and interpret which makes it challenging to define value in a healthcare setting. Therefore, the definition of value requires clarification. The first component in this definition refers to health outcomes. Health outcomes are condition-specific and multi-dimensional, such that no single outcome captures the results of care (Porter, 2010). Furthermore, outcomes IJAHP Article: Moons, Pintelon, Jorissen, De Ridder, Everaerts/Identification of multi- stakeholder value in medical decision-making International Journal of the Analytic Hierarchy Process 85 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 should include both patients’ near-term and longer-term health circumstances in order to encompass the ultimate results of care and sufficient measurement of risk factors. Porter (2010) proposes a hierarchy for ordering patient outcomes as follows: level 1 = health status achieved or retained, level 2 = recovery process, and level 3 = sustainability of health. Costs make up the second component in the value definition, and refer to the total costs of the full care cycle rather than individual services (Porter, 2010). Porter (2010) introduces ‘integrated practice units’ which cover the full care cycle for patients with similar symptoms or medical conditions in order to avoid healthcare providers tendency to focus on individual services for treatment. A value-based healthcare delivery system helps hospital managers measure both outcomes and costs. Although monitoring costs allows for improving efficiency in healthcare, focusing only on cost reduction can possibly lead to low-quality effective care. Therefore, outcomes must be considered relative to costs. Lee et al. (2016) present a value-driven approach and show that achieving better quality at a lower cost is possible. First, treatment costs are identified based on actual patient use or time spent in each treatment of the full care cycle. Next, an opportunity index is calculated by multiplying the coefficient of cost variation with the total direct cost. Highly variable and high cost factors offer the greatest opportunities for value improvement. In addition, thresholds are determined for quality or outcome indicators to compute a perfect care index. The objective is to strive for a perfect care index of 100%, which means that the threshold is met for every aspect of the treatment. The final step combines the perfect care indicators with cost variability into a quality over cost ratio to improve care by reducing costs and/or improving quality. Lee et al. (2016) showed an 11% reduction in costs with improved health outcomes for total joint replacement. From a cost perspective, a justified estimation of the average cost of the treatment (radical prostatectomy in this case study) is required to determine value. This is challenging due to a lack of accurate cost information or information on how costs relate to the outcomes achieved (Porter & Lee, 2013). Time-driven activity-based costing is a suitable technique for understanding different types of costs, and allows for substantial cost reductions without negatively affecting outcomes (Kaplan & Porter, 2011). The literature discusses direct and indirect costs that should be taken into account when determining a base-case average cost. Typical cost drivers include operating room cost, medical supply cost, surgeon professional fees, hospital room cost, pharmaceuticals and medication cost, investment costs for medical equipment, laboratory cost, etc. (Hayes et al., 2013; Hyams et al., 2012; Lotan, Cadeddu & Gettman, 2004; Swartenbroekx et al., 2012). 2.2 Prostate cancer treatments Prostate cancer is the most common form of cancer observed among elderly men in Belgium, though it has a high survival rate of 97% (Belgian Cancer Registry, 2017). Typically, prostate cancer occurs in three phases, and the aggressiveness of the tumor behavior increases with each phase from a localized cancer to metastatic cancer. In this section, three treatment options are briefly presented for localized prostate cancer as follows: expectant management, surgery and radiation therapy (Litwin & Tan, 2017; Prostate Cancer Foundation, 2014): IJAHP Article: Moons, Pintelon, Jorissen, De Ridder, Everaerts/Identification of multi- stakeholder value in medical decision-making International Journal of the Analytic Hierarchy Process 86 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682  Expectant management is a treatment for patients with low-risk prostate cancer (e.g. prostate-specific antigen (PSA) is less than 10ng/mL) in order to monitor tumor behavior while not undergoing definitive therapy. Watchful waiting is suggested for palliative patients, whereas active surveillance intends to cure patients who develop significant disease through PSA testing or prostate biopsies. The major benefit is that patients do not perceive side effects with regard to their physical functioning.  Surgery or radical prostatectomy involves removing the prostate and seminal vesicles. Open radical prostatectomy is the traditional procedure characterized by an incision in the lower abdomen. This technology has been widely replaced with robotic radical prostatectomy, which requires smaller incisions for inserting the surgical robot’s arms. Although recovery time is observed to be lower for the latter type of surgery, the choice depends on the preference of the urologist. Typical side effects of surgery are urinary incontinence or erection problems.  Radiation therapy is used for killing or damaging the DNA of the localized or locally advanced prostate cancer cells with ionizing radiation or photons. Technological advancements introduce non-uniform radiation beams to reduce impact on the surrounding tissues in order to minimize side effects, such as bowel or hormonal problems. The patient chooses the preferred treatment with assistance from experts or evidence- based decision support tools. Research shows that patients often prefer surgery, even though experts recommend alternative prostate cancer treatments (Roth et al., 2008). In the remainder of this work, we focus on radical prostatectomy treatment. Patient preferences and value for prostate cancer treatments are studied in the literature (Loeb, 2016), but discrepancies remain between what patients value most and what experts think that patients value most. The increasingly central role of patients suggests an emerging role for shared medical decision-making in treatment selection (Litwin & Tan, 2017). 2.3 MCDM in healthcare The healthcare sector is characterized by complexity, uncertainty and conflicting interests between multiple stakeholders, which makes MCDM a suitable approach to evaluate different quality criteria while taking into account multiple stakeholders’ perspectives (Adunlin, Diaby & Xiao, 2015; Forbes et al., 2018; Lee, Donaldson & Cook, 2003). MCDM is defined as “any method that establishes criteria, weights them in terms of importance, and scores each alternative on each criterion to create an overall assessment of value” (Marsh et al., 2014a). This approach is especially useful for making consistent, rational and transparent decisions for complex problems with possibly conflicting perspectives (Thokala et al., 2016). MCDM techniques can be divided into three categories including value measurement, outranking and goal programming. Value measurement methods evaluate alternatives based on a single overall value, such that alternative A is preferred to alternative B whenever the value for A is higher. This value can be estimated by multiplying the performance score of the different alternatives with the weights assigned to each criterion (Marsh et al., 2014b). A second method uses an outranking approach for comparing alternatives. It relies on the dominance principle and does not require any numerical value to select the best alternative. An alternative is dominant when it is preferred for IJAHP Article: Moons, Pintelon, Jorissen, De Ridder, Everaerts/Identification of multi- stakeholder value in medical decision-making International Journal of the Analytic Hierarchy Process 87 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 important criteria, whereas dominated alternatives are eliminated (Thokala et al., 2016). Goal programming is the third approach used to measure alternatives by computing distance from the ideal and most negative results. However, not all MCDM methods are recommended for solving any decision problem. In this research, the goal is to prioritize the quality criteria. Therefore, the nature of the decision problem does not allow the application of goal programming methods since we are interested in identifying the most ideal result instead of using it as input. Precise values for criteria weights are essential and because of this the value measurement approach is preferred to the outranking method. Value measurement methods are the most applied MCDM methods in healthcare, and some typical examples are the AHP, Analytic Network Process (ANP), PAPRIKA and Multi-Attribute Utility (Value) Theory (MAU(V)T) (Marsh et al., 2014a). Typically, there are six main components in a value measurement approach that include the definition of the problem statement, criteria identification, performance measurement, alternatives scoring, criteria weighting and calculation of aggregate scores. Please see Zardari et al. (2015) and Marsh et al. (2017) for more information on the theoretical foundations and different weighting methods that support MCDM. The AHP is the most suitable method for this problem because it allows simplification of the complexity inherent in a hospital system by integrating qualitative and quantitative criteria, and it is able to deal with multiple conflicting stakeholder objectives. Moreover, AHP is a relative approach, using pairwise comparisons, and ensures transparent and systematic decision-making. The AHP simplifies the complex decision problem by forming a hierarchy, which consists of a goal, objectives and criteria (Saaty, 1990a, 2008). Stakeholders are asked to pairwise compare criteria on a 1-9 ratio scale. Based on the stakeholder judgments, local priority vectors are extracted to calculate criteria weights. Pairwise comparisons are perceived to be easier than comparing all of the criteria at once, and increase the involvement of the decision-maker when prioritizing quality indicators. In addition, a consistency ratio is included to ensure the quality of the judgments. Furthermore, AHP is a powerful tool for decision-making because of its flexibility and ability to capture both quantitative and qualitative criteria (Hariharan et al., 2004). For these reasons, AHP is used in the majority of cases to generate weights for different criteria in healthcare applications (Marsh et al., 2017). However, decision- makers should also pay attention to the limitations and challenges when performing an MCDM study. These challenges include rank reversal, interdependency, conflicting preferences, uncertainty, etc. MCDM has a few applications in healthcare. Schmidt et al. (2015) systematically reviewed AHP applications in five domains of healthcare research, namely the development of clinical guidelines, healthcare management, government policy, shared decision-making and biomedical innovation. Marsh et al. (2014a) also distinguished six types of MCDM applications in healthcare, ranging from hospital selection to prioritization of patients for access to health services. More specifically, in prostate cancer treatments, Richman et al. (2005) applied AHP as a decision-making tool to produce individual, rational and clinically appropriate decisions without physician bias. 3. Methodology The movement towards value-based healthcare pushes hospitals to restructure organization, measurement and reimbursement of care delivery processes. According to IJAHP Article: Moons, Pintelon, Jorissen, De Ridder, Everaerts/Identification of multi- stakeholder value in medical decision-making International Journal of the Analytic Hierarchy Process 88 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 Porter (2010), the overarching goal for any stakeholder must be to maximize value for patients by achieving the best outcomes at the lowest cost. Value improvement provides a competitive advantage to any healthcare organization and its stakeholders. However, stakeholders often have conflicting goals for measuring the quality of care. Moreover, today’s decision-making is typically based on the hospital’s perspective, and therefore ignores intangible variables such as the patient’s perceived value (Forbes et al., 2018). This research was conducted at the University Hospital UZ Leuven (Belgium) during the first half of 2018. The hospital specializes in several medical disciplines, but we focus on the urology department, and more specifically (robot-assisted) radical prostatectomy treatment. The decision problem was to identify a definition for value in this treatment, depending on multiple, possibly conflicting perspectives of the stakeholders. Value is defined as the ratio of quality over cost. First, relevant quality indicators were identified and prioritized using the AHP. The AHP is a transparent tool that supports decision- makers by visualizing quality criteria in a hierarchy structure and assigning weights to these criteria (Saaty & Vargas, 2006). The pairwise comparison process was conducted with all of the stakeholders that are involved in the treatment. In total, 33 patients, 5 urologists, 3 members of the nursing staff, 2 representatives of hospital management, 2 general practitioners and 1 representative of the health insurance sector participated in this case study. Based on the multiple stakeholder judgments, different criteria were assessed to determine what constitutes quality of care in this setting. In the second step, the key quality criteria were compared with the associated cost factors to determine the best opportunities for improving value. 3.1 AHP in medical decision-making In the first step of the AHP methodology (Saaty, 1990a), the decision problem was decomposed into a hierarchical structure. The overall goal is shown at the top of the hierarchy, namely identifying the multi-stakeholder definition of value in prostate cancer treatment. At the second level, quality and costs were determined to be the objectives to measure value. For each objective, several criteria were identified from the literature review and discussion with a number of the stakeholders. Figure 1 shows the hierarchy structure to determine value in prostate cancer treatment. An overview of the selected criteria, abbreviations and their definitions can be found in Appendix A. IJAHP Article: Moons, Pintelon, Jorissen, De Ridder, Everaerts/Identification of multi- stakeholder value in medical decision-making International Journal of the Analytic Hierarchy Process 89 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 Figure 1 AHP hierarchy structure The second step of the AHP uses pairwise comparisons to extract each stakeholder’s preferences in order to prioritize the quality criteria. A maximum of 13 quality-related criteria were included in the hierarchy in order to satisfy the time constraint for judging pairwise comparisons. Value measurement should encompass all of the provided services that aim to meet the desired patient outcomes. In the case of radical prostatectomy, this involves the three stages of the treatment that include pre-operative, per-operative and post-operative. The quality criteria are divided into three groups depending on the treatment stage. The pre-operative stage includes criteria that are relevant for measuring services before the surgery, such as PSA-level, Gleason score and clinical stage. In the second stage, the per-operative criteria measure the quality of the surgical procedure, and whether or not it is assisted by a robot. These criteria involve total time spent in the operating room, surgical margins and the necessity of blood transfusions. Finally, the post-operative stage refers to longer-term patient outcomes that are a result of the treatment, such as urinary incontinence, erectile dysfunction and time to return to normal functioning. This categorization of the criteria allows them to be compared within the same treatment stage, but ignores interdependencies between different treatment stages. According to experts, it is difficult to compare criteria at different stages because of the different nature of the criteria (e.g. operative factors versus operation outcomes). The AHP is a suitable method because it ignores these interdependencies. The weights for the quality criteria were determined using pairwise comparisons. Each stakeholder expressed his preference by comparing two criteria on a ratio scale from 1 to 9, with 1 representing equal importance and 9 extreme preference for one criterion. An example of a pairwise comparison between erectile dysfunction and urinary incontinence is shown in Figure 2. The obtained ratios were inserted into a pairwise comparison matrix for all of the criteria. From this matrix, the weights for the respective criteria were calculated via the principal eigenvector method (Saaty, 1990a). The resulting weights represent a measure for the relative importance of the quality criteria. IJAHP Article: Moons, Pintelon, Jorissen, De Ridder, Everaerts/Identification of multi- stakeholder value in medical decision-making International Journal of the Analytic Hierarchy Process 90 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 Figure 2 Pairwise comparison between erectile dysfunction and urinary incontinence Based on the weights assigned to each criterion in the separate treatment stages, the overall quality score can be calculated. The IOM (2001) defines six objectives of quality of care, namely safe, effective, patient-centered, timely, efficient and equitable care. The criteria were reallocated to these pillars of quality in order to derive the final weights for the quality aspects. The quality objectives as defined by the IOM (2001) are discussed in Table 1. In addition to the quality indicators, information on cost types was required to identify the definition of value for a radical prostatectomy. In this case study, the average cost of the different aspects of the treatment was estimated based on the approach suggested by the Belgian Health Care Knowledge Center (KCE) (Swartenbroekx et al., 2012). The cost components included the operating room, investment, surgeon wage, hospital stay, medication and overhead costs. Figure 1 shows the hierarchy with the quality criteria and cost components. Lee et al. (2016) proposed a value-driven method to identify value improvement opportunities. Table 1 Overview of six dimensions of quality of care according to IOM Quality objective Description Criteria Safe Avoiding injuries to patients from the care that is intended to help them Complications Blood transfusions Effective Providing services based on scientific knowledge to all who could benefit and refraining from providing services to those not likely to benefit (avoid overuse and underuse, respectively) Surgical margins Surgeon experience Additional therapy Patient- centered Providing care that is respectful of and responsive to individual patient preferences, needs and values, and ensuring that patient values guide all clinical decisions Urinary incontinence Erectile dysfunction Time to return to normal functioning Timely Reducing waiting time and sometimes harmful delays for both those who receive and those who provide care Waiting time is not included in this study Efficient Avoiding waste, including waste of equipment, supplies, ideas and energy Total time in operating room Length of stay Equitable Providing care that does not vary in quality because of personal characteristics such as gender, ethnicity, geographic location, and socioeconomic status PSA level Gleason score Clinical stage IJAHP Article: Moons, Pintelon, Jorissen, De Ridder, Everaerts/Identification of multi- stakeholder value in medical decision-making International Journal of the Analytic Hierarchy Process 91 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 4. Results This section describes the results of applying AHP to the value measurement prioritization problem in medical decision-making. Value is defined as the ratio of quality over cost. For every stakeholder group, AHP-based priority vectors were computed, which contained weights for the quality criteria in three treatment stages. Typical problems such as inconsistencies in the pairwise comparisons, conflicts between actors or knowledge gaps are addressed. The criteria weights were reassigned to the IOM-based hierarchy, and represented the six dimensions of quality of care. In addition to quality, cost components were analyzed in order to estimate the average cost of radical prostatectomy treatment. Finally, a multi-stakeholder definition of value was determined by combining the results from the quality and cost criteria into a value ratio, and opportunities for value improvement are identified. 4.1 Prioritization of quality of care criteria Value measurement requires a clear definition of quality of care. The AHP is used to determine the weights that represent the relative importance of several quality criteria while considering the possibly conflicting interests of different stakeholder groups. In the AHP, pairwise comparisons are used to elicit preferences for the included criteria. Due to the incomparable nature of the three treatment stages in radical prostatectomy, namely pre-operative, per-operative and post-operative, the criteria were split up and compared accordingly (see Figure 1). As an example, we discuss the prioritization of the quality criteria according to 11 pre-operative patients. The same procedure was applied to other stakeholder groups. A priority vector, containing the weights for each criterion, for each pre-operative patient was obtained based on the pairwise comparison matrix. Equation 1 presents the original pairwise comparison matrix PWO and priority vector wO for the criteria involved in the pre-operative treatment stage. More information on deriving weights according to the principal eigenvector method can be found in Saaty (1990). 𝑃𝑊𝑂 = ( 𝑃𝑆𝐴 𝐶𝑆 𝐺𝑆 ) ( 1 9 1 7 1 9 7 1 5 1 5 1)  𝑤𝑂 = ( 0.2344 0.0606 0.7050 ) (1) 𝐶𝑅𝑂 = 0.7276 The consistency of this matrix was checked by calculating the consistency ratio (CR). This CR should be lower than 10% to ensure qualitative judgments. In this example, the CR exceeded the threshold value and therefore further computations were required. The weighted consistency method by Jarek (2016) was applied to reduce the inconsistency. In this approach, the pairwise comparison matrix PWO is adapted by multiplying the priority vector wO with the reciprocal of its transpose (𝑤𝑂 ∗( 1 𝑤𝑂 ) 𝑇 ). This new matrix takes into account the original stakeholder judgments by taking the geometric mean of the original and adapted matrix. This process continues until the CR constraint is satisfied (i.e., CR is lower than 10%). Equation 2 contains the new pairwise comparison matrix PWN and the corresponding priority vector wN. IJAHP Article: Moons, Pintelon, Jorissen, De Ridder, Everaerts/Identification of multi- stakeholder value in medical decision-making International Journal of the Analytic Hierarchy Process 92 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 𝑃𝑊𝑁 = ( 𝑃𝑆𝐴 𝐶𝑆 𝐺𝑆 ) ( 1 5 1 4 1 5 4 1 8 1 8 1)  𝑤𝑁 = ( 0.2370 0.0643 0.6986 ) (2) 𝐶𝑅𝑁 = 0.0904 The resulting AHP-based priority vector, which contains weights for every criterion in the different treatment stages, was calculated for each stakeholder in each stakeholder group. Whenever the CR exceeded its limit, the weighted consistency method was applied. The next step was to determine one ranking for the stakeholder group by integrating the priority vectors of the 11 pre-operative patients. However, different experiences during the treatment or personal characteristics could cause conflicts within the same stakeholder group. An average would not satisfy any stakeholder. Song and Hu (2009) proposed using the cluster similarity approach to combine stakeholders’ conflicting views within one stakeholder group. The degree of similarity is calculated for all of the vector combinations by determining the distance between two vectors dij. This distance value ranges between 0 and 1, indicating the degree of similarity between the vectors. For example, pre-operative patient 3 and 5 have a similarity degree as seen in Equation 3. 𝑑35 = 𝑤3𝑤5 ‖𝑤3‖‖𝑤5‖ = ∑𝑤3𝑤5 √∑𝑤3 2 √∑𝑤5 2 = 0.7638 (3) The distance values for all of the vector combinations were combined into a similarity score matrix, which was used for deciding on the cluster groups. A threshold value of dij equal to or greater than 0.75 was used to decide if the two vectors belonged to the same cluster. This step was iterated for every row in the matrix and resulted in a final cluster grouping. Next, the cluster weights were determined based on the cluster size. A bigger cluster received a higher weight since it represented more stakeholders and would therefore have a higher impact on the result. The cluster weights are calculated as follows in Equation 4. 𝑤(𝐶1) = 𝑆1 𝑆1 2 +𝑆2 2 = 8 82 +32 = 0.1096 (4) 𝑤(𝐶2) = 0.0411 # This assumes that the pre-operative patients are divided into two clusters C1 and C2, with respective cluster sizes S1 = 8 and S2 = 3. Cluster C1 contains the most stakeholders and therefore has the greatest weight. Finally, one ranking of quality criteria was defined, which reflected the preferences of the stakeholder group. The weighted arithmetic mean (WAM) was used to multiply the cluster weights w(Ci) and the priority vectors wi in the respective cluster Ci, and sum for all of the clusters. The final ranking for pre-operative patients is shown in Table 2. IJAHP Article: Moons, Pintelon, Jorissen, De Ridder, Everaerts/Identification of multi- stakeholder value in medical decision-making International Journal of the Analytic Hierarchy Process 93 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 Table 2 Ranking of quality criteria according to 11 pre-operative patients PSA CS GS TOR SM SE BT 0.0943 0.0606 0.1251 0.0409 0.1267 0.2491 0.0799 LOS COMP UI ED TTRF AT 0.0185 0.0650 0.0612 0.0275 0.0274 0.0237 This priority vector was calculated based on the hierarchy shown in Figure 1, which divides the criteria into the pre-, per- and post-operative treatment stages. However, the main goal was to determine a definition for quality of care. Therefore, the criteria weights were reassigned to the corresponding pillars of quality according to the IOM (2001). Table 3 presents the resulting quality weights according to the pre-operative patients. Table 3 Weights for six quality dimensions: pre-operative patient group vs. all stakeholders Quality of care Weight (pre-operative patients) Weight (all stakeholders) Equitable 0.28 0.214 Timely Not available Not available Effective 0.3995 0.3968 Efficient 0.0595 0.0497 Safe 0.145 0.1337 Patient-centered 0.116 0.2144 For each stakeholder group, the quality criteria were prioritized as described above. The final goal was to combine all of the stakeholders’ viewpoints into one priority vector that contained the quality criteria weights. However, the obtained quality weights show conflicts between stakeholder groups. In addition, not every stakeholder group is equally important to the decision-making process. Therefore, we needed a weighting method to determine the weights of each stakeholder group in order to define the overall ranking of the quality criteria that represent all of the stakeholder groups. A similar approach was used as before when tackling conflicts within the stakeholder groups, namely the WAM based on cluster similarity that was used to compute the overall priority vector. The seven stakeholder groups were assigned to clusters based on their degree of similarity, as explained in Equation 3. Three clusters remained, and the cluster weights were computed according to Equation 4, and WAM was applied to derive the final weights. A reallocation of the weights was required to determine the quality according to the IOM definition of quality of care (see Table 3). From the table, we find that the pillars effectiveness, patient-centeredness and equity mostly contribute to high-quality care. Effective care is represented by the high weights attributed to surgical margins and surgeon experience, whereas equity and patient-centeredness are achieved by focusing on Gleason score and limiting post-operative consequences such as urinary incontinence or additional therapy, respectively. 4.2 Cost estimation in radical prostatectomy In this work, the average costs for the different aspects of radical prostatectomy treatments at the hospital under study were estimated based on the approach suggested by IJAHP Article: Moons, Pintelon, Jorissen, De Ridder, Everaerts/Identification of multi- stakeholder value in medical decision-making International Journal of the Analytic Hierarchy Process 94 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 the KCE (Swartenbroekx et al., 2012). The KCE manual provides a generic methodology that is applicable to all Belgian hospitals. At UZ Leuven, 170 radical prostatectomy procedures are performed each year. At our facility, the average length of stay (LOS) is 3.05 days and the average duration of the surgical procedure (TOR) is 2 - 3 hours, which is similar to the estimated 184 minutes suggested by Niklas et al. (2016). The direct costs are related to different aspects of the treatment, including the cost of the hospital stay, physician salaries, investment costs for the robot, medication cost and the operating room cost. In addition, overhead costs are included for heating, facility, maintenance, cleaning and other indirect costs. The total cost for a radical prostatectomy is estimated to be €6684 (7316 USD). 4.3 Multi-stakeholder value in radical prostatectomy The final step in identifying a multi-stakeholder definition of value involves combining the obtained results for the quality criteria and cost components to find the best opportunities for improving value. Lee et al. (2016) suggest a value-driven approach to address the needs of healthcare decision-makers. The methodology starts with identifying costs related to different aspects of the treatment and determining the average cost of a radical prostatectomy, as mentioned in Section 4.2. Next, the costs were allocated to the corresponding quality aspects of the treatment. In the second step, an opportunity index was used to identify potential areas for cost reduction. The opportunity index was calculated based on the variability of the costs. Cost variability is derived by dividing the standard deviation by the mean of the different types of costs. This is essential to identify aspects of the treatment that have a high cost or high variability. Due to the limited availability of data, however, the cost variability cannot be derived and will be considered in future work. The third step as proposed by Lee et al. (2016) focused on the key quality indicators needed to achieve perfect care. The thresholds for the perfect care indicators were determined with binary variables. For example, the threshold value for surgeon experience is 100 cases. The binary variable equals 1 if the surgeon has performed more than 100 cases and 0 if otherwise. Reaching 100% for all of the indicators would result in perfect care, and hence care with a higher value. In the final step, quality and cost information were combined in order to select the best opportunities to improve value. The most important quality criteria, as identified in Table 3, were linked to the respective cost types. For example, the length of stay (LOS) is influenced by the cost of the hospital stay and medication costs, while urinary incontinence (UI) is influenced by investment costs in medical equipment, operating room costs and physician salary. By analyzing the relationships, the opportunities for value improvement were identified for the criteria with high weights and a high opportunity index. Due to the absence of appropriate cost data, the opportunity index cannot be calculated. Therefore, another approach was used to calculate the ratio of the quality criteria and the estimated costs related to this aspect of the treatment. This ratio indicates the relative importance of a quality criterion per euro. For example, the value ratio of surgical margins equals 0.42, which represents a high relative importance per euro, and therefore a good opportunity for value improvement. Table 4 gives an overview of the value ratios for each of the key quality indicators. IJAHP Article: Moons, Pintelon, Jorissen, De Ridder, Everaerts/Identification of multi- stakeholder value in medical decision-making International Journal of the Analytic Hierarchy Process 95 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 Table 4 Value ratio based on quality weight and cost criteria Quality weight Cost (€) Value ratio SM 0.1586 3782 0.42 SE 0.1458 3782 0.39 UI 0.0971 3782 0.26 AT 0.0925 3782 0.21 COMP 0.0873 4084 0.26 TTRF 0.0605 4084 0.15 ED 0.0568 3782 0.15 LOS 0.0178 676 0.24 5. Discussion Value-based healthcare is characterized by the relationship between quality of care and associated costs. The objective of this research was to identify a definition for value in (robot-assisted) radical prostatectomy that was dependent on multiple, possibly conflicting, stakeholder preferences. The stakeholders included in this study were patients, urologists, nursing staff, hospital management, general practitioners and payers. The AHP is the most suitable method for solving a prioritization problem to address the needs of healthcare decision-makers. However, a few challenges must be overcome to gain the full potential of the AHP methodology. 5.1 Challenges of AHP Inconsistencies can occur due to limitations of knowledge or the complex nature of the decision problem. Saaty (2008) allows for 10% of inconsistent judgments in order to achieve qualitative results. However, when this threshold is exceeded, a method is needed to reduce the inconsistency. In this work, the weighted consistency method by Jarek (2016) was applied to reduce the inconsistency without changing the stakeholder’s judgments. On the other hand, knowledge limitation can lead to incomplete pairwise comparisons. These gaps can be filled by taking the geometric mean of other stakeholder judgments for the respective criterion (Hua, Gong & Xu, 2008). The goal was to find a multi-stakeholder definition of value for improving medical decision-making. However, stakeholders often have conflicting views, which may lead to discrepancies in the value ranking. Reaching a consensus is often believed to be the best approach to balance conflicting views (Dyer & Forman, 1992). In this work, we applied a cluster similarity approach, as proposed in Song and Hu (2009), to deal with conflicts between and within stakeholder groups. The results show differences between pre- operative patients and post-operative patients. The former group values effective and equitable care (e.g. PSA, GS, SE) more than the latter group, who focus mainly on the quality of outcomes (e.g. TTRF, UI, AT) and therefore assigns higher weights to the post- operative, patient-centered criteria. This difference can be explained by taking into account the situation of the patient. One has already been through all of the treatment stages, and the other is just starting the treatment. For example, the PSA-level received a high priority from the pre-operative patients, whereas the other stakeholders gave it lower weight. Although the PSA has few medical implications for further treatment outcomes, this criterion is often misunderstood by pre-operative patients, and therefore more IJAHP Article: Moons, Pintelon, Jorissen, De Ridder, Everaerts/Identification of multi- stakeholder value in medical decision-making International Journal of the Analytic Hierarchy Process 96 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 information about the implications of the PSA level are needed. As shown in Figure 3, equitable care is very important to the payers, nursing staff and pre-operative patients as they assigned higher weights to the pre-operative criteria. Another difference was observed in the post-operative phase, where the urologists and hospital management attached more importance to patient-centeredness, such as complications (COMP) and additional therapy (AT) since these indicate how they performed during the per-operative stage. From the results, we also observed some similarities among the stakeholders, such as the low importance of the operation time (TOR) and length of stay (LOS), as these criteria reflect cost factors or efficiency, which are relatively unimportant when compared with criteria that are related to the quality of care. Effective care, on the other hand, received high weights from all of the stakeholders involved. Surgical margins (SM) are related to the quality of patient outcomes and can predict recurrence (Fontenot & Mansour, 2013). In addition, the research shows that surgeon experience (SE) impacts the patient outcome as well as the probability of complications (Fossati et al., 2017; Di Pierro et al., 2014). In addition, preferences can differ within stakeholder groups due to personal characteristics or previous experiences with medical treatments. A further classification of the pre- and post-operative patient groups according to their age and level of education would give more insight into the different preferences. The preliminary results showed that age does not necessarily influence preferences since older patients’ assigned similar weights to erectile dysfunction and complications when compared to younger patients. Furthermore, the level of education had an influence on the pre-operative criteria, as patients with a lower level of education attached more importance to the PSA level than patients with a higher level of education. As described above, a better understanding of this medical term is required. However, further research is required to obtain statistically significant results for a larger sample size. IJAHP Article: Moons, Pintelon, Jorissen, De Ridder, Everaerts/Identification of multi- stakeholder value in medical decision-making International Journal of the Analytic Hierarchy Process 97 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 Figure 3 Radar plot showing different stakeholder preferences for quality of care indicators 5.2 Limitations This research also has some limitations. Due to the small sample size in this case study, the results cannot be validated for all of the stakeholder groups. A larger sample size would lead to more reliable results and allow for stratification to identify preference differences within stakeholder groups. Furthermore, the AHP approach does not allow for interdependency between criteria to simplify the problem. However, the relative importance of the treatment stages is not necessarily independent, since poor pre- or per-operative conditions may cause undesired patient outcomes. Moreover, uncertainty perceived by stakeholders during pairwise comparisons is not considered in this AHP approach. This can be resolved by incorporating fuzzy logic into the AHP. Finally, the cost estimations in this work were not very accurate due to the limited availability of data. As a consequence, the value-driven approach by Lee et al. (2016) could not be implemented. An alternative approach was used to calculate the value ratios in Table 4, which are not very accurate due to limited information on cost variability and how the different cost components relate to the quality aspects. The value ratios are useful to identify relevant criteria to improve the value of the treatment by limiting cost variability. The high value ratio for the length of stay is remarkable. Due to low costs and the variability of hospital stays, limiting the variability could improve value. In addition, value can be improved by offering training opportunities to surgeons and increasing cost- awareness. IJAHP Article: Moons, Pintelon, Jorissen, De Ridder, Everaerts/Identification of multi- stakeholder value in medical decision-making International Journal of the Analytic Hierarchy Process 98 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 5.3 Future perspectives Further research should focus on determining the cost drivers and cost variability to identify significant value improvement opportunities. Furthermore, an alternative method can be used for weighting the stakeholder groups to obtain the overall ranking of the quality indicators. Instead of cluster similarity, weights for stakeholder groups can be calculated based on the size of the groups or the knowledge of the stakeholder about each respective criterion. In the latter method, urologists will give higher weights for the Gleason score, whereas patients would give higher weights for post-operative consequences. The resulting weights can be varied with a sensitivity analysis to investigate the impact on the overall ranking (Forbes et al., 2018). Finally, alternatives can be added to the hierarchy. In this case study, the alternatives represent the different treatment options (i.e., surgery, radiation therapy or waiting). The identified criteria weights were used to assign an overall score for each alternative. As a result, the AHP helps decision makers choose the treatment that provides the most value based on a ranking of the alternatives. If the model is extended with additional alternatives, this allows it to be validated in terms of functionality and consistency. 6. Conclusion This work stresses the need to rigorously measure value using MCDM techniques in value-based prostate cancer care by clearly defining quality of care and determining the associated costs, while considering all of the stakeholders involved in the radical prostatectomy care cycle. Healthcare providers that increase value will be the most competitive. The AHP is used as an innovative and structured approach to address the needs of healthcare decision-makers. The major challenge lies in mapping all of the stakeholder preferences into one value definition in order to identify opportunities for quality improvement and cost containment. In this research, an overall ranking of the criteria was constructed according to the six dimensions of quality of care. The results show that the pillars effective, patient-centered and equitable care contribute most to value creation. Although the information on cost aspects was not accurate, a value ratio was computed to identify opportunities for value improvement. This case study shows the possibilities and challenges of applying AHP as a guideline for medical decision-making. IJAHP Article: Moons, Pintelon, Jorissen, De Ridder, Everaerts/Identification of multi- stakeholder value in medical decision-making International Journal of the Analytic Hierarchy Process 99 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 REFERENCES Adunlin, Georges, Vakaramoko Diaby and Hong Xiao (2015). Application of Multicriteria Decision Analysis in health care: a systematic review and bibliometric analysis. Health Expectations 18(6), 1894–1905. doi: https://doi.org/10.1111/hex.12287 De Bekker-Grob, E. W., Bliemer, M.C.J., Donkers, B. et al. (2013). Patients’ and urologists’ preferences for prostate cancer treatment: a discrete choice experiment. British Journal of Cancer, 109, 633-640. doi: https://doi.org/10.1038/bjc.2013.370 Belgian Cancer Registry, BCR. 2017. Cancer Fact Sheet Prostate Cancer. Dyer, R.F., and Forman, E.H. (1992). Group decision support with the Analytic Hierarchy Process. Decision Support Systems, 8(2), 99-124. doi: https://doi.org/10.1016/0167-9236(92)90003-8 Fontenot, P.A. & Mansour, A.M. (2013). Reporting positive surgical margins after radical prostatectomy: time for standardization. BJU International, E290-299. doi: https://doi.org/10.1111/j.1464-410x.2012.11640.x Forbes, J., Hebb, A. and Mu, E. (2018). Ethical decision making in action: evaluating hospital care attendance approaches. International Journal of the Analytic Hierarchy Process, 10(3), 313-347. doi: . https://doi.org/10.13033/ijahp.v10i3.592 Fossati, N., Di Trapani, E., Gandaglia, G.et al. (2017). Assessing the impact of surgeon experience on urinary continence recovery after robot-assisted radical prostatectomy: results of four high-volume surgeons. Journal of Endourology, 31(9). doi: https://doi.org/10.1089/end.2017.0085 Hariharan, Seetharaman, Prasanta K. Dey, Harley S. L. Moseley, Areti Y. Kumar, and Jagathi Gora (2004). A new tool for measurement of process‐based performance of multispecialty tertiary care hospitals. International Journal of Health Care Quality Assurance, 17(6), 302–12. doi: https://doi.org/10.1108/09526860410557552 Hayes, J.H., Ollendorf, D.A., Pearson, S.D. et al. (2013). Observation versus initial treatment for men with localized, low-risk prostate cancer: a cost-effectiveness analysis. Annals of Internal Medicine. doi: https://doi.org/10.7326/0003-4819-158-12-201306180- 00002 Hua, Z., Gong, B. and Xu, X. (2008). A DS-AHP approach for multi-attribute decision making problem with incomplete information. Expert Systems with Applications, 34(3), 2221-2227. doi: https://doi.org/10.1016/j.eswa.2007.02.021 Hyams, E.S., Partin, A.W. Pierorazio, P.M. et al. (2012). Impact of robotic technique and surgical volume on the cost of radical prostatectomy. Journal of Endourology, 27(3). doi: https://doi.org/10.1089/end.2012.0147 Institute of Medicine (IOM). 2001. Crossing the quality chasm: A new health system for the 21th Century. IJAHP Article: Moons, Pintelon, Jorissen, De Ridder, Everaerts/Identification of multi- stakeholder value in medical decision-making International Journal of the Analytic Hierarchy Process 100 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 Institute of Medicine (IOM). 2006. Performance measurement: Accelerating improvement. Washington, DC. Jarek, S. (2016). Removing invonsistency in pairwise comparison matrix in the AHP. Multiple Criteria Decision Making, 11, 63–76. doi: https://doi.org/10.22367/mcdm.2016.11.05 Kaplan, R.S. and Porter, M.E. (2011). How to solve the cost crisis in health care. Harvard Business Review. Lee, R.C., Donaldson, C. and Cook, L.S. (2003). The need for evolution in healthcare decision modeling. Medical Care, 41(9),1024–33. doi: https://doi.org/10.1097/01.mlr.0000083746.54410.cf Lee, V.S., Kawamoto, K., Hess, R. et al. (2016). Implementation of a value-driven outcomes program to identify high variability in clinical costs and outcomes and association with reduced cost and improved quality. JAMA - Journal of the American Medical Association, 316(10), 1061-1072. doi:10.1001/jama.2016.12226 Litwin, M.S. and Tan, H.J. (2017). The diagnosis and treatment of prostate cancer: a review. JAMA - Journal of the American Medical Association, 317(24), 2532-2542. doi: 10.1001/jama.2017.7248. Loeb, S. (2016). Evidence-based versus personalized prostate cancer screening: Using baseline prostate-specific antigen measurements to individualize screening. Journal of Clinical Oncology, 34(23), 2684-2686. doi: https://doi.org/10.1200/jco.2016.68.2138 Lotan, Y., Cadeddu, J.A. and Gettman, M.T. (2004). The new economics of radical prostatectomy: cost comparison of open, laparoscopic and robot assisted techniques. Journal of Urology, 172(4), 1431-1435. doi: https://doi.org/10.1097/01.ju.0000139714.09832.47 MacLennan, S., Bekema, H.J., Williamson, P.R. et al. (2015). A core outcome set for localised prostate cancer effectiveness trials: protocol for a systematic review of the literature and stakeholder involvement through interviews and a Delphi survey. Trials, 16(1), 76. doi: https://doi.org/10.1186/s13063-015-0598-0 Maestrini, V., Luzzini, D., Maccarrone, P. and Caniato, F. (2017). Supply chain performance measurement systems: a systematic review and research agenda. International Journal of Production Economics, 183(A), 299-315. doi: https://doi.org/10.1016/j.ijpe.2016.11.005 Marsh, K. Goetghebeur, M., Thokala, P. and Baltussen, R. (2017). Multi-Criteria Decision Analysis to support healthcare decisions. Springer International Publishing. doi: https://doi.org/10.1007/978-3-319-47540-0 Marsh, K., Lanitis, T., Neasham, D. et al. (2014a). Assessing the value of healthcare interventions using Multi-Criteria Decision Analysis: a review of the literature. IJAHP Article: Moons, Pintelon, Jorissen, De Ridder, Everaerts/Identification of multi- stakeholder value in medical decision-making International Journal of the Analytic Hierarchy Process 101 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 PharmacoEconomics, 32(4), 345–65. Marsh, K., Lanitis, T., Neasham, D. et al. (2014b). Assessing the value of healthcare interventions using Multi-Criteria Decision Analysis: a review of the literature. PharmacoEconomics. doi: https://doi.org/10.1007/s40273-014-0135-0 Martin, N.E., Massey, L., Stowell, C. et al. (2015). Defining a standard set of patient- centered outcomes for men with localized prostate cancer. European Urology, 67(3), 460-467. doi: https://doi.org/10.1016/j.eururo.2014.08.075 Mukherjee, A. (2008). Redefining health care: creating value-based competition on results. International Journal of Pharmaceutical and Healthcare Marketing. Niklas, C., Saar, M., Berg, B. et al. (2016). Da Vinci and open radical prostatectomy: comparison of cinical outcomes and analysis of insurance costs. Urologia Internationalis, 96(3). doi: https://doi.org/10.1159/000431104 Di Pierro, G.B., Wirth, J.G., Ferrari, M. et al. (2014). Impact of a single-surgeon learning curve on complications, positioning injuries, and renal function in patients undergoing robot-assisted radical prostatectomy and extended pelvic lymph node dissection. Urology, 84(5), 1106-1111. doi: https://doi.org/10.1016/j.urology.2014.06.047 Porter, Michael E (2010). What is value in health care? The New England Journal of Medicine, 363, 2477-2481. doi: 10.1056/NEJMp1011024 Porter, M.E. and Lee, T.H. (2013). The strategy that will fix health care. Harvard Business Review. Porter, M.E. and Lee, T.H. (2016). From volume to value in health care: the work begins. JAMA - Journal of the American Medical Association, 316(10), 1047-1048. Prostate Cancer Foundation, PCF (2014). Understanding surgery for prostate cancer. Richman, M.B., Forman, E.H., Bayazit, Y. et al. (2005). A novel computer based expert decision making model for prostate cancer disease management. Journal of Urology, 174(6), 2310-2318. doi: https://doi.org/10.1097/01.ju.0000181829.07078.22 Roth, A.J., Weinberger, M.I. and Nelson, C.J. (2008). Prostate cancer: quality of life, psychosocial implications and treatment choices. Future Oncology, 4(4), 561-568. doi: https://dx.doi.org/10.2217%2F14796694.4.4.561 Saaty, T.L. (1990a). How to make a decision: the Analytic Hierarchy Process. European Journal of Operational Research, 48(1), 9–26. Saaty, T.L. (1990b). How to make a decision: the Analytic Hierarchy Process. European Journal of Operational Research, 48(1), 9-26. doi: https://doi.org/10.1016/0377- 2217(90)90057-I Saaty, T.L. (2008). Decision making with the Analytic Hierarchy Process. International IJAHP Article: Moons, Pintelon, Jorissen, De Ridder, Everaerts/Identification of multi- stakeholder value in medical decision-making International Journal of the Analytic Hierarchy Process 102 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 Journal of Services Sciences, 1(1), 83. Saaty, T.L. and Vargas, L.G. (2006). Decision making with the Analytic Network Process. Economic, political, social and technological applications with Benefits, Opportunities, Costs and Risks. International Series in Operations Research Management Science. Springer. Schmidt, K., Aumann, I., Hollander, I. et al. (2015). Applying the Analytic Hierarchy Process in healthcare research: a systematic literature review and evaluation of reporting. BMC Medical Informatics and Decision Making, 15(1), 112. doi: https://doi.org/10.1186/s12911-015-0234-7 Song, Y. and Hu, Y. (2009). Group decision-making method in the field of coal mine safety management based on AHP with clustering. 6th International Conference on Information Systems for Crisis Response and Management (ISCRAM 2009). Stephani, V., Crommelynck, A., Durant, G. et al. (2018). Financiering van Ziekenhuisverblijven Met Een Grote Variabiliteit in Het Zorgproces. Swartenbroekx, N., Obyn, C., Guillaume, P. et al. (2012). Handleiding Voor Op-Kosten- Gebaseerde Prijsbepaling van Ziekenhuisinterventies. Brussel. Thokala, P., Devlin, N., Marsh, K. et al. (2016). Multiple Criteria Decision Analysis for health care decision making - an introduction: Report 1 of the ISPOR MCDA emerging good practices task force. Value in Health, 19(1), 1–13. doi: https://doi.org/10.1016/j.jval.2015.12.003 Zardari, N.H., Ahmed, K., Moniruzzaman Shirazi, S. and Yusop, Z.B. (2015). Weighting methods and their effects on Multi-Criteria Decision making model outcomes in water resources management. International Journal of Operations Research. Springer. IJAHP Article: Moons, Pintelon, Jorissen, De Ridder, Everaerts/Identification of multi- stakeholder value in medical decision-making International Journal of the Analytic Hierarchy Process 103 Vol. 12 Issue 1 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 APPENDIX A Overview of quality criteria, definitions and abbreviations Criterion Definition Abbreviation PSA level Prostate Specific Antigen level prior to the surgery PSA Clinical stage Cancer stage before the treatment starts CS Gleason score Prostate cancer score assigned based on microscopic appearance of tumor before surgery GS Time in operating room Total time patient is in the operating room TOR Surgical margins Thickness of normal tissue around cancer cells, examined from pathologic report. Positive surgical margins mean that the cancer cells touch the edge, whereas negative surgical margins mean there is a ring of normal tissue around cancer cells SM Surgeon experience Number of patients that the surgeon has treated during the past year SE Blood transfusion Necessity of one or more blood transfusions during surgery BT Length of stay Number of days the patient stays in the hospital after surgery LOS Complications Occurrence and severity of undesired consequences after surgery COMP Urinary incontinence Occurrence of unwanted urine leakage from bladder and/or pain during urination UI Erectile dysfunction Ability to have an erection and quality of the erection ED Time to return to normal functioning Number of days after surgery until patient can function normally as he could before surgery TTRF Additional therapy Necessity of ongoing interventions after surgery AT