Emergency. 2016; 4 (2): 111-113 EV I D E N C E BA S E ME D I C I N E Evidence Based Emergency Medicine; Part 5 Receiver Op- erating Curve and Area under the Curve Saeed Safari1, Alireza Baratloo1, Mohamed Elfil2∗, Ahmed Negida3. 1. Emergency Department, Shohadaye Tajrish Hospital, Shahid Beheshti University of Medical Sciences, Tehran, Iran. 2. Faculty of Medicine, Alexandria University, Egypt. 3. Faculty of Medicine, Zagazig University, Egypt. Received: March 2016; Accepted: April 2016 Cite this article as: Safari S, Baratloo A, Elfil M, Negida A. Evidence Based Emergency Medicine; Part 5 Receiver Operating Curve and Area under the Curve . Emergency. 2016; 4(2):111-113. 1. ABSTRACT Multiple diagnostic tools are used by emergency physicians, every day. In addition, new tools are evaluated to obtain more accurate methods and reduce time or cost of conven- tional ones. In the previous parts of this educational series, we described diagnostic performance characteristics of di- agnostic tests including sensitivity, specificity, positive and negative predictive values, and likelihood ratios. The receiver operating characteristics (ROC) curve is a graphical presenta- tion of screening characteristics. ROC curve is used to deter- mine the best cutoff point and compare two or more tests or observers by measuring the area under the curve (AUC). In this part of our educational series, we explain ROC curve and two methods to determine the best cutoff value. 2. Definition ROC curve is a graphical presentation, which is created by plotting the true positive rate (sensitivity) against the false positive rate (1-specificity) at various threshold settings (1,2). 3. The area under ROC curve It is known that when sensitivity increases, false positive rate also increases. Therefore, ROC curve allows us to check sen- sitivity and false positive rate (1- specificity) at any point on the curve. The area under the curve is a measurement of the overall quality of the diagnostic test. Tests with the same AUC show the same overall diagnostic performance, but not the same sensitivity and specificity. AUC is interpreted as ∗Corresponding Author: Mohamed Elfil; Faculty of Medicine, Alexandria Uni- versity, Egypt. Address: 601 Delma Plaza, Smouha, Alexandria, Egypt.Tel: +201285818091. E-mail: M_elfil13@alexmed.edu.eg. Figure 1: ROC curve for a test with a sensitivity and specificity of 100% (A). ROC curve for a useless test (B). the probability that a random person with the disease has a higher test measurement than a random person who is healthy. Accuracy of a test is measured by the AUC, which can be calculated by many statistical software (1,2). Based on a rough classifying system, AUC can be interpreted as fol- lows: 90 -100 = excellent; 80 - 90 = good; 70 - 80 = fair; 60 - 70 = poor; 50 - 60 = fail. In figure 1, the line (A) represents the ROC for an ideal diag- nostic test. This curve represents a sensitivity and specificity of 100%. The AUC for this line is equal to 100, which is not common in practice. On the other hand, the diagonal line (B) represents ROC for a useless diagnostic test. This can be explained by noting that the true positive rate of the diagnos- This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: www.jemerg.com M_elfil13@alexmed.edu.eg S. Safari et al. 112 Figure 2: ROC curve of TRISS system Figure 3: ROC curve of APACHE III system tic test is equal to the false positive rate at any cutoff point. The AUC for this line is equal to 45, which is why such test is of no real value. 4. Determining the best cutoff point on ROC curve There are two methods to choose the best cutoff point. 5. The first method It is known that the upper left corner of the graph represents the point with 100% sensitivity and 100% specificity. There- fore, the point on the curve that gives the highest sensitiv- ity and the highest specificity will be the nearest point to the upper left corner of the graph. To determine the nearest point, we should measure the distance between each point and the upper left corner and choose the nearest point as the best cutoff point. To measure the distance of each point from the upper left corner, we can use the following equa- tion: Distance = √ [(1 − se n si t i v i t y )2 + (1 − s p e c i f i c i t y )2] The point with the shortest distance should be chosen as the best cutoff point. 6. The second method The second method is called Youden index where we aim at maximizing the gap between true positive rate (repre- sented as: sensitivity) and false positive rate (represented as 1- specificity). In this method, we calculate (sensitivity + specificity) for all points and Youden index is the point where the sum of sensitivity and specificity is the highest. 7. Example In the recently published study by Mazandarani and col- leagues (4), they compared the ability of trauma injury severity score (TRISS) and acute physiology and chronic health evaluation (APACHE) III to predict mortality of in- tensive care unit admitted trauma patients. ROC curves for the methods are shown in figure 2 and 3. Figure 2 shows the ROC curve for TRISS; we notice that the curve is nearest to the upper left corner in the cutoff point (13.2) and therefore, the maximum sensitivity and specificity are achieved at this point (sensitivity = 76.52%; specificity = 95.65%). Figure 3 shows the ROC curve for APACHE III; we notice that the curve is nearest to the upper left corner in the cutoff point (51) and therefore, maximum sensitivity and specificity are achieved at this point (sensitivity = 73.91%; specificity = 78.26%). 8. Notes * This method might yield more than one cutoff point and therefore choosing between them is challenging. * This method gives sensitivity and specificity equal priority in the diagnostic test; sometimes the cost of screening is high and the disease is rare, in these conditions, sensitivity and specificity are not given the same priority when determining the best cutoff point. The method of integrating the cost of This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: www.jemerg.com 113 Emergency. 2016; 4 (2): 111-113 screening and prevalence of the disease with sensitivity and specificity while determining the best cutoff is rarely used in medical literature. References 1. Kumar R, Indrayan A. Receiver operating characteris- tic (ROC) curve for medical researchers. Indian Pediatr. 2011;48(4):277–87. 2. Altman DG, Bland JM. Diagnostic tests 3: receiver operat- ing characteristic plots. BMJ. 1994;309(6948):188. 3. Akobeng AK. Understanding diagnostic tests 3: re- ceiver operating characteristic curves. Acta Paediatr. 2007;96(5):644–7. 4. Acute Physiology and Chronic Health Evaluation (APACHE) III Score com- pared to Trauma-Injury Severity Score (TRISS) in Predicting Mortality of Trauma Patients. 2015;3:1–4. This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: www.jemerg.com ABSTRACT Definition The area under ROC curve Determining the best cutoff point on ROC curve The first method The second method Example Notes References