Hrev_master [Emergency Care Journal 2016; 12:6314] [page 75] Which clinical scoring system should we use for the evaluation of chest pain in the Emergency Department? A review Nicola Parenti, Federica Agrusta, Antonio Luciani Department of Emergency Medicine, Modena Hospital, Modena, Italy Abstract Many clinical prediction rules and scoring systems have been developed to predict Acute Coronary Syndrome in-patient with chest pain (CP) in Emergency Department. In this review we check and compare the level of validity and reliability of the TIMI (thrombolysis in myocar- dial infarction); HEART (history, ECG, age, risk factors, troponin); GRACE (global registry of acute coronary events). We used as eligibility criteria: all studies, reviews and meta-analysis on validity and reliability of clinical score sys- tems for CP conducted on all ages of patients in all languages. The selection of articles included in the review was performed accord- ing to PRISMA guidelines. We collected one systematic review, one meta-analysis and eleven studies. The HEART score showed the best validity in predicting the outcomes tested with a mean AUROC value of 0.86 (range 0.83- 0.88); the GRACE score showed a good validity: mean AUROC value=0.78 (range 0.70-0.82); the TIMI a moderate validity: mean AUROC=0.67 (range 0.42-0.79). The only included study on the reliability showed that the TIMI score had a poor to moderate reliabil- ity: weighted kappa range=k= 0.30-0.43. In conclusion, in this review the HEART and GRACE scores showed the best validity in pre- dicting acute coronary syndromes and major cardiac events. To our knowledge there is only one study on the reliability of TIMI score that showed a poor to moderate inter-rater reliabil- ity. There are no studies on the reliability of other score systems. Introduction Chest pain is a common complaint in Emergency Department (ED). It is often asso- ciated to serious diseases like acute coronary syndrome (ACS) with a significant morbidity and mortality. For these reasons, many clinical prediction rules and scoring systems have been developed to manage this condition (ACS). Ideally a clinical scoring system should help to guide clinicians during their evaluation and improve the standardization of clinical man- agement. For the previous reasons many Emergency and Intensive scientific societies have sug- gested spreading an early use of scores for the evaluation of patients in the ED with chest pain.1,2 The major clinical prediction rules for chest pain are described in a recent Systematic Review:3 the TIMI (thrombolysis in myocardial infarction); HEART (history, ECG, age, risk fac- tors, troponin); GRACE (global registry of acute coronary events). According to the literature, a clinical score system used in ED should also be easy and rapid to use. Stiell et al.4 suggested the methodologic standards for the development of clinical deci- sion rules in Emergency Medicine: before using these tools they should be tested in vari- ous setting (e.g. primary care, hospital, emer- gency) for the major quality indexes (the reli- ability and validity in predict diagnosis and prognosis). For these reasons we reviewed the literature to check and compare the level of validity and reliability of the major score systems devel- oped for the evaluation of chest pain in ED. Materials and Methods The primary aim was to check the state of art of studies on the validity and reliability of clinical score systems for chest pain. We used the following eligibility criteria to include reports: all studies, reviews and meta- analysis on validity and reliability of clinical score systems for CP conducted on all ages of patients in all languages. In particular, we con- sider the studies which used the following validity indexes: accuracy, sensitivity, speci- ficity measures; ROC curves with areas under the ROC curves (AUCs); the following reliabil- ity indexes: kappa coefficient (weighted and un-weighted), intraclass correlation coeffi- cient, Pearson correlation coefficient and spearman rank correlation coefficient. The selection of articles included in the review was performed in a three-phase process (Figure 1) according to PRISMA guidelines. In the first phase, one author conducted a literature search of PubMed database. The systematic search of the international literature published from 1941 through June 2016 was conducted using the following key words: chest pain and clinical prediction rule and emergency service OR chest pain and risk score and emergency service. Ninety-seven and one hundred and sixty-eight citations have been found. In the second phase, the author performed a screening by regarding eligibility criteria of the list of articles (title and abstract) selected in phase one. All duplicates were removed. In this way potentially useful articles were select- ed with their full text. In the third phase, all full-text articles have been examined to select the studies that met the inclusion criteria. The studies remaining after this phase were included in the analysis. Results We collected one systematic review, one meta-analysis and eleven studies:3,5-16 the meta-analysis, the review and ten studies test- ed the validity in predicting ACS and major car- diac events of TIMI, GRACE and HEART score; one study the validity and reliability. The char- acteristics of studies are shown in Table 1. The HEART score showed the best validity in pre- dicting the outcomes tested with a mean AUROC value of 0.86 (range 0.83-0.88) but we found only two studies on this score. The GRACE score has been tested in several studies and setting (emergency department, cardiology and ICU) and it showed a good validity: mean AUROC value=0.78 (range 0.70- 0.82). In this review the TIMI score showed a mod- erate validity: mean AUROC=0.67 (range 0.42- 0.79). The good performance of the GRACE score has also been confirmed in five comparison studies with other scores (Table 1). The test characteristics of major clinical score systems for Chest Pain described in the Emergency Care Journal 2016; volume 12:6314 Correspondence: Nicola Parenti, Department of Emergency Medicine, Modena Hospital, via S. Giovanni del cantone 23, 41121 Modena, Italy. Tel: +39.051.9912065; +39.377.4568851. E-mail: nipar71@yahoo.it Key words: Chest pain; Clinical scoring system; Review; Validity; Reliability. Conflicts of interest: the authors declare no potential conflict of interest. Received for publication: 8 September 2016. Accepted for publication: 17 November 2016. This work is licensed under a Creative Commons Attribution 4.0 License (by-nc 4.0). ©Copyright N. Parenti et al., 2016 Licensee PAGEPress, Italy Emergency Care Journal 2016; 12:6314 doi:10.4081/ecj.2016.6314 Non co mmerc ial us e o nly [page 76] [Emergency Care Journal 2016; 12:6314] systematic review and meta-analysis are shown in Table 2. Fanaroff et al.3 suggested that the TIMI and HEART risk score have excel- lent accuracy for ACS: patients with a TIMI score of 5 or higher have a summary LR for ACS of 6,8; among high-likelihood patients (HEART score rage of 7-10) the LR for diagno- sis of ACS was 13. D’Ascenzo et al.,16 after a revision of seven derivation studies (25,525 patients) and fif- teen validation studies (257,654 patients), concluded that GRACE risk score performed better than the TIMI when predicting ACS. The only included study on the reliability showed that the Goldman, the TIMI and Sanchis score had a poor to moderate reliabil- ity: weighted kappa range=0.18-0.30; k=0.30- 0.43; K=0.18-0.43 respectively.15 A comparison of the overall validity in pre- dicting outcome of the major clinical predic- tion scores for chest pain is shown in Figure 2. Discussion In this review the HEART and GRACE score systems for the evaluation of chest pain in Emergency Department showed the best valid- ity in predicting acute coronary syndromes and major cardiac events. According to the system- atic review3 and to the meta-analysis16 the best score systems were the TIMI and HEART; the GRACE respectively. To our knowledge there is only one study on the reliability of TIMI score that showed a poor to moderate inter-rater reliability. There are no studies on the reliability of other score sys- tems. For several years, researchers tried to develop a risk score for chest pain patients. Most of these scores are difficult to use and are only validated for a selected group of patients such as STEMI or non-STEMI patients in the coronary care unit. The major and useful scores are the TIMI,17 Review Table 1. Characteristics of studies included. Reference (author, year) Target population setting Score Design Outcome Results Leite et al., 20155 233 patients HEART Retrospective Mortality, AUROC=0.88 ED Observational AMI, Revascularization at 6 weeks Sanchis et al., 20056 646 patients TIMI Prospective Mortality, AUROC=0.66 AMI at 1 yr Rawlings et al., 20127 104 pat TIMI Prospective Mortality NSTEMI AUROC=0.67 for death ED at 30 days AUROC=0.42 for NSTEMI Macdonald et al., 20148 219 pat TIMI Multicenter Prospective obs Mortality AMI AUROC=0.71 ED at 30 days Lyon et al., 20079 1000 pat GRACE Prospective ACS, GRACE: AUROC=0.80 ED TIMI cardiac arrest, TIMI: AUROC=0.79 death at 30 days De Araujo Gonçalves et al., 200510 460 pat GRACE Prospective Death , AMI at 1 yrs GRACE: AUROC=0.72 TIMI TIMI: AUROC=0.59 PURSUIT PURSUIT AUROC=0.63 Yan et al., 200711 1728 pat GRACE Multicenter In hospital GRACE: AUROC=0.79 TIMI Prospective and death at 1 year TIMI: AUROC=0.69 PURSUIT PURSUIT AUROC=0.77 Ramsay et al., 200712 347 pat GRACE Prospective Death, AMI, GRACE: AUROC=0.82 TIMI In-hospital at 3 months TIMI: AUROC=0.74 Backus et al., 201313 2440 pat GRACE Multicenter Major cardiac GRACE: AUROC=0.70 ED TIMI Rospective Events at 6 weeks TIMI: AUROC=0.75 HEART HEART: AUROC=0.83 Granger et al., 200314 11389 pat GRACE Retrospective In hospital death AUROC=0.84 Manini et al., 200915 148 ED pat TIMI Prospective cohort NSTEMI Sens=35-53% GOLDMAN Reliability Spec=72-86% SANCHIS Reliability=poor-moder AMI, acute myocardial infarction; ACS, acute coronary syndromes. In this list the two meta-analyses are excluded. Figure 1. Review process. Non co mmerc ial us e o nly [Emergency Care Journal 2016; 12:6314] [page 77] the GRACE,14 and the HEART.13 The TIMI risk score (2000) is derived from the Thrombolysis in Myocardial Infarction (TIMI)-11B trial, a multinational, randomized clinical trial, comparing unfractionated heparin to enoxaparin, which included all patients with confirmed ACS. Seven elements compose the TIMI score for unstable angina/NSTEMI: age ≥65 years, ≥3 classical risk factors for coronary artery disease (CAD), known CAD, use of Aspirin in the past 7 days, severe angina in the past 24 hours, elevated cardiac markers and ST-deviation ≥0.5 mm. Each of these elements can be assigned with 0 or 1 points, resulting in a score of 0-7. The TIMI score predicts the risk of all causes mor- tality, MI and severe recurrent ischemia requiring urgent revascularization within 14 days after admission. The GRACE score (2003) was developed in a multinational registry of 11,389 ACS patients (global registry of acute coronary events). The scoring system, consisting of hemodynamic, laboratory, ECG and patient specific findings: Killip class for congestive heart failure (CHF), systolic blood pressure at presentation (SBP), heart rate at presentation (HR), age, creati- nine level, cardiac arrest at admission, ST-seg- ment deviation on the index ECG and elevated cardiac enzyme levels. Each element has its own scoring, resulting in a possible score ranging from 1 to 372. Event rates increased significantly with increasing GRACE-scores, ranging from ≤0.2 to ≥52% chance of in-hospi- tal death. Recently (2008), the HEART risk score was developed for chest pain patients presenting to the ED. The composition of the HEART score was not based on multivariate regression analysis but on the decision making clinical factors according to expert opinion. The HEART score is composed of five parameters of clinical judgement: history, ECG, age, risk fac- tors and troponin. By appreciating each of these five elements with 0, 1 or 2 each patient will receive a score of 0-10. The HEART score divides patients into low (0-3), intermediate (4-6) or high-risk groups (7-10), with mean risks of an event of 0.9, 12 and 65%, respective- ly. One limit of the TIMI and GRACE scores is that they have been originally developed for patients with confirmed ACS in the setting of coronary unit. In a second time they have been tested in the ED setting. The HEART, instead, is the only score, which was developed for chest pain patients present- ing to the ED. One other limit of all the previous scores is that they could not be used to discriminate other causes of chest pain (e.g. aortic dissec- tion, pulmonary embolism, non cardiac chest pain). But in our opinion the main limit of all the previous score systems is the lack of research on their reliability. In fact, according to litera- ture,4 before using a clinical score it could be very important to test its inter-rater and intra- rater reliability. Among the studies collected in this review, the most studied score in the ED setting is the TIMI with four studies conducted on 3763 patients; the HEART and GRACE score showed the best validity in the ED setting: AUROC range 0.83-0.88 and 0.70-0.80 respectively. But which could be the most useful and effectiveness clinical score systems to evaluate a patient with chest pain in the daily ED clini- cal practice? In our opinion the best score in a suspicious of ACS in ED setting should be faster, simpler, intuitive, valid in predicting the main out- comes (short and long term mortality, AMI, major cardiac events), reliable. But to our knowledge, until now there is not a score with all these characteristics. Moreover it is very difficult to compare the results on the perform- ance of score models evaluated in studies with different setting, design and outcome tested. Anyway according to the results of this and other reviews and meta-analysis, probably the useful scores in ED could be the HEART and GRACE because both showed good validity in predicting major cardiac events in short and long term. Our opinion has also been support- ed by European Society of Cardiology which suggests the use of GRACE1 and recently by the experts’ opinion from ACEP (American College of Emergency Physician) who proposed the HEART score. The main advantages of HEART are that it Review Table 2. Test characteristics of clinical score systems for chest pain from meta-analysis and systematic review. SCORE Threshold LR AUROC HEART Low Probability 0-3 0.20 Indeterminate 4 0.79 Mod. Probability 5-6 2.4 High Probability 7-10 13 TIMI ° 0.66 Low probability 0-1 0.31 (95% CI 0.64 -0.68) Indeterminate 2 0.94 °° 0.73 Mod. Probab. 3-4 2.4 (95% CI 0.69- 0.78) High Probab. 5-7 6.8 HFA/CSANZ Low probability Low to mod high 0.24 Mod. Probab. 2.8 GRACE °0.83 (95% CI 0.82-0.84); °°0.82 (95% CI 0.80-0.89) HEART, history, ECG, age, risk factors, troponin; TIMI, thrombolysis in myocardial infarction risk score; HFA/CSANZ, Heart Foundation of Australia and Cardiac Society of Australia and New Zealand rule; GRACE, global registry of acute coronary events. °The authors considered the overall AUROC for the ability of GRACE and TIMI to predict in –hospital and first year mortality, major cardiac events, ACS diagnosis, cardiac arrest among Seven derivation studies (25,525 patients); °°15 validation studies (257,654 patients). Figure 2. Comparison of validity in predicting outcome of clinical score for chest pain. All value of AUROC are means among the studies included. The outcome tested were: death at 6 weeks, at 30 days; ACS, MACE (major cardiac events). Non co mmerc ial us e o nly [page 78] [Emergency Care Journal 2016; 12:6314] has been developed for patients with CP in ED, it is fast, simple and intuitive, it also considers the patients history. Its limitations are: few studies published on its validity in predicting CP outcomes; no data on its reliability. Conclusions In our opinion the score actually in use for the management of patient with CP in the ED does not respect all the quality indexes sug- gested by the experts. Anyway, among the scores until now tested, probably the HEART and GRACE scores could be the most useful for their good validity in predicting outcomes and large validation in ED setting. Further research on their reliability should be conduct- ed. More studies on their comparison are desirable. References 1. Hamm CW, Bassand JP, Agewall S, H et al. ESC guidelines for the management of acute coronary syndromes in patients pre- senting without persistent ST-segment elevation Eur Heart J 2011;32:2999-3054. 2. Amsterdam EA, Kirk JD, Bluemke DA, et al. Testing of low-risk patients presenting to the emergency department with chest pain a scientific statement from the American Heart Association Circulation. Circulation 2010;122:1756-76. 3. Fanaroff AC, Rymer JA, Goldstein SA, et al. Does this patient with chest pain have acute coronary syndrome? The rational clinical examination systematic review. J Am Med Assoc 2015;10:1955-65. 4. Stiell IG, Wells GA. Methodologic standards for the development of clinical decision rules in Emergency Medicine. Ann Emerg Med 1999;33:437-47. 5. Leite L, Baptista R, Leitão J, et al. Chest pain in the emergency department: risk stratification with Manchester triage sys- tem and HEART score. BMC Cardiov Dis 2015;15:48. 6. Sanchis J, Bodí V, Núñez J, et al. New risk score for patients with acute chest pain, non-ST-segment deviation, and normal troponin concentrations a comparison with the TIMI risk score. J Am Coll Cardiol 2005;46:443-9. 7. Rawlings C, Oglesby K, Turner J, Sen A. Comparison of two clinical scoring sys- tems in risk stratification of non-ST eleva- tion acute coronary syndrome patients in predicting 30-day outcomes. Emerg Med J 2012;29:40-2. 8. Macdonald SP, Nagree Y, Fatovich DM, Brown SG. Modified TIMI risk score can- not be used to identify low-risk chest pain in the emergency department: a multicen- tre validation study. Emerg Med J 2014;31: 281-5. 9. Lyon R, Morris AC, ,Caesar D, et al. Chest pain presenting to the Emergency Department-to stratify risk with GRACE or TIMI? Resuscitation 2007;74:90-3. 10. De Araujo Gonçalves P, Ferreira J, et al. TIMI, PURSUIT, and GRACE risk scores: sustained prognostic value and interaction with revascularization in NSTE-ACS. Eur Heart J 2005;26:865-72. 11. Yan AT, Yan RT, Tan M, et al. Risk scores for risk stratification in acute coronary syndromes: useful but simpler is not nec- essarily better. Eur Heart J 2007;28:1072-8. 12. Ramsay G, Podogrodzka M, Mcclure C, Fox KAA. Risk prediction in patients present- ing with suspected cardiac pain: the GRACE and TIMI risk scores versus clini- cal evaluation. Q J Med 2007;100:11-8. 13. Backus BE, Six AJ, Kelder JC, et al. Chest pain in the emergency room; a multicenter validation of the HEART score. Crit Path Cardiol 2010;9:164-9. 14. Granger CB, Goldberg RJ, Dabbous O, et al. Predictors of hospital mortality in the glob- al registry of acute coronary events. Arch Intern Med 2003;163:2345-53. 15. Manini AF, Dannemann N, Brown DF, et al. Limitations of risk score models in patients with acute chest pain. Am J Emerg Med 2009;27:43-8. 16. D'Ascenzo F, Biondi-Zoccai G, Moretti C, et al. TIMI, GRACE and alternative risk scores in acute coronary syndromes: a meta-analysis of 40 derivation studies on 216,552 patients and of 42 validation stud- ies on 31,625 patients. Contemp Clin Trials 2012;33:507-14. 17. Antman EM, Cohen Mk, Bernink PJLM, et al. The TIMI risk score for unstable angi- na/non-ST elevation MI. J Am Med Assoc 2000;284:835. Review Non co mmerc ial us e o nly