Archives of Academic Emergency Medicine. 2022; 10(1): e52 OR I G I N A L RE S E A RC H Predicting the 28-Day Mortality of Non-Trauma Patients using REMS and RAPS; a Prognostic Accuracy Study Omid Garkaz1, Farzin Rezazadeh2, Saeed Golfiroozi3, Sahar Paryab4, Sadaf Nasiri5, Hamidreza Mehryar6∗, Mousa Ghelichi-Ghojogh7 1. School of Public Health, Shahroud University of Medical Sciences, Shahroud, Iran. 2. Emergency Medicine Department, Urmia University of Medical Sciences, Urmia, Iran. 3. Department of Emergency Medicine, School of Medicine, Golestan University of Medical Sciences, Gorgan, Iran. 4. School of Nursing and Midwifery, Shahroud University of Medical Sciences, Shahroud, Iran. 5. Urmia University of Medical Sciences, Urmia, Iran. 6. Department of Emergency Medicine, School of Medicine, Urmia University of Medical Sciences, Urmia, Iran. 7. Epidemiology Department, Golestan University of Medical Sciences, Gorgan, Iran. Received: April 2022; Accepted: May 2022; Published online: 4 July 2022 Abstract: Introduction: Various scoring systems have been designed for calculating the mortality risk of patients. This study evaluated the accuracy of Rapid Emergency Medicine Score (REMS) and Rapid Acute Physiology Score (RAPS) in predicting the 28-day mortality of non-trauma patients. Methods: This prospective cross-sectional study was conducted on 1003 adult non-trauma patients, who referred to the emergency department of Imam Khomeini Hospital, Urmia, Iran, in the second half of 2018, using the census sampling. We determined the screening performance characteristics of REMS and RAPS in predicting the 28-day mortality of patients. Results: This study examined 1003 non-trauma patients with a mean age of 61.5±18.05 years (60.6% male). The mean REMS (8.7 ± 3.2 vs. 6.0 ± 3.6; p < 0.001) and RAPS (3.7 ± 2.8 vs. 2.7 ± 2.0; p < 0.001) scores were significantly higher in deceased cases. Sensitivity and specificity of REMS in predicting the risk of non-trauma patients’ mortality were 85.19% (95%CI: 78.05% - 90.71%) and 78.34% (95%CI: 75.45% - 81.04%), respectively. While, the Sensitivity and specificity of RAPS in this regard were 61.39% (95%CI: 53.33% - 69.02%) and 71.12% (95%CI: 67.94% - 74.16%), respectively. The area under the receiver operating characteristic (ROC) curve of REMS and RAPS were 0.72 (95% CI: 0.68 -0.75) and 0.62 (95% CI: 0.56 - 0.65) in predicting the patients’ 28-day mortality, respectively (p = 0.001). Conclusion: The total accuracies of REMS and RAPS in predicting the 28-day mortality of non-trauma patients were in good and poor range, respectively. The screening performance characteristics of REMS were a little better in this regard. Keywords: Emergencies; Emergency Service, Hospital; Mortality; Clinical Decision Rules; Prognosis Cite this article as: Garkaz O, Rezazadeh F, Golfiroozi S, Paryab S, Nasiri S, Mehryar H, Ghelichi-Ghojogh M. Predicting the 28-Day Mortality of Non-Trauma Patients using REMS and RAPS; a Prognostic Accuracy Study. Arch Acad Emerg Med. 2022; 10(1): e52. https://doi.org/10.22037/aaem.v10i1.1601. 1. Introduction Various scoring systems have been designed for calculating the mortality risk of patients. These systems are methods de- signed to quantify the severity of the disease and the patient’s ∗Corresponding Author: Hamidreza Mehryar; Resalat Boulevard, Emer- gency Alley, Urmia, Iran. Postal address: 5714783734, Email: hamidreza- mehryar2010@gmail.com, ORCID: http://orcid.org/0000-0002-3267-8647. condition by integrating the various properties affecting it (1, 2). Several scoring systems have been proposed to assess disease severity in recent decades (3, 4). They are mainly used in crit- ically ill patients and their common goal is to measure devi- ations in various physiological variables to provide an objec- tive measure of the severity of the disease known to physi- cians worldwide. A wide range of applications of the tools en- visioned by Hazy are described (5). Classifying the severity of illness in the emergency department (ED), along with an ac- This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: http://journals.sbmu.ac.ir/aaem O. Garkaz et al. 2 curate history of the disease, can classify critically ill patients. Scoring systems can also be used to evaluate the use of hospi- tal resources and compare different emergency departments in short- and long-term. Scoring systems could also be a po- tential triage tool for emergency nurses. One of these scor- ing systems is Acute Physiology and Chronic Health Evalua- tion II )APACHEII(, , described by Knaus et al., which classi- fies the severity of disease (6). However, the APACHE II score contains several blood chemical variables and is, therefore, not suitable for rapid assessment in the emergency depart- ment. Rapid Acute Physiology Score (RAPS) is a summary of APACHE II, which was initially used as a hospital scoring system for the patients transported by helicopter (7). An im- portant advantage of RAPS is the simplicity of its assessment method. Rapid Emergency Medicine Score (REMS) is an attenuated version of APACHE II score and can predict mortality in non- surgical patients (8). A study on performance of REMS in predicting the in-hospital mortality of trauma patients con- firmed its simplicity and accuracy in this regard (9). Due to the limited number of variables in the RAPS and REMS, it is possible to evaluate and calculate scores based on these models and easily use them in ED. However, only a few studies have compared the two models (7, 10, 11), and there are still controversies over their use in evaluating the mortality of non-trauma patients. Given this, we intended to evaluate the accuracy of REMS and RAPS in predicting the 28-day mortality of non-trauma patients. 2. Methods 2.1. Study design and setting This descriptive cross-sectional study was conducted on 1003 non-trauma patients, who referred to the emergency department of Imam Khomeini Hospital, Urmia, Iran, in the second half of 2018, using census method. We deter- mined the screening performance characteristics of REMS and RAPS in predicting the 28-day mortality of adult non- trauma patients. This study was registered in Urmia University of Medical Sciences with the ethics code of IR.UMSU.REC.1397.213. While following the patients, we ex- plained the research plan to them and ensured their willing- ness to participate in the study. The participants were as- sured that their information would be kept confidential. 2.2. Participants In this study, adult non-trauma patients with complete pa- tient files were included. We excluded patients who had died before reaching the ED. 2.3. Data collection procedures A predesigned checklist that included information about pa- tients’ age, gender, and final outcome (deceased or dis- charged), and the REMS and RAPS was filled out for all pa- tients using their profiles. REMS variables included age, level of consciousness, mean blood pressure, respiration rate, and oxygen level; and the RAPS consisted of pulse rate, blood pressure, respiration rate, and Glasgow Coma Scale (GCS) score (appendix 1 and 2). All the information was recorded upon patients’ arrival at the emergency department and pa- tients were also contacted for a 28-day follow-up of death or recovery. An emergency physician was responsible for data gathering. 2.4. Statistical analysis The data were analyzed using SPSS18, and the findings were reported using descriptive statistics (mean ± standard devi- ation) or frequency (%). The screening performance charac- teristics of REMS and RAPS in predicting the 28-day mortal- ity of non-trauma patients were calculated using a calculator (http://vassarstats.net/clin1.html) and the area under the re- ceiver operating characteristic (ROC) curve. The analysis re- sults were reported with 95% confidence interval (CI). 3. Results This study examined 1003 non-trauma patients with a mean age of 61.5±18.05 years (60.6% male). Table 1 compares the baseline characteristics and REMS and RAPS between de- ceased and discharged patients. The mean REMS (8.7 ± 3.2 vs. 6.0 ± 3.6; p < 0.001) and RAPS (3.7 ± 2.8 vs. 2.7 ± 2.0; p < 0.001) scores were significantly higher in deceased cases. Table 2 shows the screening performance characteristics of REMS and RAPS in predicting the risk of 28-day mortality in non-trauma patients. Sensitivity and specificity of REMS in predicting the risk of in hospital mortality of non-trauma patients were 85.19% (95%CI: 78.05% - 90.71%) and 78.34% (95%CI: 75.45% - 81.04%), respectively. While, the Sensitivity and specificity of RAPS in this regard were 61.39% (95%CI: 53.33% - 69.02%) and 71.12% (95%CI: 67.94% - 74.16%), respectively. The area under the ROC curve of REMS and RAPS was 0.72 (95% CI: 0.68 -0.75) and 0.62 (95% CI: 0.56 - 0.65), respec- tively, (figure 1; p = 0.001) in predicting the 28-day mortality. 4. Discussion Based on the findings of the present study, it seems that REMS is a better predictor of the 28-day mortality of non- trauma patients in emergency department compared to RAPS. In this study, there was a significant difference be- tween the two groups (discharged and deceased patients) in This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: http://journals.sbmu.ac.ir/aaem 3 Archives of Academic Emergency Medicine. 2022; 10(1): e52 terms of mean REMS. The mean REMS of deceased patients was higher compared to that of discharged patients. There was also a significant difference between the two groups (dis- charged and deceased patients) regarding the mean RAPS. The mean RAPS of deceased patients was higher than that of discharged patients. In a similar study, Seak et al. (12) evaluated the performance of REMS in predicting in-hospital mortality in non-surgical emergency department patients. In their study, REMS was a powerful predictor of in-hospital mortality in patients admit- ted to the emergency department with a wide range of com- mon internal disorders. The results of that study were similar to our study. In a study by Ha et al. (13), on predicting the prognosis of patients based on REMS and the Worthing Phys- iological Assessment System (WPS) in the emergency depart- ment, both REMS and WPS had a good performance in pre- dicting death in critically ill patients. In similar studies, Plunkett et al. (14) and Nakhchivan et al. (7) showed that REMS is a better predictor of mortality at the time of admission of inpatients compared to RAPS. The re- sults of these studies are consistent with our study. REMS seems to be a powerful predictor of in-hospital mortality among patients referring to the emergency department with a wide range of common internal disorders. One of the lab- oratory scoring methods was described based on biochem- istry tests on admission, which identified high-risk patients for in-hospital death. Risk profiles on admission could be a means of improving outcomes (15). Identifying high-risk patients can be the basis for purposeful intervention after emergency medical care. This study confirms that the REMS has better discriminatory power than the RAPS regarding in- hospital mortality. Olson et al. used similar methods to cal- culate REMS and evaluate the predictive power of RAPS. They found that REMS was a better predictor of nosocomial mor- tality compared to RAPS (16). In our study, the accuracy of REMS in prediction of mortal- ity was 72.2%, this rate was 62.6% for RAPS based on the area under the ROC curve (AUROC). Sensitivity of REMS (72.2%) was higher than the sensitivity of RAPS (62.6%). The cut-off points for REMS and RAPS criteria were 6.5 and 2.5, respec- tively. According to the linear regression model, it can be concluded that age, MAP, HR, respiratory rate (RR), Spo2, and GCS (54.7%) could predict the REMS criterion. Olsson’s es- timation of AUROC for RAPS (0.65) was very similar to ours (0.56 - 0.65), though their estimation of AUROC for REMS (0.85) was significantly higher (0.68 - 0.75). Bahrmann et al. (17) found that all six components of REMS were associated with hospital mortality, while the relationship between mean arterial pressure and mortality was not significant in multi- variate analysis. Our findings and results clearly suggest that blood pressure is not a good predictor of mortality. In con- trast to our study, heart rate and respiration were indepen- dent predictors of mortality in their study. 5. Limitations The study was performed in a single center and thus, the results may not be generalizable. Further studies need to be carried out to validate the REMS and determine whether heart rate and respiratory rate are independent predictors of mortality. Ideally, risk classification tools for emergency care should be developed through pilot studies to identify all po- tentially useful variables and find variables having indepen- dent relationships with outcome, extract scores, and validate them in a different population. It should be noted that a risk classification tool that is useful for predicting death may not be useful for triage or clinical practice. Predicting mortality in triage or clinical practice may require distinguishing be- tween avoidable and unavoidable mortality. 6. Conclusion It seems that REMS is a better predictor of the 28-day mor- tality of non-trauma patients in an emergency department compared to RAPS. It should be noted that the total accuracy of REMS and RAPS in this regard was in good and poor range, respectively. 7. Declarations 7.1. Acknowledgments The researchers are grateful for the support of Urmia Univer- sity of Medical Sciences, the medical records staff of Imam Khomeini Hospital in Urmia, the Deputy Minister of Devel- opment and Technology, and all those who cooperated with us in doing this research. 7.2. Financial resources This study was financially supported by the Vice Chancellor for Research and Technology of Urmia University of Medical Sciences. 7.3. Authors’ contributions Omid Garkaz (first author), was a statistical analyst (10%), Farzin Rezazadeh (second author), author of the article (15%), Saeed Golfiroozi (third author) author of the intro- duction (%10), Sahar Paryab (Fourth author) author of the discussion (20%), Sadaf Nasiri, lead researcher/ (fifth au- thor) (10%), Hamid Reza Mehryar (sixth author), assistant re- searcher (25%), and Mousa Ghelichi-Ghojogh (seventh au- thor) author of methodology (%10). The last version of manuscript was read and approved by all authors. This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: http://journals.sbmu.ac.ir/aaem O. Garkaz et al. 4 7.4. Conflict of interest The authors declare that there is no conflict of interest re- garding the publication of this article. References 1. Oliveira VMd, Brauner JS, Rodrigues Filho E, Susin RG, Draghetti V, Bolzan ST, et al. Is SAPS 3 better than APACHE II at predicting mortality in critically ill trans- plant patients? Clin. Sci. 2013;68(2):153-8. 2. Villeneuve A, Joyal J-S, Proulx F, Ducruet T, Poitras N, Lacroix J. Multiple organ dysfunction syndrome in crit- ically ill children: clinical value of two lists of diagnostic criteria. Ann. Intensive Care. 2016;6(1):1-7. 3. Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med. 1985;13(10):818-29. 4. Garkaz O, Mehryar HR, Khalkhali HR, Salari Lak S. Fac- tors affecting the severity of traffic accident injuries; a cross-sectional study based on the Haddon matrix. Trauma Mon. 2020;25(1):52-8 5. Hyzy RC. ICU scoring and clinical decision making. Chest. 1995;107(6):1482-3. 6. Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II-A Severity of disease classification system: Reply. Crit Care Med. 1986;14(8):755-802. 7. Nakhjavan-Shahraki B, Baikpour M, Yousefifard M, Nikseresht ZS, Abiri S, Razaz JM, et al. Rapid acute phys- iology score versus rapid emergency medicine score in Trauma Outcome Prediction; a comparative study. Arch. Acad. Emerg. Med. 2017;5(1).1-8 8. Nakhjavan-Shahraki B, Yousefifard M, Faridaalaee G, Shahsavari K, Oraii A, Hajighanbari MJ, et al. Perfor- mance of physiology scoring systems in prediction of in- hospital mortality of traumatic children: A prospective observational study. J Clin Orthop Trauma. 2017;8(2):43- 8. 9. Imhoff BF, Thompson NJ, Hastings MA, Nazir N, Mon- cure M, Cannon CM. Rapid Emergency Medicine Score (REMS) in the trauma population: a retrospective study. BMJ open. 2014;4(5):e004738. 10. Toloui A, Neishaboori AM, Alavi SNR, Gubari MI, Khaneh AZS, Ghahfarokhi MK, et al. The Value of Physiological Scoring Criteria in Predicting the In-Hospital Mortality of Acute Patients; a Systematic Review and Meta-Analysis. Arch. Acad. Emerg. Med. 2021;9(1).e60 11. Hilderink MJ, Roest AA, Hermans M, Keulemans YC, Ste- houwer CD, Stassen PM. Predictive accuracy and feasi- bility of risk stratification scores for 28-day mortality of patients with sepsis in an emergency department. Eur J Emerg Med. 2015;22(5):331-7. 12. Seak C-J, Yen DH-T, Ng C-J, Wong Y-C, Hsu K-H, Seak JC- Y, et al. Rapid Emergency Medicine Score: A novel prog- nostic tool for predicting the outcomes of adult patients with hepatic portal venous gas in the emergency depart- ment. PloS one. 2017;12(9):e0184813. 13. Ha DT, Dang TQ, Tran NV, Vo NY, Nguyen ND, Nguyen TV. Prognostic performance of the Rapid Emergency Medicine Score (REMS) and Worthing Physiological Scoring system (WPS) in emergency department. Int J Emerg Med. 2015;8(1):1-8. 14. Plunkett PK, Byrne DG, Breslin T, Bennett K, Silke B. Increasing wait times predict increasing mortality for emergency medical admissions. Eur J Emerg Med. 2011;18(4):192-6. 15. Conway R, Byrne D, O’Riordan D, Silke B. Patient risk pro- filing in acute medicine: the way forward? QJM: Int. J. Medical Sci. 2015;108(9):689-96. 16. Olsson T, Terent A, Lind L. Charlson Comorbidity In- dex can add prognostic information to Rapid Emergency Medicine Score as a predictor of long-term mortality. Eur J Emerg Med. 2005;12(5):220-4. 17. Bahrmann A, Benner L, Christ M, Bertsch T, Sieber CC, Katus H, et al. The Charlson Comorbidity and Barthel In- dex predict length of hospital stay, mortality, cardiovas- cular mortality and rehospitalization in unselected older patients admitted to the emergency department. Aging Clin Exp Res. 2019;31(9):1233-42. This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: http://journals.sbmu.ac.ir/aaem 5 Archives of Academic Emergency Medicine. 2022; 10(1): e52 Figure 1: Area under the receiver operating characteristic (ROC) curve of Rapid Emergency Medicine Score (REMS) and Rapid Acute Physiol- ogy Score (RAPS) in predicting the risk of in-hospital mortality of non-trauma patients in emergency department (p = 0.001). Appendix 1: Rapid Emergency Medicine Score (REMS) Variable Score 0 1 2 3 4 5 6 Age (year) <45 - 45-54 55-64 - 65-74 >74 Emergency Score 70-109 - 110-129 50-69 130-159 >159 ≤49 - - Heart Rate (/minutes) 70-109 - 110-139 55-69 140-179 40-54 179 ≤39 - - Respiratory rate (/minutes) 12-24 25-34 10- 11 6-9 35-49 >49 ≤5 - - SpO2 (%) >89 86-89 - 75-85 <75 - - GCS 14 or 15 11-13 8-10 5-7 3 or 4 - - GCS: Glasgow Coma Scale; SaO2: oxygen saturation. Appendix 2: Rapid Emergency Medicine Score (RAPS) Variable Points +4 +3 +2 +1 0 +1 +2 +3 +4 MAP 160≥ 130-159 110-129 - 70-109 - 50-69 - 49≤ HR 180≥ 140-179 110-139 - 70-109 - 55-69 40-54 39≤ Resp* 50≥ 35-49 - 25-34 12-24 10-11 6-9 - 5≤ GCS - - - - 14≥ 11-13 10-8 5-7 4≤ MAP: Mean arterial pressure; HR: heart rate; Resp: respirations; GCS: Glasgow Coma Scale. Score of 0 is normal. *Spontaneous or, if not spontaneous, ventilated rate. This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: http://journals.sbmu.ac.ir/aaem O. Garkaz et al. 6 Table 1: Comparing the baseline characteristics as well as Rapid Emergency Medicine Score (REMS) and Rapid Acute Physiology Score (RAPS) of studied cases between survived and non-survived cases Variable Outcome P value Deceased Discharged Age (year) Mean ± SD 66.5±13.9 56.5 ± 22.2 < 0.001 Gender Male 427(70.2) 181(29.8) < 0.001 Female 330(83.6) 65(16.4) Vital Signs SBP (mmHg) 112.9 ± 31.4 125.5 ± 27.7 <0.001 DBP (mmHg) 69.5 ± 20.0 79.0 ± 20.8 <0.001 MAP (mmHg) 84.1 ± 22.8 94.7 ± 21.7 <0.001 HR (/minute) 101.1 ± 25.8 92.2 ± 22.2 <0.001 SaO2 (%) 91.9 ± 15.1 92.0 ± 7.3 <0.001 RR (/minute) 18.0 ± 2.9 21.1 ± 3.5 <0.001 GCS 8.1 ± 2.2 9.9 ± 2.2 <0.001 Predicting models REMS 8.7 ± 3.2 6.0 ± 3.6 <0.001 RAPS 3.7 ± 2.8 2.7 ± 2.0 <0.001 Data are presented as mean ± standard deviation (SD) or frequency (%). SBP: systolic blood pressure; DBP: Diastolic blood pressure; MAP: Mean arterial pressures; HR: Heart Rate; SaO2: oxygen saturation; RR: respiratory rate. GCS: Glasgow coma scale. Table 2: Screening performance characteristics of Rapid Emergency Medicine Score (REMS) (cut-off = 6.5) and Rapid Acute Physiology Score (RAPS) (cut-off = 2.5) in predicting the risk of in-hospital mortality among non-trauma patients Character REMS (95%CI) RAPS (95%CI) True positive 115 97 True Negative 680 601 False positive 188 244 False negative 20 61 Sensitivity 85.19(78.05-90.71) 61.39(53.33-69.02) Specificity 78.34(75.45-81.04) 71.12(67.94-74.16) Positive predictive value 37.95(34.61-41.42) 97.58(97.17-97.94) Negative predictive value 97.14(95.77-98.08) 8.84(7.35-10.60) Positive likelihood ratio 3.93(3.40-4.55) 2.13(1.81-2.50) Negative likelihood ratio 0.19(0.13-0.28) 0.54(0.44-0.66) Accuracy 79.26(70.37-84.25) 61.88(58.79-64.90) CI: confidence interval. This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: http://journals.sbmu.ac.ir/aaem Introduction Methods Results Discussion Limitations Conclusion Declarations References