Emergency (****); * (*): *-* This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Copyright © 2014 Shahid Beheshti University of Medical Sciences. All rights reserved. Downloaded from: www.jemerg.com 158 Emergency (2014); 2 (4): 158-161 ORIGINAL RESEARCH Inter-Rater Agreement of Emergency Nurses and Physicians in Emergency Severity Index (ESI) Triage Mehrdad Esmailian, Majid Zamani*, Fatemeh Azadi, Faezeh Ghasemi Department of Emergency Medicine, Isfahan University of Medical Sciences, Isfahan, Iran Abstract Introduction: Triage is one of the most important systems in patients prioritizing at the time of arrival to hospi- tal. Based on the severity of the injury and the need for treatment, this system manages patients in the least time, which could lead to rotation of patients with high reliability and safety. Currently, the most accepted method for triage is emergency severity index (ESI) system, considered as five-level triage method, too. This method were implemented in Al Zahra Hospital of Isfahan by trained nurses since March to May 2010. This study was aimed to evaluate the accuracy of emergency nursing triage using ESI. Methods: This prospective cross sectional study was carried out on 601 patients referred to Al-Zahra hospital of Isfahan through May 2010. The patients’ triage level were determined by physicians and nurses separately and the results compared. To define the level of agreement between two groups (inter-rater agreement), the kappa index was evaluated. To specify the associa- tion between the time interval of initial triage and patient final status, Chi-Square test was applied using SPSS 18 statistical software. Results: There was no significant difference between results of nurses and physicians triage (P<0/0001). The agreement level (kappa index) between two groups was 94% (95% CI: 0.931-0.957). Of 601 patients, 44.1% ones were hospitalized at the emergency department, 52.6% discharged and 3.3% died. The av- erage of time interval between nursing triage and physician visit was 9.55 minutes at the level one triage, 21.64 minutes at level two, 26.03 minutes at level three, 26.93 minutes at level four, and 11.70 minutes at level five. Conclusion: It seems that there is an acceptable inter-rater agreement between emergency nurses and physi- cians regarding patients’ triage in terms of ESI system. Key words: Triage; nurses; physician-nurse relations; emergency medicine Cite this article as: Esmailian M, Zamani M, Azadi F, Ghasemi F. Inter-rater agreement of emergency nurses and physicians in emergency severity index (ESI) triage. Emergency. 2014;2(4):158-61. Introduction:1 he rate of emergency department (ED) visits is increasing and need for accurate and reliable tools is inevitable for classification of patients based on severity of emergency (1-5). Various methods have been designed for this purpose such as Traffic di- rector, Manchester system, Canadian system, spot – check, and comprehensive triage (6-11). Through these methods, a five-level severity index or emergency se- verity index (ESI) recognized as a valid and accurate system which is not only prioritize patients, but also follow their treatment process for better access to med- ical facilities and services (12, 13). In comparison with the three-level triage, this system has higher validity and reliability, and successfulness in use of financial, human and time resources (14, 15). ESI first was de- signed in United States of America in 1990 and now has *Corresponding Author: Majid Zamani; Al-Zahra Hospital, Soffeh Blvd, Isfahan, Iran. Tel: +989131585086; Fax: +983116685555. Email: mzamani93@yahoo.com Received: 13 May 2014; Accepted: 5 June 2014 been recognized as the gold standard of triage in many countries, such as Australia, Canada, and United King- dom (1, 12, 16-20). Because of simplicity, ease of learn- ing, operational and conceptual approach, this triage method seems to be the most appropriate system for our country (Iran), too. Since the knowledge and expe- rience of nurses are very important in accurate triage, several studies examined the role of their knowledge and awareness in prioritizing of patients (2, 21). ESI triage system was implemented by trained nurses through May 2010 in Alzahra Hospital of Isfahan. In the present study, the accuracy of ESI triage by emergency nurses was evaluated. Methods: Six hundred and one patients referred to the ED of Al- zahra hospital, Isfahan, Iran, were triaged based on ESI version four recommendation by physicians and trained nurses during March to April 2008. They tri- aged patients separately while were unaware to the results of each other. The study protocol was approved by the ethics committee of Medical University of Isfa- T mailto:mzamani93@yahoo.com This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Copyright © 2014 Shahid Beheshti University of Medical Sciences. All rights reserved. Downloaded from: www.jemerg.com 159 Esmailian et al Emergency (2014); 2 (3): ***-*** han. A checklist was fulfilled for each patients, con- tained demographic data (age, sex, date, and cause of admission), nurses and physicians triage results, time interval between the initial triage and first visit by phy- sicians, the outcome of the patient, and the time of dis- charge from the ED. Hospitalization, discharging from ED and mortality was considered as the outcomes. The sample size was calculated considering to p=0.50, d=0.1, and α=0.05 (n=96). All data were analyzed with chi-square and Pearson correlation tests using SPSS version 18 statistical software. P<0.05 was considered as significant. The weighted kappa index (κ) was used for assessment of Inter-rater reliability between the triage of nurse and physician (17).The weighted kappa with 95% confidence interval was reported. Kappa in- dex less than 0.2, 0.2-0.4, 0.6-0.8 and more than 0.8 were considered as week, moderate, good, and excel- lent, respectively (22, 23). Results: There was no significant difference between Nurses and physicians triage. Total calculated weighted Kappa was 94% (95% CI: 0.931-0.957; p<0.0001). Table 1 shows the percentage of agreement between two groups in different levels of triage. Of 601 patients, 316 (52.6%) were discharged from ED, 265 (44.1%) hospitalized, and 20 (3.3%) died (Table 2). The time interval be- tween triage and first physician visit was 0.0 minutes in level one, 12.64±5.0 in level two, 26.03± 9.6 in level three, 62.93±17.3 in level four, and 110.70 ±26.8 in lev- el five (Table 3 ). Discussion: Our finding revealed that nurses and physicians triage had more than 90% overlap at all levels of triage indi- cated the high accuracy of nursing triage. Several studies had been shown the accuracy of nurse triage decision. For example, Goransson et al demon- strated that registered nurses triage had only 58% agreement with the expected acuity rating. But in this study the wide range (22–89%) of accurate triage clear- ly was a big limitation (24). Abbasi et al., reported low reliability of nursing triage (21). In other hand, the re- sult was in line with Gorason et al., declared the high accuracy of nursing triage (25). The total agreement level of this study was acceptable based on recommen- dation of Australian college of emergency medicine, determined the kappa of 0.60 as the minimum accepta- Table 1: The number (%) of agreements between two groups in different levels of emergency severity index (ESI) triage  Total 5 4 3 2 1 ESI level 11 0 0 0 0 11 (100) 1 141 0 0 3 138 (95) 0 2 415 0 5 404 (98) 6 0 3 24 0 22 (81) 2 0 0 4 10 10 (100) 0 0 0 0 5 601 10 27 409 144 11 Total ESI: Emergency severity index Table 2: Final outcomes of patients with different levels of triage after 24 hours  Triage levels Outcome of patients (%) Hospitalized Discharged from ED Died 1 3 (27.3) 2 (18.2) 6 (54.5) 2 80 (55.6) 54 (37.5) 10 (6.9) 3 175 (42.8) 232 (56.7) 2 (0.5) 4 5 (18.5) 20 (74.1) 2 (7.4) 5 2 (20) 8 (80) 0 (0) Total 265 (44.1) 316 (52.6) 20 (3.3) ED: Emergency department Table 3: The average time interval between initial nursing triage and physician examination (minute)  ESI level N* (%) Intervals Standard** 1 (Resuscitation) 11 (100) 0 0 2 (Emergent) 138 (95) 12.64±5.0 10-15 3 (Urgent) 404 (98) 26.03± 9.6 30-60 4 (Non- Urgent) 22 (81) 62.93±17.3 60-120 5 (Referred) 10 (100) 110.70 ±26.8 120-240 *Numbers of agreements between two groups, ** Maximum time to point of care (based on Manchester triage system) This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Copyright © 2014 Shahid Beheshti University of Medical Sciences. All rights reserved. Downloaded from: www.jemerg.com Emergency (2014); 2 (4): 158-161 160 ble agreement level (2). Similar to our investigation Gorason et al. and Worster et al. achieved the moderate (0.46) and good (0.76) agreements, respectively (26, 27). Based on previous works there were not any significant relation among accuracy of nursing triage and personal characteristics, attitude, and experience level (24, 28). However, nurses’ knowledge has direct relation with appropriate triage of patients (29). It mandates the im- provement of staffs’ knowledge regarding the level of patients’ emergency severity; it could be led to mini- mize the waiting time, enhance the quality of services, and reduce mortalities (30). Training is a key factor in change and improvement of nurses’ knowledge. In this regards, several new training techniques can be used to teach the triage such as gaming technology, web-based training, and etc. (31) which improve the accuracy of the triage process. The prolonged time interval between initial nursing triage and physician examination lead to harmful delays in achieving timely emergency care (32). In contrast, shortening the duration of time be- tween patient presentation and treatment, may in- crease levels of patient satisfaction and reduced ED overcrowding (33). It could be concluded that trained nurses are able to perform appropriate triage of ED pa- tients and improve the patients’ safety and satisfaction, consequently. Conclusion: It seems that there is an acceptable inter-rater agree- ment between emergency nurses and physicians re- garding patients’ triage in terms of ESI system. Acknowledgments: The authors appreciate the insightful cooperation of staffs of the Emergency Department of Alzahra Hospital of Isfahan, Iran. Conflict of interest: None Funding support: None Authors’ contributions: All authors passed four criteria for authorship contribu- tion based on recommendations of the International Committee of Medical Journal Editors. References: 1. Fernandes CM, Tanabe P, Gilboy N, et al. Five-level triage: a report from the ACEP/ENA Five-level Triage Task Force. J Emerg Nurs. 2005;31(1):39-50. 2. Goransson K, Ehrenberg A, Marklund B, Ehnfors M. Accuracy and concordance of nurses in emergency department triage. Scand J Caring Sci. 2005;19(4):432-8. 3. Platts-Mills TF, Travers D, Biese K, et al. Accuracy of the Emergency Severity Index triage instrument for identifying elder emergency department patients receiving an immediate life-saving intervention. Acad Emerg Med. 2010;17(3):238-43. 4. Shojaee M, Faridaalaee G, Yousefifard M, et al. New scoring system for intra-abdominal injury diagnosis after blunt trauma. Chin J Traumatol. 2014;17(1):19-24. 5. Hashemi B, Baratloo A, Rahmati F, et al. Emergency Department Performance Indexes Before and After Establishment of Emergency Medicine. Emergency. 2013;1(1):20-3. 6. Roukema J, Steyerberg EW, van Meurs A, Ruige M, van der Lei J, Moll HA. Validity of the Manchester Triage System in paediatric emergency care. Emerg Med J. 2006;23(12):906- 10. 7. van Veen M, Steyerberg EW, Ruige M, et al. Manchester triage system in paediatric emergency care: prospective observational study. BMJ. 2008;337:a1501. 8. van Veen M, Steyerberg EW, Van't Klooster M, et al. The Manchester triage system: improvements for paediatric emergency care. Emerg Med J. 2012;29(8):654-9. 9. Dallaire C, Poitras J, Aubin K, Lavoie A, Moore L, Audet G. Interrater agreement of Canadian Emergency Department Triage and Acuity Scale scores assigned by base hospital and emergency department nurses. CJEM. 2010;12(1):45-9. 10. Fernandes CM, McLeod S, Krause J, et al. Reliability of the Canadian Triage and Acuity Scale: interrater and intrarater agreement from a community and an academic emergency department. CJEM. 2013;15(4):227-32. 11. Gravel J, Gouin S, Goldman RD, et al. The Canadian Triage and Acuity Scale for children: a prospective multicenter evaluation. Ann Emerg Med. 2012;60(1):71-7.e3. 12. Christ M, Grossmann F, Winter D, Bingisser R, Platz E. Modern triage in the emergency department. Dtsch Arztebl Int. 2010;107(50):892-8. 13. Manos D, Petrie DA, Beveridge RC, Walter S, Ducharme J. Inter-observer agreement using the Canadian Emergency Department Triage and Acuity Scale. CJEM. 2002;4(1):16-22. 14. Eitel DR, Travers DA, Rosenau AM, Gilboy N, Wuerz RC. The emergency severity index triage algorithm version 2 is reliable and valid. Acad Emerg Med. 2003;10(10):1070-80. 15. Lohr KN, Schroeder SA. A strategy for quality assurance in Medicare. N Engl J Med. 1990;322(10):707-12. 16. Shelton R. The Emergency Severity Index 5-level triage system. Dimens Crit Care Nurs. 2009;28(1):9-12. 17. Richardson JR, Braitberg G, Yeoh MJ. Multidisciplinary assessment at triage: a new way forward. Emerg Med Australas. 2004;16(1):41-6. 18. Storm-Versloot MN, Ubbink DT, Chin a Choi V, Luitse JS. Observer agreement of the Manchester Triage System and the Emergency Severity Index: a simulation study. Emerg Med J. 2009;26(8):556-60. 19. van der Wulp I, Schrijvers AJ, van Stel HF. Predicting admission and mortality with the Emergency Severity Index and the Manchester Triage System: a retrospective observational study. Emerg Med J. 2009;26(7):506-9. 20. R SR. the witching hour: overcrowded emergency departments7. . Emerg Med news. 1992 (21):40-1. 21. Abbasi E NA, Nabipour I, Emami SR. Assessment of the level of knowledge of Physicians in Bushehr Province about preparedness and response for nuclear emergency. ISMJ. 2005;7:183–9. [Persion]. 22. Esmailian M, Haj Zargarbashi E, Masoumi B, Karami M. Accuracy of Ultrasonography in Confirmation of Adequate Reduction of Distal Radius Fractures. Emergency. 2013;1(1): 7-10. 23. Heydari F, Esmailian M, Dehghanniri M. Diagnostic This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Copyright © 2014 Shahid Beheshti University of Medical Sciences. All rights reserved. Downloaded from: www.jemerg.com 161 Esmailian et al Emergency (2014); 2 (3): ***-*** Accuracy of Ultrasonography in the Initial Evaluation of Patients with Penetrating Chest Trauma. Emergency. 2014;2(2):77-80. 24. Göransson KE, Ehrenberg A, Marklund B, Ehnfors M. Emergency department triage: Is there a link between nurses’ personal characteristics and accuracy in triage decisions? Accid Emerg Nurs. 2006;14(2):83-8. 25. Goransson KE, von Rosen A. Interrater agreement: a comparison between two emergency department triage scales. Eur J Emerg Med. 2011;18(2):68-72. 26. Goransson KE, Ehrenberg A, Ehnfors M. Triage in emergency departments: national survey. J Clin Nurs. 2005;14(9):1067-74. 27. Worster A, Sardo A, Eva K, Fernandes CM, Upadhye S. Triage tool inter-rater reliability: a comparison of live versus paper case scenarios. J Emerg Nurs. 2007;33(4):319-23. 28. Martin A, Davidson CL, Panik A, Buckenmyer C, Delpais P, Ortiz M. An Examination of ESI Triage Scoring Accuracy in Relationship to ED Nursing Attitudes and Experience. J Emerg Nurs.[In Press]. 29. Aloyce R, Leshabari S, Brysiewicz P. Assessment of knowledge and skills of triage amongst nurses working in the emergency centres in Dar es Salaam, Tanzania. Afr J Emerg Med. 2014;4(1):14-8. 30. Vatnøy TK, Fossum M, Smith N, Slettebø Å. Triage assessment of registered nurses in the emergency department. Int Emerg Nurs. 2013;21(2):89-96. 31. Hosseini M, Bekry G, Mozaffari HR, et al. Effect of Educational Intervention on Oral Health Behaviour based on Health Belief Model in Female Secondary School Students of Paveh in 2011. Edu Res Med Sci J. 2014;2(3):2-9. 32. Göransson KE, Ehrenberg A, Ehnfors M. Triage in emergency departments: national survey. J Clin Nurs. 2005;14(9):1067-74. 33. Finamore SR, Turris SA. Shortening the wait: a strategy to reduce waiting times in the emergency department. J Emerg Nurs. 2009;35(6):509-14.