Emergency. 2017; 5 (1): e22 OR I G I N A L RE S E A RC H The Accuracy of Emergency Physicians in Ultrasono- graphic Screening of Acute Appendicitis; a Cross Sectional Study Ebrahim Karimi1, Mohammad Aminianfar2, Keivan Zarafshani1, Arash Safaie3∗ 1. Emergency Department, Be’sat Hospital, Aja University of Medical Sciences, Tehran, Iran. 2. Department of Infectious and Tropical Diseases, Be’sat Hospital, Aja University of Medical Sciences, Tehran, Iran. 3. Emergency Department, Sina Hospital, Tehran University of Medical Sciences, Tehran, Iran. Received: June 2016; Accepted: August 2016; Published online: 10 January 2017 Abstract: Introduction: Diagnostic values reported for ultrasonographic screening of acute appendicitis vary widely and are dependent on the operator’s skill, patient’s gender, weight, etc. The present study aimed to evaluate the ef- fect of operator skill on the diagnostic accuracy of ultrasonography in detection of appendicitis by comparing the results of ultrasonography done by radiologists and emergency physicians. Methods: This prospective di- agnostic accuracy was carried out on patients suspected to acute appendicitis presenting to EDs of 2 hospitals. After the initial clinical examinations, all the patients underwent ultrasonography for appendicitis by emergency physician and radiologist, respectively. The final diagnosis of appendicitis was based on either pathology report or 48-hour follow-up. Screening performance characteristics of appendix ultrasonography by emergency physi- cian and radiologist were compared using STATA 11.0 software. Results: 108 patients with the mean age of 23.91 ± 7.46 years were studied (61.1% male). Appendicitis was confirmed for 37 (34.26%) cases. Cohen’s kappa coef- ficient between ultrasonography by the radiologist and emergency physician in diagnosis of acute appendicitis was 0.51 (95% CI: 0.35 – 0.76). Area under the ROC curve of ultrasonography in appendicitis diagnosis was 0.78 (95% CI: 0.69 – 0.86) for emergency physician and 0.88 (95% CI: 0.81 – 0.94) for radiologist (p = 0.052). Sensi- tivity and specificity of ultrasonography by radiologist and emergency physician in appendicitis diagnosis were 83.87% (95% CI: 67.32 – 93.23), 91.5% (95% CI: 81.89 – 96.52), 72.97% (95% CI: 55.61 – 85.63), and 83.10% (95% CI: 71.94 – 90.59), respectively. Conclusion: Findings of the present study showed that the diagnostic accuracy of ultrasonography carried out by radiologist (89%) is a little better compared to that of emergency physician (80%) in diagnosis of appendicitis, but none are excellent. Keywords: Appendicitis; ultrasonography; decision support techniques; emergency service, hospital; diagnosis © Copyright (2017) Shahid Beheshti University of Medical Sciences Cite this article as: Karimi E, Aminianfar M, Zarafshani K, Safaie A. The Accuracy of Emergency Physicians in Ultrasonographic Screening of Acute Appendicitis; a Cross Sectional Study. Emergency. 2017; 5 (1): e22. 1. Introduction Appendicitis is a common surgical emergency in young adult males presenting to emergency department (ED) following abdominal pain (1, 2). Diagnosis of appendicitis is a chal- lenge for the medical team, as most of the time its classic signs and symptoms, such as pain, are not present and ∗Corresponding Author: Corresponding author: Arash Safaie; Departmnet of Emergency Medicine, Sina Hospital, Tehran University of Medical Sciences, Tehran, Iran. Tel: +989111366966 E-mail: dr.safaie@yahoo.com laboratory tests do not have enough predictive value in this regard (3). Meanwhile, rapid diagnosis and timely treatment can improve these patients’ management and reduce their hospital stay (4). Laparotomy is the gold standard tool in diagnosis and treatment of appendicitis, but is invasive and has its own limitations and dangers. This has led the researchers to evaluate the diagnostic accuracy of other diagnostic tools such as computed tomography (CT) scan, magnetic resonance imaging (MRI), and ultrasonography in detection of appendicitis. Using each of these tests has its own advantages and limitations and the diagnostic values reported for them varies between different studies (5, 6). This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: www.jemerg.com E. Karimi et al. 2 This might be the reason that unnecessary laparotomies still have a high prevalence in abdominal pains suspected to be appendicitis (3). Currently, bedside ultrasonography is deemed one of the most valuable screening tests in ED (7-11). This rapid diagnostic method can provide valuable results for the medical team in a short time with minimum cost (12). However, using ultrasonography for this purpose is still under debate. Diagnostic values reported for ultrasonog- raphy vary widely and are dependent on the operator’s skill, patient’s gender, weight, etc. (13). Yet, there is still no correct understanding of the mentioned factors on the diagnostic accuracy of ultrasonography. Therefore, the present study was done aiming to evaluate the effect of operator skill on the diagnostic accuracy of ultrasonography in detection of appendicitis by comparing the results of ultrasonography done by radiologists and emergency physicians. 2. Methods 2.1. Study design This prospective diagnostic accuracy was carried out on pa- tients suspected to acute appendicitis presenting to EDs of Be’sat and Sina Hospitals, Tehran, Iran, during 2014 and 2015 with the aim of comparing the ultrasonographic screening results of patients by emergency physicians and radiologists. Protocol of the present study was approved by the Ethics Committee of Aja University of Medical Sciences. Before in- clusion in the study, written informed consent for participa- tion was obtained from all the patients or their relatives. The researchers adhered to the principles of the Declaration of Helsinki throughout the study. 2.2. Participants Samples were chosen by probability convenience method. Patients with pain in the right lower quadrant of the ab- domen suspected with acute appendicitis, who visited EDs of the 2 hospitals were included. Exclusion criteria consisted of acute abdomen in need of emergency laparotomy, con- firmed diagnosis other than appendicitis (like ureteral stone), not undergoing ultrasonography by radiologist, emergency physician not being blinded to the result of the ultrasonog- raphy by radiologist, unavailability of laparotomy data or not being able to follow the patient, and discharge against medi- cal advice. 2.3. Procedure After the initial clinical examinations, all the patients under- went ultrasonography for appendicitis by emergency physi- cian and radiologist, respectively. The final diagnosis of ap- pendicitis was based on pathology report (in patients un- dergoing surgery) or 48-hour follow-up (reference test). Ul- trasonography was done using an ultrasonography machine (HS2000, Honda, Korea) with a linear probe and 5 – 7.5 MHz frequency. Ultrasonographic diagnosis of acute appendici- tis (figure 1) was based on > 6mm outer diameter of the appendix, not being compressible, presence of appendicol- itis, loss of bowel movements, and free fluid accumulation around the appendix (14, 15). The site of probe placement in the right lower quadrant was where the most tenderness was found in the clinical examination and to reduce bowel gases and the distance of the probe to appendix, a gentle con- tinuous pressure was applied to the site before carrying out ultrasonography. Figure 2 shows the method of doing ultra- sonography in patients. Then the patients were followed. If the patient was sent to the operation room, their pathology result was counted as the confirmed diagnosis of presence or absence of appendicitis. However, the final diagnosis of patients who did not undergo laparotomy and were only fol- lowed for at least 48 hours was determined based on their follow-up and with the help of other diagnostic tests such as CT scan. The diagnostic and treatment process of the patients was done without considering ultrasonography re- sults. In addition, in patients who were discharged from ED, 48 hours after discharge, follow-up was done on the phone to evaluate the persistence or improvement of the symptoms and those who still had pain were invited to ED for further evaluation. These cases were followed and their final diagno- sis was done based on laparotomy or further 48 hour follow up. 2.4. Statistical Analysis Minimum sample size needed for the present study was cal- culated to be 106 patients by considering 98.5% specificity of ultrasonography, 39.4% prevalence of appendicitis (12), α = 0.05 and d = 0.05. Data analysis was done using STATA 11.0 software. Area under the receiver operating character- istic (ROC) curve, sensitivity, specificity, positive and nega- tive predictive value, positive and negative likelihood ratio and finally, Brier score of the emergency physician and ra- diologist for acute appendicitis diagnosis were compared. To evaluate the agreement between the results of the radiolo- gist and emergency physician, Cohen’s kappa coefficient was calculated. Presence of difference between the results of the emergency physician and radiologist was assessed using Mc- Nemar’s chi square test. Diagnostic value of the test was con- sidered excellent if between 90-100%, good if 80 - 90%, fair if 70 – 80%, poor if 60 – 70%, and fail if 50 – 60%. Significance level was considered p < 0.05. This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: www.jemerg.com 3 Emergency. 2017; 5 (1): e22 Figure 1: Sonographic views of appendix. Figure 2: Location of probe for appendix ultrasonography. 3. Results: 108 patients with the mean age of 23.91 ± 7.46 years were studied (61.1% male). Finally, based on the reference test ap- pendicitis was confirmed for 37 (34.26%) cases. Using ultra- sonography, emergency physician and radiologist were able to diagnose 27 and 31 cases out of the 37, respectively. Co- hen’s kappa coefficient between ultrasonography by the ra- diologist and emergency physician in diagnosis of acute ap- pendicitis was 0.51 (95% confidence interval (CI): 0.35 – 0.76). Area under the ROC curve of ultrasonography in appendicitis diagnosis was 0.78 (95% CI: 0.69 – 0.86) for emergency physi- cian and 0.88 (95% CI: 0.81 – 0.94) for radiologist (figure 1). Although it seems that area under the curve for radiologist is higher than emergency physician, the difference is only bor- derline (p = 0.052). Area under the ROC curve of ultrasonog- raphy performed by radiologist in men (AUC = 0.86; 95% CI: 0.77 – 0.95) and women (AUC = 0.89; 95% CI: 0.77 – 1.0) was not different (p = 0.68). These rates were (AUC = 0.79; 95% CI: 0.69 – 0.89) for men and (AUC = 0.77; 95% CI: 0.61 – 0.92) for women in ultrasonography performed by emergency physi- cian, which did not show a difference (p = 0.80) (figure 2). Di- agnostic value of ultrasonography by radiologist and emer- gency physician are reported in table 1. Sensitivity and speci- ficity of ultrasonography by radiologist in appendicitis diag- nosis were 83.87% and 91.5%, respectively. These values for emergency physician were 72.97% and 83.10%, respectively. This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: www.jemerg.com E. Karimi et al. 4 Table 1: Screening performance characteristics of ultrasonography performed by radiologist and emergency physician in detection of acute appendicitis Value Radiologist Emergency physician True positive 31 27 True negative 65 59 False positive 6 12 False negative 6 10 Sensitivity 83.78 (67.32-93.23) 72.97 (55.61-85.63) Specificity 91.5 (81.89-96.52) 83.10 (71.94-90.59) Positive predictive value 83.78 (67.32-93.23) 69.23 (52.27-82.45) Negative predictive value 91.55 (81.89-96.52) 85.51 (74.49-92.46) Positive likelihood ratio 9.91 (4.55-21-60) 4.32 (2.49-7-50) Negative likelihood ratio 0.18 (0.08-0.37) 0.32 (0.19-0.56) Accuracy 88.89 (82.96-94.48) 79.63 (72.03-87.23) Figure 3: Comparison of area under the receiver operating characteristic (ROC) curve for ultrasonography carried out by radiologist and emergency physician.âĂČ The accuracies calculated were about 89% and 80%, respec- tively. Brier score of the ultrasonography performed by ra- diologist was 0.11 and its scaled reliability was 0.01. These values were 0.20 and 0.05 for ultrasonography performed by emergency physician. These findings indicate the good pre- dictive accuracy and reliability for both specialists in diagno- sis of acute appendicitis using ultrasonography. This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: www.jemerg.com 5 Emergency. 2017; 5 (1): e22 Figure 4: Comparing area under the receiver operating characteristic (ROC) curve of ultrasonography carried out by radiologist and emer- gency physician for diagnosing acute appendicitis based on patients’ gender. 4. Discussion: Findings of the present study showed that although di- agnostic value of ultrasonography by radiologist is a little better than that of emergency physician, none are excellent. Reported diagnostic value of ultrasonography in appendici- tis diagnosis varies between studies (16, 17). The results of a meta-analysis showed that sensitivity and specificity of ultrasonography in appendicitis diagnosis are 86% and 81%, respectively. Positive and negative predictive values reported in the study were 84% (46-95%) and 85% (60 – 97%), respectively. The researchers of the meta-analysis believe that a variety of factors are responsible for the difference between studies. The most important reported factor was the dependence of ultrasonography on operator’s skill (18). Although the findings of the present study confirms this hypothesis to some extent, the diagnostic value of ultra- sonography by radiologist was not significantly different from that of emergency physician and the difference was on the borderline. In addition to the dependence of ultrasonog- raphy on the operator’s skill, some studies believed that the diagnostic value of ultrasonography is also dependent on the patient’s gender. These studies express that due to anatomic differences, differentiation of acute abdominal pains is very difficult in women of childbearing age (19-21). For this purpose, in the present study the diagnostic value of ultrasonography was evaluated based on patients’ gender. However, the findings showed that the diagnostic value of ultrasonography for appendicitis diagnosis does not vary between women and men. Some studies have attempted to increase the sensitivity of ultrasonography by adding other diagnostic tests. For example Aspelund et al. added MRI and showed that in children suspected to appendicitis, a radiation-free diagnostic protocol screening the suspected cases, using ultrasonography and MRI, was a simple method and had a value equal to CT scan (22). In contrast, other studies have questioned the combined strategy for diagnosis of appendicitis. In a study, Leeuwenburgh et al. showed that although screening based on ultrasonography and CT scan for suspected patients has equal value to MRI, both methods classify about half of those with ruptured appendicitis as healthy. These researchers concluded that triage of appendicitis based on imaging is not appropriate for conservative treatment and may cause huge mistakes in patient management (23) since false positive results could lead to an increase in unnecessary appendectomy, while false negative results might cause a delay in treatment and therefore, worsening of the patient’s condition. To solve this problem, the researchers suggest to not solely rely on imag- ing evaluations for diagnosis of appendicitis. Rather, use a mixture of diagnostic techniques including history taking and clinical examination, scoring systems, inflammatory biomarkers, in addition to imaging studies (18, 24). 5. Limitations: Among the limitations of the present study is its observa- tional nature .Therefore, eliminating all the confounding fac- tors and probable biases from the study was not possible. Among these items are convenience sampling that makes se- lection bias probable. In addition, the present study is a 2- centered one and this makes generalizability of the data to all clinical conditions a bit hard. Finally, not evaluating weight This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: www.jemerg.com E. Karimi et al. 6 and body mass index (BMI), which can affect the findings, is another limitation of this study. 6. Conclusion: Findings of the present study showed that the diagnostic ac- curacy of ultrasonography carried out by radiologist (89%) is a little better compared to that of emergency physician (80%) in diagnosis of appendicitis, but none are excellent. 7. Appendix 7.1. Acknowledgements The authors wish to thank all Emergency Department staff of Sina and Be’sat Hospitals for their contribution. This article is extracted from the residency thesis of Keivan Zarafshan. 7.2. Authors contribution All authors passed four criteria for authorship contribution based on recommendations of the International Committee of Medical Journal Editors. 7.3. Conflict of interest None. 7.4. Funding None. References 1. Humes D, Simpson J. Acute appendicitis. Bmj. 2006;333(7567):530-4. 2. Buckius MT, McGrath B, Monk J, Grim R, Bell T, Ahuja V. Changing epidemiology of acute appendicitis in the United States: study period 1993–2008. Journal of Surgi- cal Research. 2012;175(2):185-90. 3. Kollar D, McCartan D, Bourke M, Cross K, Dowdall J. Predicting acute appendicitis A comparison of the Al- varado score, the appendicitis inflammatory response score and clinical assessment. World journal of surgery. 2015;39(1):104-9. 4. Bower RJ, Bell MJ, Ternberg JL. Controversial aspects of appendicitis management in children. Archives of Surgery. 1981;116(7):885-7. 5. Livingston EH, Woodward WA, Sarosi GA, Haley RW. Dis- connect between incidence of nonperforated and perfo- rated appendicitis: implications for pathophysiology and management. Annals of surgery. 2007;245(6):886-92. 6. Andersson M, Kolodziej B, Andersson R. Structured Man- agement of Patients with Suspected Acute Appendicitis Using a Clinical Score and Selective Imaging (STRAPP- SCORE). 2015. 7. Yousefifard M, Baikpour M, Ghelichkhani P, Asady H, Nia KS, Jafari AM, et al. Screening Performance Char- acteristic of Ultrasonography and Radiography in De- tection of Pleural Effusion; a Meta-Analysis. Emergency. 2016;4(1):1. 8. Yousefifard M, Baikpour M, Ghelichkhani P, Asady H, Darafarin A, Esfahani MRA, et al. Comparison of Ultra- sonography and Radiography in Detection of Thoracic Bone Fractures; a Systematic Review and Meta-Analysis. Emergency. 2016;4(2):55. 9. Rahimi-Movaghar V, Yousefifard M, Ghelichkhani P, Baikpour M, Tafakhori A, Asady H, et al. Application of Ultrasonography and Radiography in Detection of Hemothorax: a Systematic Review and Meta-Analysis. EMERGENCY-An Academic Emergency Medicine Jour- nal. 2016;4(3):116-26. 10. Hosseini M, Ghelichkhani P, Baikpour M, Tafakhori A, Asady H, Ghanbari MJH, et al. Diagnostic Accuracy of Ultrasonography and Radiography in Detection of Pulmonary Contusion; a Systematic Review and Meta- Analysis. Emergency. 2015;3(4):127. 11. Ebrahimi A, Yousefifard M, Kazemi HM, Rasouli HR, Asady H, Jafari AM, et al. Diagnostic accuracy of chest ultrasonography versus chest radiography for identifica- tion of pneumothorax: a systematic review and meta- analysis. Tanaffos. 2014;13(4):29-40. 12. Mallin M, Craven P, Ockerse P, Steenblik J, Forbes B, Boehm K, et al. Diagnosis of appendicitis by bedside ul- trasound in the ED. The American journal of emergency medicine. 2015;33(3):430-2. 13. Kaewlai R, Lertlumsakulsub W, Srichareon P. Body mass index, pain score and Alvarado score are useful predic- tors of appendix visualization at ultrasound in adults. Ul- trasound in medicine & biology. 2015;41(6):1605-11. 14. Pickuth D, Heywang-Kobrunner SH, Spielmann RP. Sus- pected acute appendicitis: is ultrasonography or com- puted tomography the preferred imaging technique? Eu- ropean Journal of Surgery. 2000;166(4):315-9. 15. Paulson EK, Kalady MF, Pappas TN. Suspected appendicitis. New England Journal of Medicine. 2003;348(3):236-42. 16. Elikashvili I, Tay ET, Tsung JW. The Effect of Point-of-care Ultrasonography on Emergency Department Length of Stay and Computed Tomography Utilization in Children With Suspected Appendicitis Efecto de la Ecografia en el Punto de Atencion en la Duracion de la Estancia en el Servicio de Urgencias y en la Utilizacion de la Tomografia Computarizada en los Ninos con Sospecha de Apendici- tis. Academic Emergency Medicine. 2014;21(2):163-70. 17. Pinto F, Pinto A, Russo A, Coppolino F, Bracale R, Fonio P, et al. Accuracy of ultrasonography in the diagnosis of acute appendicitis in adult patients: review of the litera- This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: www.jemerg.com 7 Emergency. 2017; 5 (1): e22 ture. Critical Ultrasound Journal. 2013;5(1):1-3. 18. Yu S-H, Kim C-B, Park JW, Kim MS, Radosevich DM. Ul- trasonography in the Diagnosis of Appendicitis: Eval- uation by Meta-analysis. Korean Journal of Radiology. 2005;6(4):267-77. 19. Angelelli G, Moschetta M, Sabato L, Morella M, Scarda- pane A, Ianora AAS. Value of “protruding lips” sign in ma- lignant bowel obstructions. European journal of radiol- ogy. 2011;80(3):681-5. 20. Lorusso F, Fonio P, Scardapane A, Giganti M, Rubini G, Ferrante A, et al. Gatrointestinal imaging with mul- tidetector CT and MRI. Recenti progressi in medicina. 2012;103(11):493. 21. Bendeck SE, Nino-Murcia M, Berry GJ, Jeffrey Jr RB. Imaging for Suspected Appendicitis: Negative Appendectomy and Perforation Rates 1. Radiology. 2002;225(1):131-6. 22. Aspelund G, Fingeret A, Gross E, Kessler D, Keung C, Thirumoorthi A, et al. Ultrasonography/MRI Versus CT for Diagnosing Appendicitis. Pediatrics. 2014;133(4):586- 93. 23. Leeuwenburgh MMN, Wiezer MJ, Wiarda BM, Bouma WH, Phoa SSKS, Stockmann HBAC, et al. Accuracy of MRI compared with ultrasound imaging and selective use of CT to discriminate simple from perforated appendicitis. British Journal of Surgery. 2014;101(1):e147-e55. 24. Nasiri S, Mohebbi F, Sodagari N, Hedayat A. Diagnos- tic values of ultrasound and the Modified Alvarado Scor- ing System in acute appendicitis. International journal of emergency medicine. 2012;5(1):1. This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: www.jemerg.com Introduction Methods Results: Discussion: Limitations: Conclusion: Appendix References