Special Feature 245Urology Journal Vol 6 No 4 Autumn 2009 Relevance of Levels of Evidence to the Urologist J Chandra Singh,1 Philipp Dahm2 Urol J. 2009;6:245-8. www.uj.unrc.ir Keywords: urology, evidence-based medicine, guidelines 1Department of Urology, Christian Medical College, Vellore, India 2Department of Urology, College of Medicine, University of Florida, Gainesville, Florida, USA Corresponding Author: J Chandra Singh, MD Department of Urology, Christian Medical College, Vellore 632 004, TN, India Tel: +91 416 228 2055 Fax: +91 416 223 2035 E-mail: chandrasingh@cmcvellore.ac.in Received March 2009 Accepted May 2009 INTRODUCTION While caring for patients, urologists have to make several clinical decisions. To counsel and give appropriate care for a single patient, various aspects including prevention, natural history, diagnosis, treatment options, prognosis and health economics may need to be addressed. For informed and shared decision making to arrive at the treatment plan, knowledge of the strongest evidence from literature on each of these aspects is essential. The best available clinical evidence is clinically relevant research, which may be from the basic sciences of medicine, but especially that derived from clinical research that is patient centered, that evaluates the accuracy and precision of diagnostic tests and prognostic markers, and the efficacy and safety of therapeutic, rehabilitative, and preventive regimens.(1) Evidence-based practice addresses each question or decision making individually based on the strongest evidence available on that particular aspect. One of the guiding principles of evidence- based medicine is the concept of a hierarchy of evidence. This refers to the fact that certain study designs are more likely than others to provide an unbiased result and represent the “truth.” Levels of evidence is a particular ranking system used to describe the strength of the results measured in a clinical trial or research study that is widely used.(2) The Table enlists the levels attributable to studies addressing various parameters. This level of evidence rating system, adapted from the orthopedic surgery literature(3) and the Center of Evidence Based Medicine,(4) was used to assess the type and levels of evidence found in the urological literature.(5) We found that only a small subset of studies published in the urological literature, approximately 1 in 7 studies, provided “high-quality evidence,” even if levels I and II combined are considered. One should note that a similar study design may be assigned different levels, depending upon the type of question addressed by the study. CLINICAL SCENARIOS Levels of Evidence A higher level of evidence is available for many pharmacological interventions, as multicenter randomized trials have been performed to prove efficacy. For example, in lower urinary tract symptoms related to benign prostatic hyperplasia, tamsulosin has been shown to give better improvement in symptom score and flow rate compared to placebo. (6) Furthermore, a dose of 0.8 mg has been shown to have significantly higher adverse effects without a proportionately higher benefit (Level I evidence). There Levels of Evidence in Urology—Singh and Dahm 246 Urology Journal Vol 6 No 4 Autumn 2009 is level I evidence demonstrating the superiority of intravesical immunotherapy with bacillus Calmette-Guerin in delaying tumor recurrence compared to trans-urethral resection alone.(7,8) Other closely related questions, for example, the evidence for intravesical bacillus Calmette- Guerin and progression of transitional cell carcinoma of the bladder is questionable.(9) The role of bacillus Calmette-Guerin in low-grade bladder cancer, comparison of bacillus Calmette- Guerin with other intravesical agents, intravesical bacillus Calmette-Guerrin versus transurethral resection alone therapeutics, etc, have been studied; however, the evidence is not similarly strong. (9) The role of bisphosphonates in advanced prostate cancer is another area that has been widely studied. One thousand nine hundred and fifty-five patients from 10 studies were included in a systematic review and meta-analysis. The study provided level I evidence for reduction of refractory bone pain and reduction of skeletal- related events in those with metastatic prostate cancer.(10) However, the evidence is not strong on choice of bisphosphonates, schedule, and cost- benefit implications. Surgical interventions with level I evidence for benefit or absence of it are limited.(11,12) It has been suggested that typically the surgeon has a personality and temperament that does not always lead to well-developed cooperation and team skills.(13) Furthermore, recruitment of surgical patients who fulfill the inclusion criteria and are amenable to randomization is another difficulty. The sample size of an adequately powered surgical trial may be large, necessitating recruitment from many centers, and with the plethora of operative choices, several challenges need to be overcome before executing a surgical trial that can provide level I evidence. Another issue specific to surgical interventions and urological devices is that of the learning curve. One proposed solution to this issue is expertise-based trials. In this study design for randomized controlled trials of surgical interventions, participants are randomized not only to a form of treatment (ie, robotic- assisted laparoscopic prostatectomy versus open retropubic prostatectomy), but also to an expert surgeon who is experienced in that technique.(14) LEVELS OF EVIDENCE IN UROLOgICAL LITERATURE Several authors have highlighted the critical need Level Therapy/Prevention/ Etiology/Harm Prognosis Diagnosis Economic Decision Analyses I ● RCT Significant difference No significant difference (narrow confidence interval) ● Systematic review† of Level I RCTs (homogenous) ● Prospective cohort study‡ ● Systematic reviews† Level I‡ ● Testing of previously developed diagnostic criteria in series of consecutive patients (with universally applied gold standard) ● Systemic review of Level I‡ ● Clinically sensible costs + alternatives, multi-way analysis; many studies used ● Systematic review Level I† II ● Prospective cohort§ ● Poor-quality RCT Limited follow-up, dropout ● Systematic review Level II Nonhomogenous Level I ● Retrospective cohort study║ ● Study of untreated controls from previous RCT ● Systematic review† Level II ● Development of diagnostic criteria on basis of consecutive patients (with universally applied gold standard) ● Systematic review Level II† ● Clinically sensible costs + alternative; limited studies used ● Systematic review Level II† III ● Case-control# ● Retrospective cohort║ ● Systematic review Level III … ● Study of nonconsecutive patients (no gold standard) ● Systematic review Level III† ● Limited alternatives + costs, poor estimates ● Systematic review Level III† IV ● Case series (no or historical controls) ● Case series ● Poor quality retrospective cohort study ● Case-control study ● Poor reference standard ● No sensitivity analyses Level of Evidence Rating System* *Adapted from the Center of Evidence-Based Medicine website, and the Journal of Bone and Joint Surgery. †A study of results from 2 or more previous studies. ‡All patients enrolled at the same point in the disease course (inception cohort) with greater than 80% follow-up. §Study was initiated after treatment was performed. ║Patients with a particular outcome were compared to those who did not have the outcome (surgeries that failed versus succeeded) and look back for exposure, etc. #Patients were compared with a control group of patients treated at the same time and institution. Levels of Evidence in Urology—Singh and Dahm Urology Journal Vol 6 No 4 Autumn 2009 247 for evidence-based guidelines in urology. (5,13) It was also noted that most articles, even in established urology journals, were retrospective case series without a control group, representing level IV evidence.(13) Several journals of other specialties have sought to raise awareness for this issue by providing a level of evidence rating with every published article. Journals such as the British Journal of Urology International have been regularly identifying the type of study and the level of evidence that a given study provides, which enables the readers to assign appropriate importance to the findings.(5) gRADE OF RECOMMENDATION AND LEVELS OF EVIDENCE Evidence-based clinical practice guidelines have been recognized as playing an important role in guiding clinical practice. They are also among the most well-known evidence-based resources that urologists are aware of and actually use.(15) Clinical practice guidelines have a special place in the organization of evidence as summaries that integrate the best available evidence for a full range of management options for a given disease.(16) To do so, they provide specific recommendations for typical index patients. The strength of such recommendations is inherently linked to the quality of available evidence that addresses a given specific question. Although several guidelines, such as those that are being developed by the European Association of Urology,(17) provide such an explicit link between the grade of recommendation and the quality of evidence, their underlying methodology has been recently drawn in to question based on research advances in guidelines methodology.(18) Specifically, the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) system for grading evidence and grading recommendations represents a major advance in guidelines methodology. The GRADE system is being increasingly used by major professional organizations including the World Health Organization.(19) An advantage of the GRADE system is that for any evidence that has been allocated a grade based on study design, not only it has explicit comprehensive criteria for downgrading and upgrading the quality of evidence ratings based on the study limitations, but also it considers other issues such as magnitude of the effect size, the underlying precision, and the relevance of the endpoint to the patient.(20) The American Urological Association has recently adopted a modified version of the GRADE system to develop their guidelines, which represents a major step forward. CONCLUSION Understanding the levels of evidence is an essential prerequisite for an evidence-based practice of urology, by allowing the reader to place a given clinical research study into context. Having unified validated levels of evidence and grade of recommendation facilitates translation of research findings to patient care. That being said, it is important to be aware of the second guiding principles of evidence-based practice, which is that “evidence alone is never enough” but needs to be integrated with an individual patient’s-specific circumstances, values, and preferences.(21) Going forward, an increasing number of evidence-based resources will be becoming available, including high-quality clinical practice guidelines, to guide an evidence-based practice of urology. CONFLICT OF INTEREST None declared. REFERENCES 1. Straus SE, Sackett DL. Using research findings in clinical practice. BMJ (Clinical research ed. 1998 Aug 1;317(7154):339-42. 2. National Cancer Institute [homepage on internet]. Levels of evidence [cited 2009 Feb 9]. Available from: http://www.cancer.gov/templates/db_alpha. aspx?CdrID=446533 3. wright Jg, Swiontkowski M, Heckman JD. Levels of evidence. The Journal of bone and joint surgery. 2006 Sep;88(9):1264. 4. Centre for Evidence-Based Medicine. Oxford Centre for Evidence-based Medicine - Levels of Evidence (March 2009) [cited 2009 Feb 9]. Available from: http:// www.cebm.net/?o=1025 5. Borawski KM, Norris RD, Fesperman SF, Vieweg J, Preminger gM, Dahm P. Levels of evidence in the urological literature. The Journal of urology. 2007 Oct;178(4 Pt 1):1429-33. 6. wilt TJ, Mac Donald R, Rutks I. Tamsulosin for benign prostatic hyperplasia. Cochrane database of systematic reviews (Online). 2003(1):CD002081. Levels of Evidence in Urology—Singh and Dahm 248 Urology Journal Vol 6 No 4 Autumn 2009 7. Shelley MD, wilt TJ, Court J, Coles B, Kynaston H, Mason MD. Intravesical bacillus Calmette-guerin is superior to mitomycin C in reducing tumour recurrence in high-risk superficial bladder cancer: a meta- analysis of randomized trials. BJU international. 2004 Mar;93(4):485-90. 8. Shelley MD, Court JB, Kynaston H, wilt TJ, Coles B, Mason M. Intravesical bacillus Calmette-guerin versus mitomycin C for Ta and T1 bladder cancer. Cochrane database of systematic reviews (Online). 2003(3):CD003231. 9. Hall MC, Chang SS, Dalbagni g, et al. guideline for the management of nonmuscle invasive bladder cancer (stages Ta, T1, and Tis): 2007 update. J Urol. 2007;178:2314-30. 10. Yuen KK, Shelley M, Sze wM, wilt T, Mason MD. Bisphosphonates for advanced prostate cancer. Cochrane database of systematic reviews (Online). 2006(4):CD006250. 11. Dall’Oglio MF, Srougi M, Antunes AA, Crippa A, Cury J. An improved technique for controlling bleeding during simple retropubic prostatectomy: a randomized controlled study. BJU international. 2006 Aug;98(2):384-7. 12. Aus g, Abrahamsson PA, Ahlgren g, Hugosson J, Lundberg S, Schain M, et al. Three-month neoadjuvant hormonal therapy before radical prostatectomy: a 7-year follow-up of a randomized controlled trial. BJU international. 2002 Oct;90(6): 561-6. 13. Mansson w. Evidence-based urology--a utopia? European urology. 2004 Aug;46(2):143-6. 14. Devereaux PJ, Bhandari M, Clarke M, Montori VM, Cook DJ, Yusuf S, et al. Need for expertise based randomised controlled trials. BMJ (Clinical research ed. 2005 Jan 8;330(7482):88. 15. Dahm P, Preminger gM, Scales CD, Jr., Fesperman SF, Yeung LL, Cohen MS. Evidence-based medicine training in residency: a survey of urology programme directors. BJU international. 2009 Feb;103(3):290-3. 16. Haynes RB, Hart LE. Evidence in context: one person’s poison is another’s acceptable risk. ACP journal club. 2008 Nov 18;149(5):2-3. 17. Fall M, Baranowski AP, Fowler CJ, Lepinard V, Malone-Lee Jg, Messelink EJ, et al. EAU guidelines on chronic pelvic pain. European urology. 2004 Dec;46(6):681-9. 18. Dahm P, Yeung LL, gallucci M, Simone g, Schunemann HJ. How to use a clinical practice guideline. The Journal of urology. 2009 Feb;181(2):472-9. 19. guyatt gH, Oxman AD, Kunz R, Vist gE, Falck-Ytter Y, Schunemann HJ. what is “quality of evidence” and why is it important to clinicians? BMJ (Clinical research ed. 2008 May 3;336(7651):995-8. 20. Dahm P, Kunz R, Schunemann H. Evidence-based clinical practice guidelines for prostate cancer: the need for a unified approach. Current opinion in urology. 2007 May;17(3):200-7. 21. Sackett DL, Rosenberg wM, gray JA, Haynes RB, Richardson wS. Evidence based medicine: what it is and what it isn’t. BMJ (Clinical research ed. 1996 Jan 13;312(7023):71-2.