1 Publication Bias in Urology Systematic Reviews and Meta-Analyses Amirmahdi Khayyamfar, MD. Student Research Committee, School of Medicine, Iran University of Medical Sciences, Tehran, Iran Sepehr Khosravi, MD. Student Research Committee, School of Medicine, Iran University of Medical Sciences, Tehran, Iran Robab Maghsoudi, MD. Department of Urology, Firoozgar Hospital, School of Medicine, Iran University of Medical Sciences Behnam Shakiba, MD. Department of Urology, Firoozgar Hospital, School of Medicine, Iran University of Medical Sciences Firoozgar Clinical Research Development Center, Iran University of Medical Sciences, Tehran, Iran. Corresponding Author: Behnam Shakiba, MD. behkiba@gmail.com, shakiba.b@iums.ac.ir Address: Urology Department, Firoozgar Hospital, Valadi St., Valiasr Ave., Tehran, Iran. Potal code: 1593748711 Tel: +98-8214-1303 2 Keywords Publication bias; systematic review; meta analysis; Urology Competing Interests: The authors declare no conflict of interest in this study. Funding: Present study was a medical doctoral dissertation and was supported by Firoozgar Clinical Research Development Center, Iran University of Medical Sciences, Tehran, Iran. (grant number: 99-1-4-17684) 3 Publication bias is one of the most important biases in systematic reviews and meta-analyses. This bias occurs when the results of an article affect its publication, in other words positive or significant findings are more likely to be published than the other probable results (1), as a result, statistically non-significant data from unpublished articles are missed out from systematic reviews and meta-analyses which can exaggerate the effect of some variables and may suggest useless or even harmful interventions and inappropriate clinical decision making (3, 4). Previous studies have shown that publication bias has been a matter of concern in the meta- analysis and systematic reviews conducted in some medical fields (5-7). Accordingly, we decided to assess the status of publication bias evaluation in systematic reviews and meta- analyses published in high impact urology journals. This study was performed on systematic reviews and meta-analysis published in four top urology journals based on their impact factor (European Urology, The Journal of Urology, BJU International and Prostate Cancer and Prostatic Diseases) in the last 5 years (search date: 2021/4/1). In the screening phase, two authors independently screened systematic reviews and meta-analysis with more than ten included studies for inclusion. After the identification of all systematic reviews and meta-analysis, 200 papers were randomly included using the random numbers generator. Two authors extracted information independently from each paper. A total of 200 papers were included in the present study. Of these, 81 (40.5%) articles were published in European Urology, 53 (26.3%) in The Journal of Urology, 45 (22.5%) in BJU International and 21 (10.5%) in Prostate Cancer and Prostatic Diseases. From the 200 included studies only 65 (32.5%) evaluated the publication bias in the review process and 31 reviews had reported publication bias in their study. According to Google scholar, the mean rank of citations was 98.17 in articles which had mentioned publication bias and 102.97 in the rest of them, which shows no significant relationship between citation number and publication bias reporting (p=0.558). Assessment of publication bias in the 5 years of study duration shows no meaningful difference between the studied years (p= 0.686). Researchers used Funnel plot, Egger’s test, Begg’s test and Trim and fill method for publication bias assessment in the included systematic reviews and meta-analysis. 4 Visual inspection of a funnel plot was the most frequent method used for evaluation of publication bias (61 from 65, 93.85%); this method was used alone in 34 articles and in combination with other methods in 27 papers. The present study confirms that publication bias was formally evaluated in a small number of reviews and meta-analysis published in urology journals. Therefore, this may be a risk factor that could decrease the robustness of outcomes and results of these studies. It seems that there is an essential need for authors, reviewers, and editors to pay better attention to evaluation of publication bias besides reporting it based on the aforementioned reporting guidelines. Nevertheless, further studies are strongly recommended to better evaluate the efficacy of publication bias evaluation on the quality of systematic reviews published in urology journals. The present study is one of the few studies that have assessed the reporting of publication bias evaluation in systematic reviews and meta-analysis and the first study to evaluate high impact urology journals; yet there are certain limitations that must be acknowledged. First, we limited our investigation to systematic reviews and meta-analysis published in four top ranked urology journals. Therefore, results from the present study need to be interpreted in light of the selected sample and cannot be generalized to all urology journals. Second, we included high impact urology journals for evaluation of high quality systematic reviews, yet we did not assess the quality of systematic reviews and meta-analysis directly. Third, this study only considered the evaluation of publication bias and did not assess the existence of publication bias in studies which had not assessed this kind of bias themselves. References: 1. Nissen SB, Magidson T, Gross K, Bergstrom CT. Publication bias and the canonization of false facts. eLife. 2016 Dec 20;5:e21451. 2. Schneck A. Examining publication bias—a simulation-based evaluation of statistical tests on publication bias. PeerJ. 2017 Nov 30;5:e4115. 3. Onishi A, Furukawa TA. Publication bias is underreported in systematic reviews published in high-impact-factor journals: metaepidemiologic study. J Clin Epidemiol. 2014 Dec;67(12):1320–6. 5 4. 5. Evaluation method Single test In combination with other tests total Funnel plot 34 27 61 Egger’s test 2 26 28 Begg’s test 1 10 11 Trim and fill method 1 0 1 6. Table 1. The publication bias tests used for publication bias assessment in the included systematic reviews and meta-analysis. 7. 8.