Evidence Summary   Additional Search Strategies May Not Be Necessary for a Rapid Systematic Review   A Review of: Westphal, A., Kriston, L., Hölzel, L.P., Härter, M., & von Wolff, A. (2014). Efficiency and contribution of strategies for finding randomized controlled trials: a case study from a systematic review on therapeutic interventions of chronic depression. Journal of Public Health Research, 3(2), 177. doi: 10.4081/jphr.2014.177   Reviewed by: Joanne L. Jordan Research Information Manager Arthritis Research UK Primary Care Centre, Keele University Keele, Staffordshire, United Kingdom Email: j.jordan@keele.ac.uk   Received: 10 Mar. 2015  Accepted: 12 May 2015      2015 Jordan. This is an Open Access article distributed under the terms of the Creative Commons‐Attribution‐Noncommercial‐Share Alike License 4.0 International (http://creativecommons.org/licenses/by-nc-sa/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly attributed, not used for commercial purposes, and, if transformed, the resulting work is redistributed under the same or similar license to this one.   Abstract   Objective – To evaluate the efficiency and contribution of additional searching strategies for finding randomized controlled trials (RCTs) in a systematic review.   Design – A methodological case study.   Setting – Biomedical literature.   Methods – A sensitive search (defined as “the ratio of the number of relevant reports identified to the total number of relevant reports in existence”) was conducted of electronic databases, Cochrane CENTRAL database, MEDLINE, EMBASE, PsycInfo, CINAHL, BIOSIS, and Web of Science databases (Science and Social Science Citation Indexes). The following additional searching strategies were conducted: hand-searching contents of relevant journals (Archives of General Psychiatry, Journal of Consulting and Clinical Psychology, and Journal of Affective Disorders), citation tracking (forwards tracking using Social Science and Science Citation Index and backwards tracking by looking through reference lists of included studies), screening reference lists of relevant systematic reviews, searching clinical trials registers (ClinicalTrials.gov and ICTRP registers), and contacting first authors of included studies to find any similar unpublished studies.   The number of articles identified by each of these methods was recorded and screened for inclusion in the systematic review. The authors calculated what they labelled as the ‘efficiency’ of each searching strategy (the number of included studies identified by the search method as a proportion of the full text articles screened) and the ‘contribution’ of the search strategies (the ratio of included studies identified by that method to the final number of included studies in the systematic review). The methodological quality of each included study was assessed using the Cochrane Risk of Bias Tool, which is a critical appraisal checklist used to judge the study’s value in the systematic review. The meta-analysis in the systematic review was conducted with and without the studies identified by the additional searching strategies to assess their impact on the review’s findings.   Main Results – In total 50 studies were identified, 42 from electronic database searches and 8 from additional search strategies. As illustrated by the results in Table 1, the most useful additional search strategy was screening reference lists of relevant systematic reviews. Journal hand-searching and contacting authors also contributed to the review.   Of the eight studies identified by the additional search strategies none were judged to have a low risk of bias (four had high risk of bias and four were unclear). Of the 42 included studies from electronic searches only 11 were judged to have a low risk of bias, whereas 9 studies had a high risk of bias and 22 were unclear.   Excluding the eight studies retrieved from additional search strategies in the systematic review meta-analysis did not influence the results on the effectiveness of the different interventions for chronic depression. These studies were found to be indexed correctly on the electronic databases, but were not identified in the initial search.   Conclusion – Additional search strategies, especially screening reference lists of systematic reviews and hand-searching relevant journals, retrieved a substantial number of relevant studies for a systematic review of interventions for treating chronic depression. However, results of the review’s meta-analysis did not differ when these additional studies (rated as either high or unclear risk of bias) were not included and search methods were time consuming. It might be reasonable to rely on electronic searching strategies when resources for conducting a systematic review are limited or when doing a “rapid review.” The benefits and limitations of additional search strategies should be considered particularly when resources or time for conducting a systematic review are limited.     Table 1 The numbers of articles retrieved and included studies from each of the searching strategies. Search Strategy Search Results Full texts screened Included in review Efficiency Contribution Electronic database search 2417 276 42 9.8% 84% Journal hand-search 19076 33 2 6.1% 4% Forward citation tracking 2979 7 0 0% 0% Backward citation tracking 1692 15 0 0% 0% Review reference screening 1191 16 5 31.3% 10% Clinical trial registers 2053 0 0 0% 0% Contacting authors 16 11 1 9.1% 2%     If the electronic database search is sensitive and includes the Cochrane CENTRAL database additional search strategies may not be necessary, but these findings should be tested in other research areas.   Commentary   This study is highly relevant for information professionals who often conduct search strategies for systematic reviews. Systematic reviews frequently take 8 to 12 months to complete and studies evaluating rapid review methods that reduce production time without compromising rigour are needed.   The Cochrane CENTRAL database (The Cochrane Library) consists of RCTs retrieved from a variety of sources, including hand-searching journals and searches of trial registries. As the authors of the study rightly point out, by including CENTRAL in their electronic database searches, additional strategies are likely to be less influential. In this study, journals with high impact factors and those containing well-known potentially relevant studies were hand searched. The authors did not mention if they checked the list of journals already hand-searched for the CENTRAL database, which could have avoided the duplication of effort required in screening several thousand article titles and abstracts.   However, Cochrane review standards (Cochrane Editorial Unit, 2013) state it is mandatory for Cochrane reviews (the gold standard systematic reviews), in addition to searching CENTRAL, to search ClincalTrials.gov, WHO International Clinical Trials Registry Platform (ICTRP) trial registers, and screen reference lists of included studies and relevant systematic reviews. These standards ensure that searches for systematic reviews are extensive and sensitive. The authors found that studies indexed in the electronic databases were missed by their search strategies. By using a variety of methods systematic reviewers can be more confident about identifying all the relevant studies.   Studies found in this review by additional search strategies did not influence the results of the meta-analysis. However, as pointed out by the authors, where results of studies are sparse or conflicting, missing even the smaller studies will have a large impact and bias the results of the systematic review. This study needs to be repeated in a broad range of reviews to fully assess the impact of additional searching strategies. Including unpublished studies (grey literature), which were excluded in this study due to limited searching, also have the potential to affect systematic review results. Searching for grey literature requires different resources and their value needs to be assessed.   The authors of this study did not count studies that had already been included by a previous strategy. They comment that the order in which searching strategies were applied will have affected the level of contribution. Therefore the contribution of the searching strategies may be underestimated and the impact of this needs further exploration.   A criticism of the study is that the electronic search strategies are not sensitive enough and may be the reason eight studies were missed. The strategies do not include indexed terms (e.g., MeSH terms) that would help to identify relevant studies. The authors state they did not consult an expert librarian or information specialist when constructing their search strategies, which might have improved their search retrieval.   Reference   Cochrane Editorial Unit. (2013) Methodological Expectations of Cochrane Intervention Reviews (MECIR): Methodological standards for the conduct of new Cochrane Intervention Reviews. Version 2.3. Available from http://editorial-unit.cochrane.org/mecir