Evidence Based Library and Information Practice Evidence Based Library and Information Practice 2011, 6.3 28 Evidence Based Library and Information Practice Article Development and Testing of a Literature Search Protocol for Evidence Based Nursing: An Applied Student Learning Experience Andy HicknerAdministrative Specialist Patient Safety Enhancement Program University of Michigan Medical School Ann Arbor, Michigan, United States of America Email: hicknera@med.umich.edu Christopher R. Friese Assistant Professor University of Michigan School of Nursing Ann Arbor, Michigan, United States of America Email: cfriese@umich.edu Margaret Irwin Research Associate Oncology Nursing Society Pittsburgh, Pennsylvania, United States of America Email: mirwin@ons.org Received: 29 Mar. 2011 Accepted: 20 July 2011 2011 Hickner, Friese, and Irwin. This is an Open Access article distributed under the terms of the Creative Commons-Attribution-Noncommercial-Share Alike License 2.5 Canada (http://creativecommons.org/licenses/by- nc-sa/2.5/ca/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly attributed, not used for commercial purposes, and, if transformed, the resulting work is redistributed under the same or similar license to this one. Abstract Objective – The study aimed to develop a search protocol and evaluate reviewers' satisfaction with an evidence-based practice (EBP) review by embedding a library science student in the process. Methods – The student was embedded in one of four review teams overseen by a professional organization for oncology nurses (ONS). A literature search protocol was developed by the student following discussion and feedback from the review team. Organization staff provided process feedback. Reviewers from both case and control groups completed a questionnaire to assess satisfaction with the literature search phases of the review process. mailto:hicknera@med.umich.edu� mailto:cfriese@umich.edu� mailto:mirwin@ons.org� Evidence Based Library and Information Practice 2011, 6.3 29 Results – A protocol was developed and refined for use by future review teams. The collaboration and the resulting search protocol were beneficial for both the student and the review team members. The questionnaire results did not yield statistically significant differences regarding satisfaction with the search process between case and control groups. Conclusions – Evidence-based reviewers' satisfaction with the literature searching process depends on multiple factors and it was not clear that embedding an LIS specialist in the review team improved satisfaction with the process. Future research with more respondents may elucidate specific factors that may impact reviewers' assessment. Introduction Research teams in evidence-based practice (EBP) must master numerous skills and follow established procedures to produce high quality products that adhere to methodologically rigorous scientific standards. The Oncology Nursing Society's (ONS) Putting Evidence into Practice (ONS- PEP™) program has produced evidence-based summaries for oncology nurses for managing symptoms and side effects of cancer and cancer treatments (Eaton, Tipton, & Oncology Nursing Society, 2009). These summaries are prepared by small teams consisting of staff nurses, advanced practice nurses, and nurse scientists. By 2008, 16 ONS-PEP™ reviews had been disseminated among the ONS membership in various forms, including print books, online resources, and quick reference cards (Doorenbos et al., 2008). As of 2009, ONS had distributed to oncology nurses over 147,000 copies of the reference cards (Saca-Hazboun, 2009). According to ONS (2008), ONS-PEP™ reviews classify interventions into six categories. These categories are ranked from “Recommended for Practice,” where there is “strong evidence from rigorously-designed studies, meta-analyses, or systematic reviews, and for which expectation of harms is small compared with the benefits,” to “Not recommended for Practice,” i.e. interventions in which there is clear evidence of harm, ineffectiveness, or cost or burden exceeding likely benefit (Oncology Nursing Society, 2008). Gobel, Beck, and O’Leary (2006) outlined the schema for evidence rating used in ONS reviews. Rutledge, DePalma, and Cunningham (2004) described the Triad Model of Research Synthesis used in ONS-PEP™ reviews. The Triad Model emphasizes collaboration between advance practice clinicians, researchers and educators. McGowan and Harris have documented numerous examples of the roles that trained medical librarians play in the development of systematic reviews used to guide EBP (Harris, 2005; McGowan & Sampson, 2005). These include clarifying the research question, the reference interview, electronic reference management, and obtaining the full text of studies as well as expert searching. Weller (2004) argued for the inclusion of medical librarians in the development of meta-analyses to maximize the rigor of the search strategies employed and to improve documentation and reproduction of those strategies. It is necessary to document and standardize the search component of an EBP synthesis so that it can be repeated by other researchers. However, Patrick, et al., found that many meta-analyses failed to do so (2004). Before the 2009-2010 reviews, there was no formal protocol for ONS-PEP™ investigators to guide the literature search phase of the review process. Responsibility for searching typically fell to one or two team members who volunteer to conduct the search; these team members may have consulted with a medical librarian, but it was not mandated that they do so. The level of detail provided about the search strategies Evidence Based Library and Information Practice 2011, 6.3 30 employed varied among reviews as well. For example, a lymphedema review listed the specific databases searched (Poage, Singer, Armer, Poundall, & Shellabarger, 2008). Rutledge, et al., (2004) documented the process of formulating the final strategy, including teams’ consultation with librarians. Page, Berger, and Johnson (2006) provided no description of the literature search strategy in the otherwise detailed review on sleep-wake disturbances. This inconsistency with describing the search process within systematic reviews is not unusual (Zhang et al. 2006). Consequently, one of the authors, a previous ONS-PEP™ investigator serving on the faculty of the University of Michigan's (UM) School of Nursing, sought to standardize and improve the literature review experience for review teams. He partnered with a master's student at UM’s School of Information to develop a protocol for the literature search process of ONS-PEP™ reviews. This goal was embraced by ONS staff to ensure that the synthesis of evidence for ONS-PEP™ resources would be grounded in a clear and reproducible literature search process. Following the completion of the 2009-2010 reviews, ONS staff surveyed ONS-PEP™ reviewers on their experiences with the reviews. The aim of this paper is to describe how the student was embedded into the process. It additionally explains the protocol developed for the literature search component of the review in order to assess reviewers’ satisfaction with the search process. Methods An intervention was devised which embedded the student in a review team. The student was assigned to the team responsible for developing the Hormonal Side Effects review. The team subsequently narrowed the scope of the review to “hot flashes,” one specific hormonal side effect. Study participants included the student, his colleagues on the Hot Flashes team, and the members of the other 2009-2010 review teams. The primary outcome of interest included the completion of a literature search protocol by the student for use in future reviews. The secondary outcome of interest was the satisfaction with the review search process of the team exposed to the intervention (“hot flashes”), as compared to that of the other review teams (the control). The investigators employed a mixed-methods evaluation consisting of both qualitative (team member interviews) and quantitative elements (a survey). Protocol Development andEembeddingSstudent in ReviewTteam The student developed a literature search protocol for use in future reviews, based on his experiences in this team, as described below. The first step in developing the protocol was to synthesize relevant studies about the literature review gathering process and evaluate the literature in the context of the current ONS-PEP™ process. A search of MEDLINE using Pubmed, incorporating the terms “evidence-based medicine,” “librarians,” “libraries,” and “systematic review” was conducted. Additional literature was identified through the process of citation chasing, defined as examining sources referenced in the most relevant articles from the MEDLINE search. Literature was reviewed on both the systematic review process and on EBP to identify best practices. ONS-PEP™ publications were also reviewed in order to understand literature search approaches, the databases searched, and the search limits previous reviewers applied. Based on a review of case studies and other articles that examined the librarian’s role in EBP, The Cochrane Handbook was identified as a potential model for adoption. The handbook provides detailed instructions for literature searching and citation management for researchers conducting Cochrane systematic reviews (The Cochrane Collaboration, 2008). Based on experience with other evidence-based resources at an early phase in the project, the student hypothesized the Cochrane Handbook could provide an appropriate roadmap for ONS-PEP™ reviews. Evidence Based Library and Information Practice 2011, 6.3 31 The student interviewed relevant staff members at ONS including the ONS librarian, the ONS-PEP™ coordinator, and a research associate assisting with the reviews. (These three individuals are hereafter referred to as “ONS staff.”) Based on their feedback, a memorandum outlining a proposal for the student’s participation in the review was submitted. After obtaining additional feedback on this proposal and approval from the ONS-PEP™ coordinator, the student was embedded in a review team to pilot test and refine the protocol. The student and his faculty advisor submitted a completed protocol for ONS, and performed a debriefing to review lessons learned and strategies for the future. Details for these activities performed are described below. The faculty mentor, student, and the ONS staff members previously mentioned discussed project options by telephone. Topics included the purpose and expected benefits of a revised searching protocol, student goals and objectives, and idealized outcomes of the proposed changes for ONS-PEP™ reviewers. Discussions occurred over the proposed protocol’s scope (i.e., determining which databases to search), and refinement of the scope of each ONS-PEP™ review using the PICO (Patient-Intervention-Comparison- Outcome) framework (Sackett, 2000). The telephone calls also established key deliverables and timelines. A formal memorandum outlining the final proposal and explaining the nature of the student’s participation in the review teams was submitted. From August until late October 2009, the student participated in conference calls with members of the Hot Flashes team every two weeks. The team was comprised of six clinical nurses and nurse practitioners, one doctorally- prepared nurse researcher, and the student. The student conducted the literature search and citation management for the review. The student departed from the team when the search stage was completed and the article review stage began. Concurrently, the student enrolled in an evidence-based health information class at the UM School of Information to provide the theoretical and practical knowledge necessary for project execution. Among many other topics, the course content included principles of evidence-based medicine; how to critically appraise a scientific study; how to find a variety of information sources such as practice guidelines, unpublished studies, and research articles; and an overview of the process for conducting a systematic review. The initial protocol draft numbered 17 pages. The protocol was revised based on comments and discussions with the student, mentor and ONS staff (Appendix). Evaluation The process was evaluated in a number of ways. An ongoing observation was carried out by the student when developing the protocol and working with the Hot Flashes team. These observations were refined and worked into the protocol that was accepted for future PEP teams. The Hot Flashes team participated in a structured discussion by telephone which was designed to elicit their views on embedding an LIS student in the PEP process. Positive and negative experiences were sought as well as their suggestions for future improvement. Approximately one month after the embedded experience, the mentor and student met with the ONS staff to identify lessons learned and to consider the impact on the ONS-PEP™ review process. One of the ONS staff members involved in the initial planning of the study conducted semi-structured phone interviews with groups of reviewers from the Hot Flashes team. The interviews were designed to elicit responses from team members regarding their experience of the entire PEP process. All team members participated in this phone interview and responded to open-ended questions to identify positive and negative aspects of their experiences, as well as to provide recommendations for process improvement. Evidence Based Library and Information Practice 2011, 6.3 32 Responses from all teams were recorded and summarized by the interviewer. A quantitative evaluation beginning in August 2010 was conducted by the ONS staff, the faculty mentor, and the student. The ONS staff developed a web-based questionnaire, using the Zarca® (Zarca Interactive, Inc., Herndon, VA) electronic survey system, and disseminated it by email to all 27 members of the 2009-10 PEP teams. Twenty –two members completed the survey, for a response rate of 81.5%. A case-control method was employed, with respondents from the Hot Flashes team analyzed as cases and respondents from all other teams categorized as controls. The questionnaire included several questions pertaining to the literature search phase of the reviews. After responses to the questionnaire were compiled, ONS staff analyzed the results using Predictive Analytics Software® (SPSS, Inc., Chicago, IL), version 18.0. Results Qualitative Results Protocol Development The search protocol was refined iteratively (see Appendix for final version). The protocol required significant adaptations to the procedures outlined in the Cochrane Handbook (Higgins & Green, 2009) because ONS-PEP™ reviews differ from Cochrane systematic reviews in purpose, methods and scope. Cochrane reviews typically evaluate the evidence pertaining to one specific intervention for a given condition, and limit results to randomized controlled trials (RCT). In contrast, ONS-PEP™ reviews are broader, evaluating multiple interventions for a condition. Rather than limiting the type of research design to randomized trials, ONS- PEP™ reviews include all types of research and summarize the limitations of the evidence. The protocol development process helped identify differences in assumptions, attitudes, and approaches among the ONS-PEP™ leadership team. This encouraged debate and acknowledgment of different perspectives among team members. Following this discussion, clearer processes emerged and were communicated across review teams. For example, while the Cochrane Handbook mandates the searching of grey (unpublished) literature for each review, ONS determined that grey literature should only be identified to "tip the balance" in cases where evidence for an intervention is ambiguous. Reasons for doing so were sound, but have important methodological implications for the results of each review. The final protocol explicated this justification. Key components of an effective search protocol that emerged during the process included specifying database and bibliographic management requirements. Experience during the Hot Flashes review, demonstrated that at the beginning of the review process, reviewers need to clearly identify the databases to be searched, search limits, and the citation management software best suited to the project. Before searching, teams should also identify ways to automatically sort and organize results to optimize the process of reviewing the retrieved studies; for example, separately filtering studies by topic (such as pharmacological versus alternative therapies) or level of evidence (such as secondary from primary level sources). This organization was seen to facilitate summary and review of various interventions. Due to time constraints, the range of experience of clinician reviewers, and the purpose of the review series, the Cochrane standards were not always appropriate or practical. However, Cochrane standards were used as a frame of reference to guide the final protocol. Based on feedback from ONS staff, the investigators found that the recommendations as originally drafted needed to be simplified for adoption by busy health care professionals who were unfamiliar with literature searching nuances and terminology. The major adaptations from the initial draft included the preparation of a one-page protocol synopsis for reviewers. The synopsis summarized the most essential points from the full protocol Evidence Based Library and Information Practice 2011, 6.3 33 and outlined sequentially the key tasks and milestones that each ONS-PEP™ team would need to accomplish (see Appendix, “The Literature Search Process for Putting Evidence Into Practice (PEP) Review Teams”). The student also modified the sequence of the search and citation management activities. The reviewers encountered other challenges in the process that made changes to the protocol necessary. Key challenges and adaptations to the resulting guidelines included issues in forming the research question and external constraints that affected the process. Clarifying the research question Initially there was disagreement among reviewers over the scope of the question. The process was delayed by the consensus-based nature of ONS-PEP™ team decision-making, and accordingly influenced the literature search process and timeline. The approach for review and synthesis of evidence is somewhat unique in the ONS-PEP™ team process. Specifically, the summary, synthesis and categorization, or weighting of evidence is done by team consensus in the application of specified characteristics of the body of the evidence. Teams are purposely constructed to include a range of nurses specialized in oncology, including nurse scientists, nurse educators, advanced practice nurses and staff nurses. This mix of individuals is used to ensure that the synthesis and production of evidence-based resources incorporates a range of knowledge, expertise and perspectives. The PICO format, along with the direct guidance and collaboration of the student enabled the team to resolve these differences and adopt a clearer charge for content review, according to ONS staff and PEP team members. One key factor that enabled the Hot Flashes team to move forward was narrowing the scope of the research question. Other teams found the PICO format confusing, suggesting that the inclusion of an information science specialist can be a critical factor in facilitating this aspect of the process. Consequently, the formulation of the PICO question assumed a greater prominence in the guidelines. Process constraints Two major constraints in the process emerged in the project: reviewer time limitations and the nature of the available evidence in the topic area. After the team narrowed the review scope from hormonal side effects to solely hot flashes, ten days remained to conduct the search and share the results with colleagues. The majority of time was consumed by eliminating duplicates, technical problems with citation management and transferring Refworks™ (ProQuest LLC, Ann Arbor, MI) libraries between the student and the ONS lead for the review team. The authors changed the guideline to increase the emphasis on the citation management process and citation management guidelines. For the Hot Flashes team, high-quality literature and consistent research findings for particular interventions were scant. In response, the final protocol provided guidance for targeted search strategies in such instances. The experience also identified improved outcomes with didactic and applied education, and benefits of collaboration between EBP reviewers and library/information science (LIS) students, which will be discussed later. Quantitative Results Results from the questionnaire are presented in Table 1. The questions were scored on a ten- point Likert scale, where 0 = highly dissatisfied to 10 = highly satisfied with the exception of the question “How difficult was it for your PEP team to define your literature search strategy?” (where lower numbers equaled less reported difficulty). Satisfaction levels were generally moderate for each question. The highest responses were for the questions “How confident are you that all appropriate and relevant evidence was captured in your PEP team literature search?” and “How satisfied are you with the final product the PEP team created?” The lowest responses were for questions one and two: the teams found it moderately difficult to define the search strategy and were only somewhat Evidence Based Library and Information Practice 2011, 6.3 34 satisfied with the process of defining the strategy. A trend of increased satisfaction with the literature search process was not observed when the student participated in the review group. Indeed, there was a trend toward negative findings in the case group compared with the control groups; however, these differences failed to reach statistical significance. Discussion The qualitative evaluation suggested that embedding LIS students in EBP projects is mutually beneficial, and that American Library Association-accredited LIS science programs should collaborate with EBP investigators to advance both clinical care and students' professional interests. In their debriefing, ONS leadership reported they appreciated the outside perspective of a librarian, as well as a skill set that complemented and extended the team review. They have now incorporated ways to embed librarians or library students in future rounds of searching to support review teams. As specified in the protocol, PICO search terms and inclusion and exclusion criteria are clearly defined for the literature search and review phases of the process. Team members first categorize, then summarize the evidence they have retrieved. Combined didactic and applied learning can improve outcomes for both LIS students and EBP reviewers. One example of the impact of formal education on the applied experience was a change in the preferred interface for searching MEDLINE. Initially, the student conducted the search for the Hot Flashes team using Pubmed. An experienced systematic reviewer pointed out that the saved strategies Table 1 Satisfaction with Review Process Case Group n= 6 Control Group n= 13* P Question Mean (SD) 1. How difficult was it for your PEP team to define your literature search strategy? 5.00 (2.68) 4.31 (2.32) .57 2. How satisfied were you with the process of defining your literature search strategy? 5.83 (3.13) 7.15 (2.08) .29 3. How confident are you that all appropriate and relevant evidence was captured in your PEP team literature search? 7.67 (1.51) 8.15 (1.52) .52 4. How satisfied were you with the process of summarizing and categorizing the evidence? 6.33 (1.37) 6.46 (2.11) .89 5. How satisfied are you with your overall experience in participating in this activity? 6.50 (2.510) 8.31 (1.653) .07 6. How satisfied are you with the final product the PEP team created? 7.50 (2.07) 8.92 (0.95) .16 *Total response on the entire survey was 22 people, but the data reported are from 13 subjects due to missing data. Evidence Based Library and Information Practice 2011, 6.3 35 in Ovid MEDLINE were significantly clearer and more elegant in presentation compared to the strategy in Pubmed. The Pubmed search details included numerous nested parentheses, making the structure of the strategy hard to unravel and to read. Thus, the final protocol was revised with the recommendation to use Ovid MEDLINE if and when team members had access to it (with the understanding that this is not always the case). This activity and study findings have resulted in practice changes for the conduct of the PEP processes. The quantitative evaluation provided mixed results. The responses to questions 3 and 6 suggest Hot Flashes team members were satisfied with their final product, and that they felt confident that all appropriate and relevant evidence was captured by the literature search. Responses to other questions suggest that the contribution of an LIS student to evidence-based reviews does not necessarily increase reviewers’ satisfaction with the literature search process. Unfortunately, the questionnaire did not probe respondents’ specific reasons for their satisfaction with the assistance of the student. The study also does not seek to objectively evaluate the quality of the search or the search results from either the case or control teams. The team members’ initial debate over the scope of the research question and the resulting time constraints may have influenced the results. It is possible that the presence of, or supervision by, an LIS professional with greater experience and expertise would have yielded different results. Consequently, the impact of the level of the LIS specialist’s experience on reviewer satisfaction merits further study, alongside the need for research on the impact of LIS specialist involvement on the quality of the search process which was identified by Zhang et al., (2006). The experience in this study suggests that if protocols are to be adopted by clinician- reviewers, a user-centered perspective is essential. To adhere to Cochrane standards (or any other protocols) at the expense of the user experience is to risk crafting a document no one will read, and mandating steps no one can or will follow. Groups that wish to develop a similar protocol for their EBP projects may find it helpful to borrow practices from human computer interaction and design such as Holtzblatt, Wendell, and Wood’s contextual inquiry (2005) and to conduct them before drafting and piloting a protocol. Since Cochrane reviews examine narrowly defined medical interventions, it may also be more appropriate to examine literature about searching in relation to more complex topics (eg Golder et al, 2008; McNally and Alborz, 2004; Ogilvie et al, 2005 ). How do these findings fit into the broader literature on this topic? There is an abundance of studies on the impact of information professionals on various outcomes in the health sciences. For example, a systematic review by Rankin, Grefsheim, and Canto (2008) summarizes a number of studies on the impact of the informationist, defined as “a new professional… with responsibility for providing highly specialized information services in the clinical setting.” Others have examined the impact of librarian services on clinical practices (McGowan, et al., 2010). However, to the authors’ knowledge, there have been no formal evaluations of the impact of an embedded LIS specialist on their outcome of interest, the satisfaction of EBP review team members. The issues that arose in the qualitative evaluation are a reminder that librarians and informationists seeking to evaluate their own effectiveness need to build evaluation into projects from the beginning. A prospective, rather than retrospective, study design for evaluation may be helpful. Having teams keep diaries or journals of each phase of the project could provide qualitative data to help identify confounding factors. Future studies could include measurement development to examine: time spent during literature search; time spent on abstract review; time spent on paper inclusion and exclusion; number of irrelevant references retrieved that had to be discarded, and team member satisfaction with the review process. Evidence Based Library and Information Practice 2011, 6.3 36 As evidence-based reviews proliferate, clinical agencies and reviewers face multiple challenges in the conduct and dissemination of review results. The authors’ experience suggests that embedding an LIS student is a feasible approach to facilitate review as well as a valuable learning experience for the student. Future investigators should consider a prospective evaluation. They should also solicit specific feedback on the evaluation of the specific role of an LIS specialist within a review team. With such data, evidence-based reviews in the future can be more confident in the deployment of valuable human resources. This study contained a number of limitations stemming from the retrospective rather than prospective nature of the evaluation. Quantitative evaluation of this project was designed to answer the question, “is there a difference among review team members in satisfaction with the process and results as a result of embedding an information specialist?” This approach did not allow us to evaluate the potential difference within a topic team with or without an information specialist. Thus there is no way to determine the impact of the specialist within the team itself – how the results would have differed in the same group of reviewers without the assistance of the information specialist. Quantitative findings were also limited by the small sample size, which did not have sufficient power to detect a significant difference between groups. A number of global factors may also confound the results. Group dynamics and experience may have contributed to perceived satisfaction with the review process for both case and control groups. Overall satisfaction with the process within the Hot Flashes ONS-PEP™ team may have also been influenced by the fact that little high quality evidence in this topic area was found. Conclusions This project sought to embed and evaluate an LIS student within an evidence based review process. The project resulted in a standard protocol to be used for searches conducted by future ONS-PEP™ teams. The protocol anticipates problems and issues encountered in this experience and provides guidelines for search documentation that improves the transparency and reproducibility of the process. The authors found that combining applied and didactic learning resulted in improvements to the protocol. The project was evaluated by both quantitative and qualitative methods. Qualitative feedback from those involved in the review which received the embedded experience demonstrated that team members were satisfied and appreciated having the perspective of an LIS professional on the team. In contrast, the quantitative evaluation suggested that the student’s contribution failed to increase reviewer satisfaction in the literature search process, however differences in survey responses failed to reach statistical significance. The experience gained in this project reinforces the need to evaluate student learning projects from the beginning in order to maximize the quality of data collected. References Doorenbos, A., Berger, A., Brohard-Holbert, C., Eaton, L., Kozachik, S., LoBiondo- Wood, G., Varricchio, C. (2008). Oncology Nursing Society Putting Evidence into Practice resources: Where are we now and what is next? Clinical Journal of Oncology Nursing, 12(6), 965-970. Eaton, L. H., Tipton, J. M., & Oncology Nursing Society. (2009). Putting Evidence into Practice: Improving oncology patient resources. Pittsburgh, PA: Oncology Nursing Society. Gobel, B. H., Beck, S. L., & O'Leary, C. (2006). Nursing-sensitive patient outcomes: The development of the Putting Evidence into Practice resources for nursing practice. Clinical Journal of Oncology Nursing, 10(5), 621-624. Evidence Based Library and Information Practice 2011, 6.3 37 Golder, S., Mason, A., & Spilsbury, K. (2008) Systematic searches for the effectiveness of respite care, Journal of the Medical Library Association, 96 (2), pp 147-152. Harris, M. R. (2005). The librarian's roles in the systematic review process: A case study. Journal of the Medical Library Association, 93(1), 81. Higgins, J., & Green, S. (Eds.). (2009). Cochrane handbook for systematic reviews of interventions version 5.0. 2 [updated September 2009] Holtzblatt, K., Wendell, J. B., & Wood, S. (2005). Rapid contextual design: A how-to guide to key techniques for user-centered design. San Francisco: Elsevier/Morgan Kaufmann. McGowan, J., Hogg, W., Rader, T., Salzwedel, D., Worster, D., Cogo, E., & Rowan, M. (2010). A rapid evidence -based service by librarians provided information to answer primary care clinical questions. Health Information & Libraries Journal, 27(1), 11-21. McGowan, J., & Sampson, M. (2005). Systematic reviews need systematic searchers. Journal of the Medical Library Association, 93(1), 74. McNally., R., & Alborz, A. ( 2004). Developing methods for systematic reviewing in health services delivery and organization: an example from a review of access to health care for people with learning disabilities. Part 1. Identifying the literature, Health Information & Libraries Journal, 21 (3), pp 182-192. Ogilvie, D., Hamilton, V., Egan, M., & Petticrew, M. (2005) Systematic reviews of health effects of social interventions: 1. Finding the evidence: how far should you go?, Journal of Epidemiology and Community Health, 59 (9), pp 804-808. Oncology Nursing Society. (2008). ONS PEP (Putting Evidence into Practice) weight of evidence classification schema: Decision rules for summative evaluation of a body of evidence. Pittsburgh, PA: Mitchell, S.A., & Friese, C.R.. Page, M., Berger, A., & Johnson, L. (2006). Putting Evidence into Practice: Evidence-based interventions for sleep-wake disturbances. Clinical Journal of Oncology Nursing, 10(6), 753- 767. Patrick, T. B., Demiris, G., Folk, L. C., Moxley, D. E., Mitchell, J. A., & Tao, D. (2004). Evidence-based retrieval in evidence- based medicine. Journal of the Medical Library Association, 92(2), 196. Poage, E., Singer, M., Armer, J., Poundall, M., & Shellabarger, M. J. (2008). Demystifying lymphedema: Development of the lymphedema Putting Evidence into Practice card. Clinical Journal of Oncology Nursing, 12(6), 951-964. Rankin, J. A., Grefsheim, S. F., & Canto, C. C. (2008). The emerging informationist specialty: A systematic review of the literature. Journal of the Medical Library Association: JMLA, 96(3), 194. Rutledge, D., DePalma, J., & Cunningham, M. (2004). A process model for evidence- based literature syntheses. Oncology Nursing Forum, 31(3), 543-550. Saca-Hazboun, H. (2009). Putting Evidence into Practice. outcomes of using the ONS PEP resources. ONS Connect, 24(4), 8-11. Sackett, D. L. (2000). Evidence-based medicine: How to practice and teach EBM. Edinburgh; New York: Churchill Livingstone. The Cochrane Collaboration. (2008). The Cochrane Library: Evidence for healthcare Evidence Based Library and Information Practice 2011, 6.3 38 decision-making. Retrieved April 15, 2010, from http://www3.interscience.wiley.com/a boutus/sharedfiles/brochures- flyers/Cochrane_library_brochure.pdf Weller, A. C. (2004). Mounting evidence that librarians are essential for comprehensive literature searches for meta-analyses and Cochrane reports. Journal of the Medical Library Association, 92(2), 163. Zhang, L., Sampson, M., & McGowan, J. (2006). Reporting of the role of the expert searcher in Cochrane Reviews. Evidence Based Library And Information Practice, 1(4), 3-16. Retrieved from http://ejournals.library.ualberta.ca/ind ex.php/EBLIP/article/view/57 Evidence Based Library and Information Practice 2011, 6.3 39 Appendix The Literature Search Process for Putting Evidence into Practice (PEP) Review Teams The purpose of the guide is to make the complex and often intimidating process of a comprehensive literature search easier for all team members involved. The guidelines are intended to be a framework. Individual team members are free to exercise their own discretion in determining whether a recommendation is appropriate for their particular topic. Individual teams, because of the question under study, may alter these suggestions. Suggestions for revision are not just welcomed, but encouraged First conference call • Identify who will conduct searches, and any additional sources to search • Work through PICO question as group, clarify scope • Confirm time frame and due date for final search results Identifying who will conduct the literature searches: The team researcher and educator are the most likely candidates for searcher(s). However, the search team could include any other team member who is comfortable with searching, and is affiliated with an institution with a robust set of database subscriptions. The following databases are recommended for all project teams: CINAHL, MEDLINE via Ovid SP, EMBASE, and Cochrane CENTRAL. Additional sources, including professional conference proceedings, may be considered individually by project teams. Developing and refining the PICO question: PICO is a method of forming a focused clinical question. Carefully working through the PICO question up front saves the searcher and team a lot of time down the road. This can be done as a team to come up with a comprehensive list of concepts and terms for the population examined, interventions searched, and outcomes measured. Don’t forget to incorporate survivorship concepts into your PICO question. Confirm time frame and due date for final search results : At the time of the first team call, the searchers should confer with the team leader to clarify the date that final search results are due, and to ensure that the timeframe for them to do this is adequate. Since PEP researchers are often working full time at their own jobs, it may be best to allow two weeks or more to complete this. By the second conference call, searches should: • Complete the PICO Outline tool (and/or other PICO worksheets) • Complete Search Planning Form from “Literature Search Process: Protocols for Researchers” • Compare/share PICO and Search Planning forms; be prepared to share and discuss any disagreements with the rest of the team during call Second conference call • Share PICO and Search Planning Strategy with team • Clarify any remaining questions/resolve any debates about scope of PICO question with team Share PICO and Search Planning Strategy with team: Searchers should be prepared to briefly share the results of their PICO work with the rest of the team. They should resolve what differences they can prior to the meeting and if any remain outstanding, they can share these with the team for further discussion. NB: The full version of the protocol and associated documents are available from the authors on request. / Evidence Based Library and Information Practice/ /