ORIGINAL RESEARCH Marie T. Ascher Jonathan D. Eldredge Hypothesis, vol. 29, no. 1, Spring/Summer 2017 5 MLA Research Section’s Research Agenda Committee Systematic Review Project: A Status Report Marie T. Ascher, MS, MPH, AHIP Lillian Hetrick Huber Endowed Director Health Sciences Library New York Medical College Valhalla, NY Jonathan D. Eldredge, MLS, PhD, AHIP Associate Professor Evidence Based and Translational Science Collaboration Coordinator Health Science Library & Informatics Center Biomedical Informatics Research, Training, and Scholarship Unit University of New Mexico Albuquerque, NM jeldredge@salud.unm.edu BACKGROUND Ambitious intentions, however well-conceived and embraced, can take time to manifest. Our elected and appointed MLA leaders have encouraged us for over 20 years to integrate the best evidence into our professional practices. These broad-based investments at the policy level now are yielding tangible results. The first MLA research policy, Using Scientific Evidence to Inform Practice, emphasized applying research evidence when making decisions [1]. This 1995 policy statement, along with MLA President Rachel Anderson’s 1997 inaugural address, helped spark the international Evidence Based Library and Information Practice (EBLIP) movement that continues to thrive today [2, 3, 4]. In fact, most MLA presidents since 1997 have called on us to integrate research evidence into our practices [5]. The newest MLA research policy, The Research Imperative, called for the Research Section to articulate an MLA research agenda [6]. The Research Section’s Research Agenda Committee ORIGINAL RESEARCH Marie T. Ascher Jonathan D. Eldredge Hypothesis, vol. 29, no. 1, Spring/Summer 2017 6 conducted two Delphi studies in 2008 and 2011 to identify the most important and answerable research questions facing the profession [7]. The second Delphi study led the Committee to develop guidelines for voluntary teams to create systematic reviews for assembling the best evidence to form the tentative answers to each of the 15 top-ranked research questions [8]. The teams were to act largely autonomously. The 15 teams, consisting of over 200 medical librarians worldwide have made progress, for the most part, in completing their systematic reviews [9, 10]. The principal benefits of these systematic reviews will be as evidence resources for answering these top-ranked questions in addition to acting as blueprints pointing to further research needed to build our knowledge base strategically. Health sciences librarians have been integral members of systematic review teams outside librarianship since the 1990s, particularly in medicine [11, 12, 13, 14, 15]. Many guidelines and experts on systematic reviews point to the need for librarians to serve on systematic review teams [16, 17, 18]. Our own profession has produced over 90 systematic reviews on subjects related to library and information practice [19]. These systematic reviews frequently confirm the recurring observation that our own knowledge base lacks sufficient amounts of rigorous research evidence. A large percentage of our evidence also resides within the gray literature rather than the peer reviewed literature, which poses challenges to identification and critical appraisal. This paper provides a status report on the MLA Research Section’s Research Agenda Committee Systematic Review Project. There has not been a comprehensive report on the project since the 2015 Open Forum held at the Annual Meeting of the Medical Library Association in Austin, TX, where representatives from all active teams provided reports [20]. To date, teams have continued to vary in their rate of progress. Some teams have moved through the process relatively quickly while others are regrouping. An evaluation of the overall project experience from the point of view of the participants is forthcoming. METHOD & RESULTS To complete this inventory of progress, the fifteen team leaders were surveyed as to their current progress and to report any research outputs to date. The status categories they had to ORIGINAL RESEARCH Marie T. Ascher Jonathan D. Eldredge Hypothesis, vol. 29, no. 1, Spring/Summer 2017 7 choose from represent discrete phases of the systematic review process: 1) Not started or regrouping; 2) Very early: Question clarification or earlier; 3) Early: Search strategy development 4) Mid: Screening abstracts; 4) Late mid: Screening articles; 5) Later: Data extraction and analysis; 6) Nearing completion: Manuscript preparation; 7) Complete and published. All teams with current leadership responded. The remaining two were described by the first author of this paper as “not started.” There is one team currently without an assigned team leader and has stalled at square one as of this writing. Results of this inquiry are reported in Table 1. Table 1. Team Leaders’ Reported Team Progress Three of the fifteen teams have completed their reviews and published articles. Six other teams are late in the systematic review process, two teams are in the middle, and four teams are very early in the process. Three teams are either regrouping or currently without leadership. The loss of leadership has been an issue for several teams throughout this ambitious project. Table 2 lists status by team and research outputs, including publication and presentation information. 3 1 0 1 1 4 2 3 0.0% 5.0% 10.0% 15.0% 20.0% 25.0% 30.0% Not started Very early Early Mid Late mid Later Nearing Complete ORIGINAL RESEARCH Marie T. Ascher Jonathan D. Eldredge Hypothesis, vol. 29, no. 1, Spring/Summer 2017 8 The original questions that emerged from the second Delphi study [7] are listed although teams were tasked with reworking their questions to make them suitable for conducting a review. Table 2. Team Progress and Research Outputs Team # 1. Question: There are still a number of relevant questions from the 2008 research agenda, but to me this is most critical: "What is the quantifiable evidence that the presence of a librarian, not just information resources, improves patient outcomes, increases research dollars, improves student outcomes (e.g., better board scores), or increases hospital intelligence (e.g., if the top hospitals have access to hospital librarians/libraries)?" Status: Complete and published Presentations: Perrier L, Farrell A, Weiss A, Lightfoot D, Aaronson E, Connor E, Epstein Brown HA, Muellenbach JM, Allee N, Ayala P, Kenny T, Constantinescu T, Brigham T. Effects of librarian- provided services in health care: a systematic review. [Contributed Paper.] Proceedings of the Canadian Health Libraries Association/Association des Bibliothèques de la Santé du Canada. 2014 June 16-20. Montreal, QC, Canada. J Can Health Libr Assoc. 2014; 34(2):92-93. https://journals.library.ualberta.ca/jchla/index.php/jchla/article/view/22905/17060 Publications: Perrier L, Farrell A, Ayala AP, Lightfoot D, Kenny T, Aaronson E, Allee N, Brigham T, Connor E, Constantinescu T, Muellenbach J, Epstein HA, Weiss A. Effects of librarian-provided services in healthcare settings: a systematic review. J Am Med Inform Assoc. 2014;21(6):1118-24. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4215058/ Note: This paper was the winner of the 2015 Ida and George Eliot Prize which is given to the authors of the work published in the preceding calendar year that has been judge most effective in furthering medical librarianship. Team # 2. Question: Is there a significant difference in patient outcomes (or research output or educational outcomes) between institutions with and without libraries? Status: Very early – question clarification https://journals.library.ualberta.ca/jchla/index.php/jchla/article/view/22905/17060 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4215058/ ORIGINAL RESEARCH Marie T. Ascher Jonathan D. Eldredge Hypothesis, vol. 29, no. 1, Spring/Summer 2017 9 Team # 3. Question: What is the added value libraries bring to education, research, and patient care in the health sciences and health care fields? Even if it is not possible to quantify benefits, documenting qualitative research results rigorous enough to stand the scrutiny of administrators and researchers would be of great value. Status: Later - data extraction and analysis Team # 4. Question: Low health literacy can result in medication errors, noncompliance of treatment regimes, poor health outcomes and even death. What is the role of the medical librarian with health care providers, community organizations, local public libraries and members of the public to improve health literacy among entire communities? Status: Later – data extraction and analysis Presentations: Klem M, Devine PJ, El-Khayat YM, Gutzman KE, Knehans A, Mills TN, Oren GA, Perryman CL, Saleh AA, Unno ZP, Vardell E. Librarians and health literacy: A scoping review. Presented at: Medical Library Association Annual Meeting. 2015 May 14-20. Austin TX. http://www.mlanet.org/d/do/1924 Team # 5. Question: What are the information needs of practicing physicians and other health care workers? The 1985 Covell article is still heavily cited but was published way back in 1985. The information environment has changed dramatically. We need to update that study in lite of new educational strategies, resources, technology and social networks. Status: Nearing completion - manuscript preparation Team # 6 Question: The explosion of information, expanding of technology (especially mobile technology), and complexity of healthcare environment present medical librarians and medical libraries opportunities and challenges. To live up to the opportunities and challenges, what kinds of skill sets or information structure do medical librarians or medical libraries are required to have or acquire so as to be strong partners or contributors of continuing effectiveness to the changing environment? Status: Nearing completion – manuscript preparation Presentations: http://www.mlanet.org/d/do/1924 ORIGINAL RESEARCH Marie T. Ascher Jonathan D. Eldredge Hypothesis, vol. 29, no. 1, Spring/Summer 2017 10 Anderson PF, Bickett S, Doucette J, Herring PR, Kammerer J, Kepsel A, Lyons T, McLachlan S, Tonnison I, Wu L. Tools for Building Our Information Future: Emerging Technologies Vital to Medical Libraries” Health Library Group Conference, Chartered Institute of Library and Information Professionals. 2014 July 24-25. University of Oxford, U.K. Anderson PF, Bickett S, Doucette J, Herring PR, Kammerer J, Kepsel A, Lyons T, McLachlan S, Tonnison I, Wu L. “Tools for Building Our Information Future: Emerging Technologies Vital to Medical Libraries.” [Poster.] Annual Meeting Medical Library Association. 2014 May 16–21. Chicago, IL. NOTE: This poster won an 2014 MLA Research Section, Research Award: Posters, Honorable Mention Anderson PF, Bickett S, Doucette J, Herring PR, Kammerer J, Kepsel A, Lyons T, McLachlan S, Tonnison I, Wu L. Future Technological Practices: Medical Librarians’ Skills and Information Structures for Continued Effectiveness in a Changing Environment. 2014 May 16–21. Annual Meeting Medical Library Association, Chicago, IL. Anderson PF, Bickett S, Doucette J, Herring PR, Kammerer J, Kepsel A, Lyons T, McLachlan S, Tonnison I, Wu L. Tools for Building Our Information Future: Emerging Technologies Vital to Medical Libraries. 2014 September 19. HLI/HLA Joint Conference 2014. State Library of Victoria, Melbourne, Victoria, Australia. https://www.alia.org.au/sites/default/files/HLA%20News-Conference2014.pdf. Anderson PF, Bickett S, Doucette J, Herring PR, Kepsel A, Lyons T, McLachlan S, Shannon C, Wu L. Developing a Replicable Methodology for Automated Identification of Emerging Technologies in Healthcare. Annual Meeting of the Medical Library Association. 2016 May 13–18. Toronto, Canada. Team # 7. Question: Does what we do matter? Longer form: Do the resources we provide - materials, reference services, and educational offerings - make a difference to our customers - save lives, shorten length of stay, improved educational outcomes, increase research dollars, improve research results? Status: Late mid - screening articles https://www.alia.org.au/sites/default/files/HLA%20News-Conference2014.pdf ORIGINAL RESEARCH Marie T. Ascher Jonathan D. Eldredge Hypothesis, vol. 29, no. 1, Spring/Summer 2017 11 Presentations: Glynn LA, Sakmar K, Lalla NJ, Berry R, Kim C, Geldenhuys PR, Lawton A, Siebert JL, McClurg C, Clemans-Taylor L, Gadd K, Ettien A. The Value to and Impact of Health Sciences Libraries and Information Services on Academic and Clinical Practices: A Systematic Review. [Paper.] Annual Meeting of the Medical Library Association. 2014 May 16–21. Chicago, IL. Team # 8. Question: How do we provide information support in a clinical world that functions based on electronic medical records systems and other similar informatics platforms and tools. What is the library's role, if any, in providing preclinical education with respect to informatics applications like electronic medical records systems? Status: Mid - screening abstracts Team # 9. Question: Do health sciences libraries and librarians have any measureable (statistically significant) positive impacts on consumer health, the outcomes of medical care, the productivity of biomedical researchers and the knowledge obtained by graduates of biomedical and health sciences training programs, and at what total cost? Status: Later - data extraction and analysis Presentations: Henderson ME, Crum JA, Fatkin KJ, Gagnon M-M, Nguyen T, Taylor M, and Vrabel M. Do Health Sciences Libraries and Librarians Have an Impact on the Cost of Health Care and Research? A Systematic Review. [Poster.] Annual Meeting of the Medical Library Association. 2015 May 14-20. Austin, TX. http://scholarscompass.vcu.edu/libraries_present/43/ Team # 10. Question: How best to objectively document library/librarian impact on the 'bottom line' (time, money saved, shorter length of stay, ROI for expensive electronic resources, support training programs/Magnet status, funded research support, etc.)? Status: Completed and published Presentations: Collins, P. Counter-measures: a systematic review of the measurement tools used to demonstrate the impact of libraries in a clinical setting. [Presentation.] Health Libraries Group. 2016 Sept 15-16. Scarborough, United Kingdom. http://www.cilip.org.uk/sites/default/files/documents/pam_collins.pdf. http://scholarscompass.vcu.edu/libraries_present/43/ http://www.cilip.org.uk/sites/default/files/documents/pam_collins.pdf ORIGINAL RESEARCH Marie T. Ascher Jonathan D. Eldredge Hypothesis, vol. 29, no. 1, Spring/Summer 2017 12 Madden A. "You say tomayto…”: Alternative viewpoints on international collaboration. [Presentation.] Health Libraries Group 2014 July 24. Oxford, United Kingdom. https://www.cilip.org.uk/sites/default/files/documents/Anne%20Madden.pdf Publications: Madden A, Collins P, McGowan S, Stevenson P, Castelli D, Hyde L, DeSanto K, O'Brien N, Purdon M, Delgado D. 2016. Demonstrating the financial impact of clinical libraries: a systematic review. Health Info Lib J. 2016; 33(3):172-189. http://onlinelibrary.wiley.com/doi/10.1111/hir.12151/abstract Team # 11. Question: As a profession, how do we measure our impact in our environment—be it clinical or academic—in such a way that it influences the decision makers in our institutions? [I "stole" this from the previous study, but I think that it is still the most important question facing us.] Status: Not started – regrouping Team # 12. Question: Does the intervention/instruction/assistance of a professional medical librarian have a long term impact on the information seeking behaviors of health care professionals? Status: Not started – regrouping Team # 13. Question: What are the most effective instructional methods for teaching informatics/knowledge management/EBP within health sciences curricula? Status: Complete and published Presentations: Pannabecker V, Dennison CC, Holyoke AN, Farrell A, Machel V, Marton C, O'Brien KK, Swanberg SM, Thuna M. Creative convergence: Conducting a systematic review project through cross-institutional, distance collaboration. [Presentation.] QuintEssential Joint Chapter Meeting of the Medical Library Association. 2014 Oct 14. Denver, CO. https://vtechworks.lib.vt.edu/handle/10919/50634 Holyoke AN, Dennison CC, Farrell A, Machel V, Marton C, O'Brien KK, Pannabecker V, Swanberg SM, Thuna M. Systematically assessing methods used by librarians to teach https://www.cilip.org.uk/sites/default/files/documents/Anne%20Madden.pdf http://onlinelibrary.wiley.com/doi/10.1111/hir.12151/abstract https://vtechworks.lib.vt.edu/handle/10919/50634 ORIGINAL RESEARCH Marie T. Ascher Jonathan D. Eldredge Hypothesis, vol. 29, no. 1, Spring/Summer 2017 13 evidence-based practice: What works best. [Presentation.] QuintEssential Joint Chapter Meeting of the Medical Library Association. 2014 Oct 14. Denver, CO. Dennison CC, Farrell A, Gore G, Swanberg SM, Machel V, Marton C, O'Brien KK, Pannabecker V, Thuna M. Effectiveness of Instructional Methods Used by Librarians for Teaching Evidence Based Practice: A Systematic Review. Canadian Health Libraries Association Annual Meeting. 2014 June 20. Montreal, Quebec, Canada. Farrell A, Dennison CC, Gore G, Holyoke AN, Machel V, Marton C, O'Brien KK, Pannabecker V, Swanberg SM, Thuna M. An Initiation to Systematic Review. Hawaii-Pacific Chapter of the Medical Library Association Annual meeting. 2014 April 4. Honolulu, HI. Publications: Swanberg SM, Dennison CC, Farrell A, Machel V, Marton C, O'Brien KK, Pannabecker V, Thuna M, Holyoke AN. Instructional methods used by health sciences librarians to teach evidence- based practice (EBP): a systematic review. J Med Libr Assoc. 2016 Jul; 104(3): 197–208. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4915637/ Team # 14. Question: In medical schools where librarians are included in the curriculum, do the students have a greater degree of information literacy than students in schools where librarians are not part of the curriculum? Status: Not started - regrouping Team # 15. Question: What skills and knowledge must librarians possess in order to be able to design tools to help researchers visualize, mine, and otherwise manage large and complex data gathered during both quantitative and qualitative research? Status: Later - data extraction and analysis Presentations: Boden C, Adamczyk A, Ambriz L, Billman BL, Booth A, Clark E, Engwall K, Johnson R, Miller- Nesbitt A, Morris M, Woznica A. Librarian knowledge and skills of tools for visualizing, mining and managing large and complex research data: A systematic review. [Poster.] Annual https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4915637/ ORIGINAL RESEARCH Marie T. Ascher Jonathan D. Eldredge Hypothesis, vol. 29, no. 1, Spring/Summer 2017 14 Meeting of the Medical Library Association. 2015 May 14-20. Austin, TX. http://eventscribe.com/2015/MLA/TwitterPoster.asp?PosterID=38894 Miller-Nesbitt A, Boden C, Adamczyk A, Ambriz L, Billman BL, Booth A, Clark E, Engwall K, Johnson R, Morris M, Woznica A. Applying the best-fit framework to systematic review data extraction. [Presentation.] 7th Qualitative and Quantitative Methods in Libraries International Conference (QQML2015). 2015 May 26-29. Paris, France. Morris M, Boden C & Nesbitt-Smith A. (2014). Distributed collaborative virtual systematic reviewing: A blueprint for the future? [Presentation.] Canadian Health Libraries Association Annual Meeting, June 16-20, 2014, Montreal, QB. https://journals.library.ualberta.ca/jchla/index.php/jchla/article/view/22906/17075 DISCUSSION Fifteen teams conducting systematic reviews on the top-ranked research questions met with varying success. One of the teams completed their systematic review in about a year and their paper was the 2015 winner of the Medical Library Association’s Ida and George Eliot Prize, awarded annually for a work that has been judged most effective in furthering medical librarianship [21]. Others have moved more slowly, but most are on track to completion. The project has resulted in presentations at meetings in the US, Australia, France, Canada, and the UK and three important systematic review papers have been published so far [21, 22, 23] with several in the pipeline. Future work of the Committee involves a centralized web resource summarizing the outcomes of the project as well as an overall evaluation of the project to inform other such potential endeavors. Although the actual overall time of completion has exceeded preliminary expectations, the project is continuously yielding valuable information and will continue to be a landmark in health sciences librarianship research. REFERENCES 1 Dalrymple PW, Bastille JD, Bradley J, Dee CR, Humphreys BL, Marshall JG, Weller AC, Webb RE. Using scientific evidence to improve information practice. Chicago, IL: Medical Library http://eventscribe.com/2015/MLA/TwitterPoster.asp?PosterID=38894 https://journals.library.ualberta.ca/jchla/index.php/jchla/article/view/22906/17075 ORIGINAL RESEARCH Marie T. Ascher Jonathan D. Eldredge Hypothesis, vol. 29, no. 1, Spring/Summer 2017 15 Association, 1995. [cited 10 March 2017]. . 2 Anderson RK. Inaugural address. Proceedings, Ninety-seventh Annual Meeting Medical Library Association, Inc. Seattle, Washington May 24-28, 1997. Bull Med Libr Assoc. 1998 Jan;86(1):137-9. PubMed PMID: 16018060; PubMed Central PMCID: PMC226342. 3 Eldredge J. Evidence-Based Librarianship; a commentary. Hypothesis. 1997 Fall; 11(3): 4-7. 4 Eldredge JD. Evidence-based practice. In: Introduction to health sciences librarianship. Edited by Sandra Wood. Binghamton, NY: Haworth Press, 2008: 245-69. 5 Eldredge JD. Integrating research into practice. Journal of the Medical Library Association 2016 Oct; 104 (4): 333-7. 6 Medical Library Association. The research imperative: the research policy statement of the Medical Library Association. Appendix 4: Action Plan. Section One [Internet]. Chicago: Medical Library Association, 2007 [cited 10 March 2017]. . 7 Eldredge JD, Ascher MT, Holmes HN, Harris MR. The new Medical Library Association research agenda: final results from a three-phase Delphi study. J Med Libr Assoc. 2012 Jul;100(3):214-8. doi: 10.3163/1536-5050.100.3.012. PubMed PMID:22879811; PubMed Central PMCID: PMC3411260 8 Ascher MT, Eldredge JD, Holmes HN, Harris MR. The MLA Research Agenda: appraising the best available evidence. Proposed Guidelines to the MLA Research Section Executive Committee. Working paper. 2012 Nov 2. [unpublished]. [Cited 10 March 2017]. . 9 Ascher MT, Holmes HN, Eldredge JD. Addressing the MLA Research Agenda questions: where are we now? Poster. International Congress of Medical Librarianship/Medical Library Association Annual Meeting. Boston, MA. May 4-8, 2013. 10 Holmes HN, Ascher MT, Eldredge JD. The MLA Research Agenda Systematic Review Project. Open Forum. Medical Library Association Annual Meeting. Austin, TX. May 17, 2015. 11 Mulrow CD. Rationale for systematic reviews. BMJ. 1994 Sep 3; 309(6954):597-9. PubMed PMID: 8086953; PubMed Central PMCID: PMC2541393. ORIGINAL RESEARCH Marie T. Ascher Jonathan D. Eldredge Hypothesis, vol. 29, no. 1, Spring/Summer 2017 16 12 Dickersin K, Scherer R, Lefebvre C. Identifying relevant studies for systematic reviews. BMJ. 1994 Nov 12; 309(6964):1286-91. PubMed PMID: 7718048; PubMed Central PMCID: PMC2541778. 13 Hunt DL, McKibbon KA. Locating and appraising systematic reviews. Ann Intern Med. 1997 Apr 1; 126(7):532-8. PubMed PMID: 9092319 14 Beverley CA, Booth A, Bath PA. The role of the information specialist in the systematic review process: a health information case study. Health Info Libr J. 2003 Jun; 20(2):65-74. PubMed PMID: 12786905. 15 Harris MR. The librarian's roles in the systematic review process: a case study. J Med Libr Assoc. 2005 Jan; 93(1):81-7. PubMed PMID: 15685279; PubMed Central PMCID: PMC545126. 16 Institute of Medicine. Finding what works in health care: standards for systematic reviews. Washington, DC; National Academies Press; 2011. 17 Foster MJ. An overview of the role of librarians in systematic review: from expert searcher to project manager. J Eur Assoc Health Inf Libr. 2015;11(3):3-7. 18 Metzendorf MI. Why medical information specialists should routinely form part of teams producing high quality systematic review: a Cochrane perspective. J Eur Assoc Health Inf Libr 2016; 12 (4): 6-9. 19 Koufogiannakis D. LIS systematic reviews [Internet]. Jan 23 2012. [Cited 17 March 2017]. . 20 Eldredge J, Ascher M, Holmes HN. MLA Research Agenda Systematic Review Project Team Updates Presentation. Annual Meeting of the Medical Library Association. 2015 May 17. Austin, TX. 21 Perrier L, Farrell A, Ayala AP, Lightfoot D, Kenny T, Aaronson E, Allee N, Brigham T, Connor E, Constantinescu T, Muellenbach J, Epstein HA, Weiss A. Effects of librarian-provided services in healthcare settings: a systematic review. J Am Med Inform Assoc. 2014;21(6):1118-24. . 22 Madden A, Collins P, McGowan S, Stevenson P, Castelli D, Hyde L, DeSanto K, O'Brien N, Purdon M, Delgado D. 2016. Demonstrating the financial impact of clinical libraries: a systematic review. Health Info Lib J. 2016; 33(3):172-189. . ORIGINAL RESEARCH Marie T. Ascher Jonathan D. Eldredge Hypothesis, vol. 29, no. 1, Spring/Summer 2017 17 23 Swanberg SM, Dennison CC, Farrell A, Machel V, Marton C, O'Brien KK, Pannabecker V, Thuna M, Holyoke AN. Instructional methods used by health sciences librarians to teach evidence- based practice (EBP): a systematic review. J Med Libr Assoc. 2016 Jul; 104(3): 197–208. . ---