Microsoft Word - Comm5082
Evidence Based Library and Information Practice 2009, 4:1
43
Evidence Based Library and Information Practice
Commentary
Systematic Reviews and Evidence Based Library and Information Practice
Alison Brettle
Research Fellow (Information)
Salford Centre for Nursing Midwifery and Collaborative Research, University of Salford
Salford, Great Britain, United Kingdom
E-mail: a.brettle@salford.ac.uk
Received: 30 Jan 2009 Accepted: 31 Jan 2009
© 2009 Brettle. This is an Open Access article distributed under the terms of the Creative Commons Attribution
License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction
in any medium, provided the original work is properly cited.
Introduction
One of the key tools of evidence based practice
for practitioners and policy makers are
systematic reviews. “Rigorous summaries of all
the research evidence relating to a specific
question…often addressing the effectiveness of
an intervention” that employ “a rigorous
methodology for searching, research, retrieval,
appraisal, data extraction, synthesis, and
interpretation” (Ciliska, Cullum, and Marks
135). They present a comprehensive summary
of research based knowledge that can aid both
practitioners and policy makers in decision
making. Following a systematic approach to
both the search and synthesis of the included
research, reduces the risk of bias and random
errors inherent in a standard literature review
process (Droogan and Song 16). When
undertaking a systematic review, details of all
elements of the search, appraisal, and synthesis
process are documented and reported to ensure
transparency. This enables readers to assess the
quality of the review process and to determine
the extent to which it has been rigorously
conducted and is free from bias.
Systematic reviews and library and
information professionals
The systematic review industry has offered
information professionals an opportunity to
play a significant supportive role within
evidence based practice, demonstrate their
traditional skills of finding and managing
information, and highlight their skills as expert
searchers (Beverley, Booth, and Bath 66).
Furthermore, librarians who are involved in the
production of systematic reviews, particularly in
healthcare, have demonstrated their ability as
evidence based library and information
practitioners by evaluating aspects of their
expert searching practice in order to provide an
evidence based for searching within systematic
reviews (Boynton et al.; McKibbon, Wilczynski
and Haynes; Sampson and McGowan; Sampson,
Evidence Based Library and Information Practice 2009, 4:1
44
McGowan et al.; Sampson, Zhang et al.;
Sampson, Barrowman et al.; Zhang, Sampson,
and McGowan).
But what about systematic reviews as a tool to
help library and information practitioners make
decisions about their practice? Systematic
reviews can help us make sense of our research
base, implement and justify services, and ensure
that individual practitioners do not have to
collect and analyse primary studies for every
decision made (McKibbon 205-207). However,
systematic reviews in library and information
science are relatively uncommon. A recent
paper located seven reviews published in library
and information science journals between 1996-
2006 (Ankem). Although this is not likely to be
the exact number of existing systematic reviews
in the domain, it provides an interesting
comparison to the 5000+ systematic reviews that
have been undertaken in health and social care
(Centre for Reviews and Dissemination, CRD
Databases http://www.crd.york.ac.uk/crdweb/).
Papers discussing librarians’ involvement in
systematic reviews tend to focus on the
searching elements (Falzon and Booth;
McGowan and Sampson; Scherrer and Dorsch;
Harris), although some discuss a wider role for
librarians (Beverley, Booth, and Bath; Swinkels,
Briddon, and Hall). In my own experience,
becoming involved as a team member and
providing literature searches for systematic
reviews gave me the confidence to undertake
systematic reviews of my own practice (Brettle
"Information Skills Training: A Systematic
Review of the Literature; Brettle "Evaluating
Information Skills Training in Health Libraries:
A Systematic Review"). This in turn provided
me with the knowledge and skills to become
more fully integrated within review teams
conducting systematic reviews at the boundaries
of health and social care (Brettle, Hill, and
Jenkins; Dugdill et al.; Hill and Brettle). This
involved critical appraisal, review management,
data synthesis, report writing, and providing
mentorship throughout the process to subject
specialists who have limited experience of
conducting systematic reviews.
McKibbon outlines the importance of systematic
reviews for librarians and provides a step by
step guide to their production (McKibbon
"Systematic reviews"). Furthermore, in the same
article, McKibbon notes a range of reasons for
undertaking systematic reviews including
summarising the volume of information
available, integrating information in areas where
there is little information available, resolving
discrepancies between studies with conflicting
results, planning for new research, and
providing teaching materials. However,
systematic reviews have most commonly been
used to address questions of effectiveness,
although they can be relevant to any area where
it is important to identify research trends or
determine the overall balance of evidence in
relation to a particular question (Macdonald).
They can also incorporate a range of research
designs (both quantitative and qualitative),
although these may present methodological and
practical challenges in finding and synthesising
literature for their inclusion (Mays, Pope, and
Popay).
Given that librarians already play a significant
role in finding information for systematic
reviews, and, according to the required
competencies of our professional bodies (Hunter
and Wake; Medical Library Association), require
the skills to carry out the appraisal and synthesis
parts of systematic reviews, why are more
systematic reviews not undertaken by librarians
to answer questions related to their own
practice? Furthermore, why do the majority of
librarians focus on the traditional searching
elements of the review process rather than the
critical appraisal?
Barriers to conducting systematic reviews
Time
Undoubtedly systematic reviews take time;
McKibbon (McKibbon "Systematic reviews")
estimates 600 hours for a narrowly focussed
Evidence Based Library and Information Practice 2009, 4:1
45
review. This time factor is a serious issue for
evidence based library and information practice,
and is not a feasible way forward for everyday
practitioners who wish to answer a pressing
question in their own environment. A debate
about the time involved and the need for
systematic reviews to advance EBLIP would
perhaps be useful in raising the issue up the
agenda of organisations who may provide funds
for library related systematic reviews.
Putting the issue of time to one side, there are
methodological issues that also need to be faced
if more systematic reviews are to be undertaken
in EBLIP. These are outlined below and arose
during the production of two systematic reviews
in evidence based library and information
practice (Brettle "Information Skills Training: A
Systematic Review of the Literature; Brettle
"Evaluating Information Skills Training in
Health Libraries: A Systematic Review").
What evidence should be included?
Systematic reviews which follow a medical
model of effectiveness, frequently include
controlled studies to provide high quality
evidence on “what works”. However, this type
of evidence is frequently lacking in the library
literature as was found in Brettle ("Information
Skills Training: A Systematic Review of the
Literature"). An alternative approach is to
include a range of evidence which “gains insight
into the complexity of interventions and
perspectives of users” (Long et al. 22). This
could include quantitative and qualitative
research studies using a range of designs; and
although this approach may not provide
definitive answers to questions of effectiveness
(according to a medical model of what
constitutes good evidence), it does provide a
summary of the evidence which exists, a
baseline from which to move forward and
highlights where and what research is needed.
All factors which are useful to practising
librarians. It has been argued that evidence
based library and information practice should be
based on a “core-centred” or all encompassing
model of research evidence, rather than a
hierarchical one (Crumley and Koufogiannakis
68). Using this model, it is appropriate for
systematic reviews in the library domain to take
a wide view of relevant evidence and include a
variety of designs appropriate to the topic or
review question at hand.
Assessing study quality
A related issue to deciding which studies to
include is determining the quality of studies
selected for inclusion in the review.
Methodologies such as randomised controlled
trials are viewed as “good quality evidence”
because of the ability of their design to reduce
bias in the study, assuring that the effects of the
intervention are due the intervention alone.
However, this does not automatically mean that
the study has been well conducted.
Furthermore, some questions or situations are
not suited to such study designs and would
more appropriately be addressed by using, for
example, a qualitative approach. A range of
critical appraisal tools exist that provide a means
of extracting detailed information and a means
of judging the quality and relevance of a study.
These may be library specific (Booth and Brice;
Glynn; Koufogiannakis, Booth and Brettle) or
adapted from other disciplines such as
healthcare, but do not offer a definitive answer
regarding the quality of a study. How does a
reviewer decide which studies to include in the
review – all those which fit the inclusion criteria,
or just those above a certain quality threshold?
If so, how is this threshold determined? One
approach is to limit the review to controlled
study designs as in a review by Garg and Turtle,
but this runs the risk of including a very limited
number of studies and reach limited
conclusions, which may not relate to practice.
Weightman and Williamson included a range of
designs and developed what they named a
“crude measure of quality,” based on
established research methodology and only
included those which were above this quality
threshold. This approach should ensure that the
conclusions were based on the best quality
evidence available, providing their definition of
a “good quality study” was reliable. But what
Evidence Based Library and Information Practice 2009, 4:1
46
does make a good library study? It would be
useful for further work to establish some kind of
consensus. By examining the quality of library
studies, it may be possible to highlight issues of
weak study design that can be addressed by
future research, and thus long term improve the
overall quality of library related research.
Searching issues
Handbooks and guidance on systematic review
production stress the need for comprehensive
searching to locate relevant studies (Centre for
Reviews and Dissemination; Higgins and
Green). Information specialists have conducted
a wide range of studies in relation to searching
health care resources to improve the
comprehensiveness and accuracy of searching
(McKibbon, Wilczynski, and Haynes; Sampson,
McGowan et al.). Within social care systematic
reviews, a number of studies have confirmed the
need to search widely and use a variety of
resources to ensure comprehensive coverage
(Brettle and Long; Golder, Mason, and
Spilsbury; McNally and Alborz). The library
literature parallels that of social care with
studies crossing a range of domains such as
library literature, education and management.
Thus it is likely that a range of databases need to
be searched in order to obtain comprehensive
coverage, but work needs to be undertaken to
quantify this and provide guidance for searchers
of the library related literature. This would
provide evidence and tools to underpin
evidence based library and information practice,
and redress the balance of librarians conducting
research into effective searching of resources
relevant to the practice of other professions, but
not that of their own!
A systematic review to address these issues
These issues related to systematic review
methodology are not limited to library and
information practice. Similar issues have been
found by the author in reviews that cross the
boundaries of health and social care and much
could be learned for library related systematic
reviews by examining the methodological
literature in these fields. What may be more
problematic for librarians is the time involved
and the skills needed to conduct systematic
reviews. All these issues are currently being
faced by a group of UK librarians who are
undertaking a systematic review related to
clinical librarianship (The North West Clinical
Librarian Systematic Review Group). The group
has come together (and successfully obtained a
small amount of funding) to conduct a review
which seeks to answer a question facing their
practice – that of how to evaluate their service.
UK NHS library policy guidelines have
recommended that libraries evaluate the impact
of their library services and that clinical librarian
(CL) services be implemented in all acute
hospitals (Hill). Evidence to date has been
inconclusive about the effectiveness of CL
services (Wagner and Byrd; Winning and
Beverley), but rigorous evaluations of CL
services will be needed to demonstrate their
value over time. Given the various models of
service provision, there is unlikely to be a “one
model fits all” approach to evaluating CL
services, therefore the review will examine these
issues and provide guidance for future
evaluations.
The group comprises seven (mainly) clinical
librarians from the North West region of the UK
and myself as a facilitator and researcher who
specialises in the conduct of systematic reviews.
At the first meeting of the group, the majority of
members suggested that their involvement in
the project was a practical opportunity for them
to gain research and critical appraisal skills as
well as insight into the complete systematic
review process that they often support.
Furthermore, operating as a group is a “safe
environment” and provides confidence and
support for those members who are less familiar
with research methods. In order to provide
insight into the process of library related
systematic reviews the group are maintaining
reflective diaries of the highs and lows of
involvement in the project and a record of the
time spent on the review. It is hoped that
publications arising from these can be used to
Evidence Based Library and Information Practice 2009, 4:1
47
advance systematic reviews in librarianship by
providing guidance for other librarians who
wish to conduct systematic reviews.
Conclusion
Systematic reviews are an important means of
summarising evidence and are useful for
librarians who are seeking to find evidence for
their practise. Furthermore, they provide an
opportunity for librarians to demonstrate and
develop their skills in expert searching and
critical appraisal. However, there are a limited
number of reviews in existence in library and
information practice – possibly due to the
practical and methodological challenges
involved in conducting them. As a profession
which is seeking to demonstrate their value and
improve their evidence base, it is important that
we seek to overcome these challenges, progress
research in this area, and provide a sound
evidence base for library and information
practice.
Note
The clinical librarian systematic review
mentioned in this commentary is seeking to
identify effective methods of evaluating clinical
librarian services from the published and grey
literature. If you have (or are aware of) any grey
literature reports which involve the evaluation
of a CL service, please get in touch with the
author of this article.
Works Cited
Ankem, Kalyani. "Evaluation of Method in
Systematic Reviews and Meta-Analyses
Published in LIS." Library &
Information Research 32.101 (2008): 91-
104.
Beverley, Catherine A., Andrew Booth, and
Peter. A. Bath. "The Role of the
Information Specialist in the Systematic
Review Process: A Health Information
Case Study." Health Information &
Libraries Journal 20.2 (2003): 65-74.
Booth, Andrew, and Anne Brice. "Research."
Health Information & Libraries Journal
18 (2001): 175-7.
Boynton, Janette, Julie Glanville, David McDaid,
and Carol Lefebvre. "Identifying
Systematic Reviews in Medline:
Developing an Objective Approach to
Search Strategy Design." Journal of
Information Science 24.3 (1998): 137-54.
Brettle, Alison, Andy Hill, and Peter Jenkins.
"Counselling in Primary Care: A
Systematic Review of the Evidence."
Counselling & Psychotherapy Research
8.4 (2008): 207-14.
Brettle, Alison. "Evaluating Information Skills
Training in Health Libraries: A
Systematic Review." Health Information
& Libraries Journal 24 (2007): 18-37.
---. "Information Skills Training: A Systematic
Review of the Literature." Health
Information & Libraries Journal
20.Suppl 1 (2003): 3-9.
Brettle, Alison J., and Andrew F. Long.
"Comparison of Bibliographic Databases
for Information on the Rehabilitation of
People with Severe Mental Illness."
Bulletin of the Medical Library
Association 89.4 (2001): 353-62.
Centre for Reviews and Dissemination.
"Undertaking Systematic Reviews of
Research on Effectiveness: CRD's
Guidance for Carrying out or
Commissioning Reviews." 2001.
University of York.
<http://www.york.ac.uk/inst/crd/report4
.htm>.
Evidence Based Library and Information Practice 2009, 4:1
48
Ciliska, Donna, Nicky Cullum, and Susan
Marks. "Evaluation of Systematic
Reviews of Treatment of Prevention
Interventions." Evidence Based Nursing
: An Introduction. Eds. Nicky Cullum, et
al. Oxford: Blackwell Publishing, 2008.
135-44.
Crumley, Ellen, and Denise Koufogiannakis.
"Developing Evidence-Based
Librarianship: Practical Steps for
Implementation." Health Information &
Libraries Journal 19.2 (2002): 61-70.
Droogan, Jacqueline, and Fujan Song. "The
Process and Importance of Systematic
Reviews." Nurse Researcher 4.1 (1996):
15-26.
Dugdill, Lindsey, Alison Brettle, Claire Hulme,
Serena McCluskey, and Andrew Long.
"Workplace Physical Activity
Interventions: A Systematic Review."
International Journal of Workplace
Health Management 1.1 (2008): 20-40.
Falzon, Louise, and Andrew Booth. "Realise-Ing
Their Potential?: Implementing Local
Library Projects to Support Evidence-
Based-Health Care." Health Information
& Libraries Journal 18 (2001): 65-74.
Garg, Anupama, and Kathleen M. Turtle.
"Effectiveness of Training Health
Professionals in Literature Search Skills
Using Electronic Health Databases: A
Critical Appraisal." Health Information
& Libraries Journal 20.1 (2003): 33-41.
Glynn, Lindsay. "A Critical Appraisal Tool for
Library and Information Research."
Library Hi Tech 24.3 (2006): 387-99.
Golder, Su, Anne Mason, and Karen Spilsbury.
"Systematic Searches for the
Effectiveness of Respite Care." Journal of
the Medical Library Association 96.2
(2008): 147-52.
Harris, Martha R. "The Librarian’s Roles in the
Systematic Review Process: A Case
Study." Journal of the Medical Library
Association 93.1 (2005): 81-7.
Higgins, Julian P.T., and Sally Green, eds.
Cochrane Handbook for Systematic
Reviews of Interventions. Version 5.0.1
[updated September 2008]. The
Cochrane Collaboration, 2008.
<http://www.cochrane-handbook.org>.
Hill, Andrew, and Alison Brettle. "The
Effectiveness of Counselling with Older
People: Results of a Systematic Review."
Counselling & Psychotherapy Research
5.4 (2005): 265-72.
Hill, Peter. "Report of a National Review of Nhs
Library Services in England: From
Knowledge to Health in the 21st
Century." 2008. National Library for
Health. 5th May 2008
<http://www.library.nhs.uk/aboutnlh/re
view>.
Hunter, Tracey, and Michelle Wake. "Guide to
Working in Health Information: Revised
Edition." Chartered Institute of Library
and Information Professionals: Health
Libraries Group, 2005. 16 December
2008
<http://www.cilip.org.uk/specialinterest
groups/bysubject/health/publications/w
orkguide/>.
Koufogiannakis, Denise, Andrew Booth, and
Alison Brettle. "Reliant: Reader's Guide
to the Literature on Interventions
Addressing the Need for Education and
Training." Library & Information
Research 30.94 (2006): 44-51.
Long, Andrew F., Mary Godfrey, Tracy Randall,
Alison Brettle, and Maria Grant.
Feasibility of Undertaking Systematic
Reviews in Social Care. Leeds:
University of Leeds, Nuffield Institute
Evidence Based Library and Information Practice 2009, 4:1
49
for Health. University of Salford,
Health Care Practice R&D Unit, 2002.
Macdonald, Geraldine. Using Systematic
Reviews to Improve Social Care.
London: Social Care Institute for
Excellence (SCIE), 2003.
Mays, Nicholas, Catherine Pope, and Jennie
Popay. "Systematically Reviewing
Qualitative and Quantitative Evidence
to Inform Management and Policy-
Making in the Health Field." Journal of
Health Services Research & Policy
10.Suppl 1 (2005): 6-20.
McGowan, Jessie, and Margaret Sampson.
"Systematic Reviews Need Systematic
Searchers." Journal of the Medical
Library Association 93.1 (2005): 74-80.
McKibbon, K. Ann. "Systematic Reviews and
Librarians." Library Trends 55.1 (2006):
202-15.
McKibbon, K. Ann, and Nancy L. Wilczynski.
"Retrieving Randomized Controlled
Trials from Medline: A Comparison of
38 Published Search Filters." Health
Information & Libraries Journal (2008).
Accessed 12 December 2008.
McNally, Rosalind, and Alison Alborz.
"Developing Methods for Systematic
Reviewing in Health Services Delivery
and Organization: An Example from a
Review of Access to Health Care for
People with Learning Disabilities. Part
1. Identifying the Literature." Health
Information & Libraries Journal 21.3
(2004): 182-92.
Medical Library Association. "Medical Library
Association: Educational Policy:
Professional Competencies for Health
Sciences Librarians". (2007): Web page
from professional organisation. Medical
Library Association. 28 April 2008.
<http://www.mlanet.org/education/polic
y/executive_summary.html>.
Sampson, Margaret., et al. "Can Electronic
Search Engines Optimize Screening of
Search Results in Systematic Reviews:
An Empirical Study." BMC Medical
Research Methodology 6:7 (2006). <
http://www.biomedcentral.com/1471-
2288/6/7>
Sampson, Margaret, and Jessie McGowan.
"Errors in Search Strategies Were
Identified by Type and Frequency."
Journal of Clinical Epidemiology 59.10
(2006): 1057-63.
Sampson, Margaret, et al. "Managing Database
Overlap in Systematic Reviews Using
Batch Citation Matcher: Case Studies
Using Scopus." Journal of the Medical
Library Association 94.4 (2006): 461-3,
e219.
---. "An Alternative to the Hand Searching Gold
Standard: Validating Methodological
Search Filters Using Relative Recall."
BMC Medical Research Methodology
6:33 (2006).
<http://www.biomedcentral.com/1471-
2288/6/33>
Scherrer, Carol S., and Josephine L. Dorsch. "The
Evolving Role of the Librarian in
Evidence Based Medicine." Bulletin of
the Medical Library Association 87.3
(1999): 322-8.
Swinkels, Annette, Jason Briddon, and Jane Hall.
"Two Physiotherapists, One Librarian
and a Systematic Literature Review:
Collaboration in Action." Health
Information & Libraries Journal 23
(2006): 248-56.
Wagner, Kay C., and Gary D. Byrd. "Evaluating
the Effectiveness of Clinical Medical
Librarian Programs: A Systematic
Evidence Based Library and Information Practice 2009, 4:1
50
Review of the Literature." Journal of the
Medical Library Association 92.1 (2004):
14-33.
Weightman, Alison L., and Jane Williamson.
"The Value and Impact of Information
Provided through Library Services for
Patient Care: A Systematic Review."
Health Information and Libraries
Journal 22.1 (2005): 4-25.
Winning, M. Alison, and Catherine. A. Beverley.
"Clinical Librarianship: A Systematic
Review of the Literature." Health
Information & Libraries Journal 20
Suppl 1 (2003): 10-21.
Zhang, Li, Margaret Sampson, and Jessie
McGowan. "Reporting the Role of the
Expert Searcher in Cochrane Reviews."
Evidence Based Library & Information
Practice 1.4 (2006): 3-16.