Microsoft Word - ART_Kouf.doc
Evidence Based Library and Information Practice 2006, 1:3
3
Evidence Based Library and Information Practice
Article
Effective Methods for Teaching Information Literacy Skills to Undergraduate Students:
A Systematic Review and Meta‐Analysis
Denise Koufogiannakis
Collections and Acquisitions Coordinator
University of Alberta Libraries
Edmonton, Alberta, Canada
E‐mail: denise.koufogiannakis@ualberta.ca
Natasha Wiebe
Research Associate/Statistician
University of Alberta
Edmonton, Alberta, Canada
E‐mail: natasha.wiebe@ualberta.ca
Received: 31 May 2006 Accepted: 02 August 2006
© 2006 Koufogiannakis and Wiebe. This is an Open Access article distributed under the terms of the
Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits
unrestricted use, distribution, and reproduction in any medium, provided the original work is properly
cited.
Structured Abstract
Objective ‐ The objective of this systematic review was to assess which library instruction
methods are most effective for improving the information skills of students at an
introductory, undergraduate level, using cognitive outcomes (measuring changes in
knowledge). The study sought to address the following questions:
1) What is the overall state of research on this topic?
2) Which teaching methods are more effective?
Methods ‐ This project utilised systematic review methodology. Researchers searched
fifteen databases and retrieved 4,356 potentially relevant citations. They reviewed the titles
and abstracts for relevance, and of those, 257 complete articles were considered in‐depth
using a predetermined inclusion/exclusion form. There were 122 unique studies that met
the inclusion criteria and were subjected to an extensive data extraction and critical
appraisal process. Of these studies, 55 met author‐defined quality criteria to provide
information on the effectiveness of different teaching methods. From this review there was
http://creativecommons.org/licenses/by/2.0
Evidence Based Library and Information Practice 2006, 1:3
4
a final group of 16 studies with sufficient information to enable meta‐analyses and
calculations of standardized mean differences.
Results ‐ The overwhelming majority of studies were conducted in the United States (88%).
Experimental or quasi‐experimental research methods were used in 79 studies (65%).
Teaching methods used in the studies varied, with the majority focused on traditional
methods of teaching, followed by computer assisted instruction (CAI), and self‐directed
independent learning (SDIL). Studies measured outcomes that correlated with Bloom’s
lower levels of learning (‘Remember’, ‘Understand’, ‘Apply’).
Sixteen studies compared traditional instruction (TI) with no instruction, and twelve of
those found a positive outcome. Meta‐analysis of the data from 4 of these studies agreed
with the positive conclusions favouring TI. Fourteen studies compared CAI with traditional
instruction (TI), and 9 of these showed a neutral result. Meta‐analysis of 8 of these studies
agreed with this neutral result. Another group of 6 studies compared SDIL with no
instruction, and meta‐analysis of 5 of these agreed that the result was positive in favour of
SDIL.
Conclusion ‐ Based on the results of the meta‐analysis, there is sufficient evidence to
suggest that CAI is as effective as TI. Evidence also suggests that both TI and SDIL are more
effective than no instruction. Additional comparative research needs to be done across
different teaching methods. Studies comparing active learning (AL), CAI, and SDIL would
greatly enrich the research literature. Further studies utilizing appropriate methodologies
and validated research tools would enrich our evidence base, and contribute to the growth
of knowledge about effectiveness of particular teaching methods.
Introduction
Information literacy is a topic of great
interest in the field of library and
information studies, particularly among
academic librarians, who view teaching as
an important role (Baruchson‐Arbib and
Bronstein; Godwin; Peacock). The National
Forum on Information Literacy defines
information literacy as “the ability to know
when there is a need for information, to be
able to identify, locate, evaluate, and
effectively use that information for the issue
or problem at hand.” Librarians are
constantly looking to improve the methods
by which they teach information skills to
undergraduate students, in order to increase
the students’ competencies in this area. The
professional literature is populated with
articles on the topic, ranging from research
studies, new innovations, and tales of
implementation in various settings. Most
librarians have their own stories of teaching
successes and failures which they pass on to
others. However, while there have been
attempts to systematically appraise the
literature as it pertains to health
professionals (Brettle; Garg and Turtle), the
general library research literature of this
field has not been gathered or summarized
in a systematic way that would facilitate an
evidence based approach towards
undergraduate level instruction (i.e., using
evidence to inform and support information
literacy initiatives). This study was an
attempt to sort through the published
literature and move beyond conjecture
surrounding the most effective methods of
teaching information literacy skills, via a
systematic review of the library research
literature relating to undergraduate teaching.
The objective of this review was to assess
which library instruction methods are most
Evidence Based Library and Information Practice 2006, 1:3
5
effective for improving the information
skills of students at an introductory,
undergraduate level, using cognitive
outcomes (measuring changes in
knowledge). Cognitive outcomes may be at
varying levels, from simply remembering
facts, to applying what was taught in a new
situation, to creating new content. It should
be noted that cognitive outcomes are only
one aspect that may be considered when
determining the success of an information
literacy program. Behavioural (measuring
changes in actions) and affective (measuring
changes in attitudes or values) outcomes are
other aspects that may be considered. This
review, however, did not attempt to
encompass all areas of research, and focused
solely on cognitive outcomes.
The study sought to address the following
questions:
1) What is the overall state of research
on this topic?
2) Which teaching methods are more
effective?
In addition, the following hypotheses were
postulated:
1) Instruction that is taught by a
librarian face‐to‐face is more
effective than instruction that is
computer‐based.
2) Instruction that encourages active
participation from students is more
effective than passive instructional
modes, such as lectures and
demonstrations.
3) Using Bloom’s taxonomy, the higher
the level of learning outcomes
measured, the more difficult it will
be to link effectiveness directly to
library instruction, and the less
likely it is that the library
instruction will result in a positive
outcome.
In terms of evidence based library and
information practice, the goal of conducting
this systematic review was to find evidence
on information literacy instructional
methods that may have a direct impact on
the way academic librarians approach
information literacy instruction for
undergraduate students.
Methods
The research methodology used was a
systematic review, including an extensive
literature search, an inclusion and exclusion
process for potential studies, extraction of
data from the included studies, and analysis
of that data.
Once the study objectives and hypotheses
had been determined, inclusion and
exclusion criteria were developed.
Categories of data required to address the
review objectives and hypotheses were also
noted for future data extraction. These
predetermined criteria helped frame the
search process.
Fifteen databases (LISA, Library Literature,
ERIC, Inspec, Academic Search Premier,
Educational Research Abstracts, CINAHL, Web
of Science, Dissertation Abstracts, Conference
Papers Index, SIGLE, CERUK, Education‐Line,
British Education Index, and Australian
Education Index) were searched for relevant
articles in the Fall of 2004. The searches
(Appendix A) were updated in May 2005,
and results combined in a single
bibliography. Researchers also checked
related bibliographies, literature reviews,
and references cited in these articles.
For a study to be included in the systematic
review, it had to meet the following criteria:
• Instruction had to be led by a librarian
or library assistant (or with librarians as
part of the instruction team) for a class
or stand‐alone session utilising any
instruction method.
Evidence Based Library and Information Practice 2006, 1:3
6
• Study subjects had to be undergraduate
students at a post‐secondary academic
institution.
• The research study had to have an
evaluative component that measured
the cognitive outcome effect of
instruction on student learning via some
test of information literacy (e.g., pre‐
and post‐tests, graded papers, or
bibliographies).
Included studies were not limited by
publication date, but were limited to the
English language. Studies were not
excluded on the basis of quality or study
methodology.
As is shown in Figure 1, 4,356 potentially
relevant citations were retrieved from the
literature search, and titles and abstracts
were reviewed for significance. Of those, 257
complete articles were considered in‐depth
using a predetermined inclusion/exclusion
form, with 108 meeting inclusion criteria.
Another 17 articles that met inclusion
criteria were identified by checking
reference lists and review articles on the
topic.
After eliminating three studies where
articles had been reported in two source
publications, there were 122 unique studies
that met the inclusion criteria and
underwent an extensive data extraction and
critical appraisal process. Appendix B
contains the list of studies reviewed.
Researchers entered data extraction
elements (Appendix C) in an Excel®
spreadsheet.
Categories for data extraction were tested on
a subset of 8 articles included in the
systematic review. Studies were critically
appraised using the checklist developed by
Morrison et al. (891). The checklist consists
of nine questions, focused on the validity
and applicability of the study being
appraised. The 122 studies were used to
present the results relating to the first
objective, namely the overall state of
research on the topic.
Following the data extraction and critical
appraisal process undertaken by DK, a
methodological quality filter was applied to
all 122 studies. Those that had a
comparative study design and compared
two different teaching methods, and whose
outcomes were based on tests of statistical
significance (n=55) were analyzed to
determine the results of effectiveness for
different teaching methods. A meta‐
analysis was conducted, by NW, on 16 of
these studies to substantiate the findings
relating to the effectiveness of different
teaching methods. The meta‐analysis section
of this paper describes this approach in
more detail.
Results
The results are presented in three parts. The
first part addresses the first research
objective: to provide an overview of the
state of research on this topic with a
description of the studies included. Part two
provides further analysis on a subsection of
the results of those studies to examine the
second study objective and determine which
teaching methods are most effective. Part
three details the meta‐analysis on specific
aspects of effectiveness, where the data
enabled this method to be used.
I. Status of Research
Description of Included Studies
The majority (78%, 97/125) of the studies
were published as journal articles (Table 1),
with large numbers coming from three main
journals, College & Research Libraries, Research
Strategies, and Journal of Academic
Librarianship. Publications were spread over
a time period spanning from 1963 to 2005,
with most publications (35%, 44/125) coming
from the current decade. The 1980s
Evidence Based Library and Information Practice 2006, 1:3
7
Figure 1: Flow Diagram of the Systematic Review Process
Potentially eligible citations
identified by literature search
(n=4,356)
Articles subjected to an in‐depth
review (n=257)
Citations excluded at stage 1
(n= 4,099)
Articles found outside of structured
literature search that met inclusion
criteria (n=17)
Articles excluded at stage 2 (n=142);
Unable to obtain (n=7)
Total number of articles included
in the systematic review (n=125)
Articles found through literature
search that met inclusion criteria
(n=108)
Total number of research studies
included in the systematic review
(n=122)
Publication overlap whereby
the same study was
published in more than one
source (n=3)
Total number of research studies
that met quality filter criteria
(n=55)
Total number of studies that were
suitable for meta‐analysis (n=16)
See Results, section I
See Results, section II
See Results, section III
Evidence Based Library and Information Practice 2006, 1:3
8
Type of publication Number
Journal articles 97
College & Research Libraries (20)
Research Strategies (14)
Journal of Academic Librarianship (11)
Other journals (52)
ERIC documents 19
Dissertations 8
Book Chapters 1
Total Publications 125
Table 1: Publication Types
produced nearly the same amount of
research on the topic (33%, 41/125). In the
1990s the numbers declined (21%, 26/125),
but there was a resurgence in the present
decade.
The vast majority (88%) of studies (107/122)
were conducted in the United States. Other
countries included were Canada (6.5%,
8/122) studies and Australia (4%, 5/122). The
UK, and Trinidad and Tobago each
contributed one study. The dominance of
studies from the United States is
overwhelming. Certainly other countries,
such as the United Kingdom and Australia,
have strong academic information literacy
programs operating. This may indicate that
the focus of research on the topic of
information literacy has been different in
other countries, and that the specific focus of
this systematic review has been dominated
by U.S. researchers. Perhaps researchers in
other countries have focused on more
qualitative aspects, rather than trying to
measure effectiveness based on cognitive
outcomes.
Study types
The research studies in this systematic
review all used quantitative research
methods to measure cognitive outcomes.
Some studies also employed qualitative
methods to measure other types of
outcomes, but those are outside the scope of
this review. Figure 2 provides a general
breakdown of study types. Most studies
were quasi‐experimental, employing a
controlled study design, but without
randomising the students to teaching
groups. Fifty‐nine of the 122 (48%) studies
fit into this categorization, including
controlled before‐and‐after studies, as well
as studies that were post‐intervention,
single time point with a control group.
Twenty (16%) of the studies were
experimental, employing randomisation and
a control group. The majority of these used
cluster randomisation, since educational
groups were often pre‐formed, and the
randomisation of individuals was beyond
the researcher’s control. Some studies were
pre‐experimental, since they did not have a
comparison or control group and focused on
exploring the change in one group of
students, using a pre‐ and post‐test (i.e.,
before‐and‐after study). Nine studies (7%)
were observational, including longitudinal
and cross‐sectional studies. The single
‘other’ study was large and
multidimensional in nature, which could
not be classified into any of the other
categories.
The studies had varying evaluation periods,
with the majority (51%, 62/122) covering a
time period of one semester.
Evidence Based Library and Information Practice 2006, 1:3
9
20
59
9
1
33
0
10
20
30
40
50
60
70
Experimental Quasi-
experimental
Pre-experimental Observational Other
Figure 2: Study Types
One group (7.4%, 9/122) of studies took
place over the course of one year, and a
further 8/122 (6.6%) took place over 2
semesters. The remaining studies’
evaluation periods ranged from 1 day to 6
years. Studies varied with respect to when
learning outcomes were measured. The
largest number (28%, 34/122) tested learning
outcomes at the end of the semester.
Another group of studies (25%, 30/122)
tested immediately following an instruction
session. The remainder ranged from the next
class following instruction, to several weeks
after instruction, to 3 years after the course
was completed. There were 17 studies (14%)
that did not report this variable.
41
16 16 15
11 11
7 5 3 1
7
0
5
10
15
20
25
30
35
40
45
Ar
ts
Lib
ra
ry
sc
ien
ce
Sc
ien
ce
s
He
alt
h s
cie
nc
es
So
cia
l s
cie
nc
es
Un
ive
rs
ity
in
tro
du
cti
on
Ed
uc
ati
on
Bu
sin
es
s
Co
mm
un
ica
tio
ns La
w
No
t s
ta
te
d /
no
t a
pp
lic
ab
le
Figure 3: Subject Areas Covered
Evidence Based Library and Information Practice 2006, 1:3
10
Teaching Methods
Active Learning Students are actively engaged in their own learning, with
the instructor taking on a facilitation role.
Computer Assisted Instruction A computer is used to deliver the instruction directly to
the student.
Learner‐centred instruction Focus is on the individual student’s unique learning
needs.
Self directed, independent
learning
Learning in which the individual has primary
responsibility for his or her education.
Traditional instruction Instructional material is transmitted to students from
teachers, and is a passive method of learning for students.
Table 2: Definitions of Teaching Methods
Academic subject areas covered
The 122 studies represented a wide range of
undergraduate subject areas. Most covered a
single subject area, but others included more
than one discipline. The highest percentage
of studies came from instruction related to
courses in English (30%, 37/122), included in
the Arts category. Another subject area with
many studies was Library Science, which
were mainly courses devoted to library
skills and taught by librarians for credit
(13%, 16/122). Figure 3 shows the number of
studies arranged by broad subject discipline.
Teaching methods
Most studies compared two or more
teaching methods. Of the 122 studies, 88
(72%) were comparative, while the
remaining 34 (28%) studies did not have a
comparison group. These non‐comparative
studies were generally evaluating the
effectiveness of a single teaching method to
determine whether students’ scores
improved following the instruction.
The teaching methods (Table 2) used in the
studies varied, with the majority focused on
traditional methods of teaching (e.g., lecture,
demonstration). Other studies dealt with
computer‐assisted instruction (CAI) (e.g.,
Web‐based tutorials); self‐directed,
independent learning (SDIL) (e.g.,
workbooks); active learning (AL) (e.g.,
problem based learning); and learner‐
centred instruction (LCI) (e.g., individual
term paper counselling). While many
teaching methods included a combination of
methods, for the purposes of this review
they were grouped according to the primary
teaching method used (Figure 4). Further
details of the studies meeting the quality
filter are in Appendix D.
Total librarian contact time with students
ranged from 15 minutes to 3 hours per week
over the course of a semester. Information
on the amount of instructional contact time
was lacking in 27% (33/122) of the studies.
Instructional topics varied among studies,
although many common themes were
identified. The most common areas for
instruction were conducting library research
and research strategies (33%, 40/122), using
the catalogue (28%, 34/122), using reference
tools (27%, 33/122), an overview of the
library and its resources (21%, 26/122
studies), literature searching (20%, 24/122),
and using computerized or electronic
resources (19%, 23/122).
Evidence Based Library and Information Practice 2006, 1:3
11
58
22 21
14
3 1 3
0
10
20
30
40
50
60
70
Traditional Computer-
assisted
instruction
Self-directed,
independent
learning
Active
learning
Learner-
centred
Various
methods
Unable to
determine
Figure 4: Primary Teaching Methods Used
Outcome measures
The vast majority of studies measured
cognitive outcomes via student scores on a
post‐test (79%, 96/122). Other outcome
measures included the graded quality of the
students’ bibliographies (7%,9/122), an
assessment of search tasks (6.5%,8/122),
assignment scores (5%,6/122), scores on term
papers or essays (3%, 4/122), and scores on a
general library skills survey (3%, 4/122).
Some studies used more than one cognitive
outcome measure.
Bloom’s taxonomy of educational objectives
(as revised by Anderson and Krathwohl)
was used to determine the level of cognitive
learning outcomes measured for the 122
included studies. The Anderson and
Krathwohl revision was used because it
provides an important update of Bloom’s
1956 taxonomy and incorporates new
knowledge into the framework. Anderson
and Krathwohl note the following structure
of the cognitive process, covering six levels
of learning:
1.0 Remember – Retrieve relevant
knowledge from long‐term
memory.
1.1 Recognizing
1.2 Recalling
2.0 Understand – Construct
meaning from instructional
messages, including oral,
written, and graphic
communication.
2.1 Interpreting
2.2 Exemplifying
2.3 Classifying
2.4 Summarizing
2.5 Inferring
2.6 Comparing
2.7 Explaining
3.0 Apply – Carry out or use a
procedure in a given situation.
3.1 Executing
3.2 Implementing
4.0 Analyze – Break material into
constituent parts and determine
how parts relate to one another
and to an overall structure or
purpose.
4.1 Differentiating
4.2 Organizing
Evidence Based Library and Information Practice 2006, 1:3
12
4.3 Attributing
5.0 Evaluate – Make judgements
based on criteria and standards.
5.1 Checking
5.2 Critiquing
6.0 Create – Put elements together
to form a coherent or functional
whole; reorganize elements into
a new pattern or structure.
6.1 Generating
6.2 Planning
6.3 Producing
(Anderson and
Krathwohl 31)
Figure 5 shows the levels of learning
outcomes measured by the 122 studies in
this review. More than one level of learning
was often assessed within a single study,
with ‘Remember’ being the most commonly
assessed area. Most studies focused on one
of the three more basic levels of learning
outcomes. There is an evident lack of
cognitive assessment research in the higher
order areas ‐‐ ‘Analyze’, ‘Evaluate’, and
‘Create’. Also of note is that 23 studies (18%)
did not provide enough information about
what they were testing to determine what
learning outcomes were being measured.
The research hypothesis for this study
anticipated that the higher the level of
learning outcomes measured (using Bloom’s
taxonomy), the more difficult it will be to
link effectiveness directly to library
instruction, and the less likely it is that the
library instruction will result in a positive
outcome. Results of this systematic review
indicate the hypothesis is partially
supported. The higher levels of learning
outcomes were less frequently measured,
suggesting a possible difficulty in testing
these areas. Furthermore, the results in the
studies that did measure higher level
outcomes varied widely, with a greater
percentage of those studies having mixed
results. The levels ‘Remember’,
‘Understand’, and ‘Apply’ seem to be easier
to measure in terms of cognitive outcomes,
while ‘Analyze’, ‘Evaluate’, and ‘Create’ are
more complex concepts that do not lend
themselves as well to quantitative
measurement. As a result, the focus of
outcome measures is more evident in areas
where demonstrable outcomes can be
achieved.
62
49
56
21
7
1
23
0
10
20
30
40
50
60
70
Remember Understand Apply Analyze Evaluate Create Unable to
determine
Figure 5: Number of Studies Assessing the Bloom (revised) Levels of Learning Outcomes
Evidence Based Library and Information Practice 2006, 1:3
13
In the areas of ‘Analyze’, ‘Evaluate’, and
‘Create’ there are more variables that cannot
easily be separated and tested, so it is more
difficult for students’ learning to be
attributed directly to the library‐led
instruction they received.
II. Effectiveness of Different Teaching
Methods
Reported outcomes by teaching method
The second objective of this review was to
determine effective teaching methods. In
order to do this, studies were categorised
and reviewed with a quality filter. Based on
the primary teaching method, studies were
categorised into five teaching methods:
active learning (AL), computer assisted
instruction (CAI), learner‐centred (LC), self‐
directed independent learning (SDIL), and
traditional instruction (TI). Four studies
could not be classified according to teaching
method.
Study results were recorded to indicate if
the intervention had a positive, negative,
neutral, or mixed result, based upon the
outcomes measured. Studies were subjected
to two quality filters. The first determined
whether the study was comparative (a
design whereby the intervention is
compared to another teaching method or
control). The second quality filter
determined whether the study outcome as
noted by the authors was based on data
assessed for statistical significance. These
quality filters were selected to remove bias
due to confounding (control groups should
be similar in every respect to the
experimental group except for the variable
being tested) and to account for differences
due to chance (statistical significance). A
total of 73 studies met these two criteria.
One additional study (Ridgeway), while not
using tests of statistical significance, did
have enough information for such tests to be
calculated, so it was included as well,
bringing the total to 74. However, 19 of
these compared different modes of delivery
or other aspects within the same overall
teaching method, so they were removed
from the analysis, since they did not
compare different teaching methods.
The 55 remaining studies compared the
main teaching intervention to a different
teaching method, or to no instruction, and
those 55 studies form the basis of the
following review and meta‐analysis in order
to draw conclusions about the effectiveness
of different teaching methods. Selected
characteristics of the 55 studies are noted in
Appendix D.
Traditional instruction
The traditional method of instruction (TI)
was the main intervention for the highest
number of studies in general (n=58).
However, only 18 of these met the
aforementioned quality criteria. Table 3
provides the number of study outcomes for
TI, with a specific breakdown by the
comparison teaching method.
Traditional Instruction (n=18)
Comparison Positive Neutral Negative Mixed Total
No instruction (NI) 12 3 0 1 16
CAI 0 0 0 1 1
Placebo 0 0 0 1 1
Table 3: Study Outcomes for Traditional Instruction
Evidence Based Library and Information Practice 2006, 1:3
14
Computer Assisted Instruction (n=17)
Comparison Positive Neutral Negative Mixed Total
Traditional 3 9 2 0 14
Self‐directed 0 2 0 0 2
No instruction 0 1 0 0 1
Table 4: Study Outcomes for Computer Assisted Instruction
The single study comparing TI against a
placebo (an inactive treatment given instead
of the treatment being evaluated, in this case
a non‐instructional film), had a mixed result.
Although TI accounts for much of the
research literature, there is a lack of
comparative research to determine
effectiveness versus other teaching methods.
Where the comparison is versus no
instruction, the studies reviewed here show
a mainly positive result, with 12 of the 16
reporting an affirmative outcome in favour
of TI, suggesting that this instructional
method is more effective than no instruction.
Within this subset of data, there was
sufficient information in 4 studies
comparing TI with no instruction, to
perform meta‐analysis and further
substantiate this finding, as explained in
Section III.
Computer Assisted Instruction
There were 22 studies for which computer
assisted instruction (CAI) was the main
intervention. Of these, 17 met the study
criteria (see Table 4), including 14
comparing CAI against TI, 2 comparing CAI
against SDIL, and 1 where CAI was
compared to no instruction. The group of 14
studies comparing CAI with TI is the largest
single grouping of studies found in this
systematic review. Based on the reported
outcomes of these CAI studies, the overall
neutral result points toward CAI being just
as effective as traditional teaching methods.
Eight of these were selected for meta‐
analysis (see Section III).
Self Directed, Independent Learning
A total of 21 studies focused on SDIL as the
main intervention, with 13 meeting the
aforementioned criteria; 7 studies
comparing SDIL with TI and 6 comparing
SDIL with NI (see Table 5). The overall
positive and neutral outcomes suggest that
SDIL is as effective as TI and more effective
than NI. Four of the studies comparing SDIL
to no instruction met the study criteria for
the meta‐analysis, discussed in Section III.
Self‐Directed Independent Learning (n=13)
Comparison Positive Neutral Negative Mixed Total
Traditional 2 5 0 0 7
No instruction 5 1 0 0 6
Table 5: Study Outcomes for Self‐directed, Independent Learning
Evidence Based Library and Information Practice 2006, 1:3
15
Active Learning (n=5)
Comparison Positive Neutral Negative Mixed Total
Traditional 1 1 1 0 3
No instruction 1 0 0 1 2
Table 6: Study Outcomes for Active learning
Active Learning
Fourteen studies focused on active learning
(AL). Of those, 5 met the criteria for further
analysis ‐‐ 3 compared AL with TI, and 2
compared AL with NI (see Table 6). Results
varied widely and indicate the studies
pertaining to AL are not conclusive and
require further research. The hypothesis,
“Instruction that encourages active
participation from students is more effective
than instructional modes that are passive,
such as lecture and demonstration,” could
not be addressed and meta‐analysis could
not be performed.
Learner Centred Instruction
Finally, three studies focused on learner‐
centred instruction (LCI) as the main
intervention. Two met the criteria for further
analysis; one comparing LCI to TI and the
other to no instruction, both with a neutral
result (see Table 7). The small numbers of
research studies in this area prevent further
analysis. Further research is required to
build the evidence base with respect to
effectiveness of LCI.
III. Meta‐analysis of comparative studies
A meta‐analysis is the pooling of estimates
of effect from individual studies asking the
same basic question. Meta‐analysis
provides an estimate of overall effect as well
as measuring the variability between these
studies. Additionally, the meta‐analysis will
weight the estimates based on how many
subjects each study tested and on how much
between‐subject variability existed. The
purpose of meta‐analysis is to provide some
guidance (ruling out chance and as much
bias as possible) around discrepant studies
and amongst small studies with
indeterminate results. Further information
about conducting meta‐analysis can be
found in Chalmers et al, Cooper and Hedges,
and Egger et al. Meta‐analysis was
performed on 16 studies for these
comparisons:
• Traditional instruction vs. no
instruction (n=4)
• Computer assisted instruction vs.
traditional instruction (n=8)
• Self‐directed independent learning
vs. no instruction (n=4)
Learner‐centred instruction (n=2)
Comparison Positive Neutral Negative Mixed Total
Traditional 0 1 0 0 1
No instruction 0 1 0 0 1
Table 7: Study Outcomes for Learner‐centred Instruction
Evidence Based Library and Information Practice 2006, 1:3
16
These three comparisons were selected
because the studies in these areas reported
enough information to calculate a
standardized mean difference (SMD), a
measure by which the studies could be
compared directly despite the use of
different measurement tools. SMD is the
difference in means of the two groups being
compared, divided by an estimate of
standard deviation (SD). The number of
participants in both the intervention and
control groups, the mean score for each
group, and the standard deviation was used
to calculate SMD.
The meta‐analysis allows for a visual
representation of the research data in a
meta‐graph using RevMan software (see
Figures 6, 7, and 8). Each row in the meta‐
graph constitutes one study. This includes
the raw data (here, the sample size, mean,
and standard deviation for each teaching
method) as well as the estimate of effect
(here, the SMD and the confidence intervals).
The best estimate of effect for each study is
represented by the square, and the
horizontal line running through the square
represents the confidence interval. The
vertical zero line is the line of no effect. If
the confidence interval crosses this line, the
estimate of effect is not significant. The
diamond at the bottom of the graph
represents the overall pooled estimate. Its
centre is the best estimate of effect, and its
width represents the pooled confidence
intervals.
This meta‐analysis used standardized mean
difference (SMD), the difference in means of
the two groups being compared, divided by
an estimate of standard deviation (SD). The
pooled SD from both instructional groups
was used. The SMD is in units of SD. For
example, as shown in Figure 7, the SMD for
Alexander’s study is 0.27 SDs. Therefore,
Alexander found a difference of 0.27 SDs
between the two groups, favouring CAI.
However, this result was not statistically
significant, since the 95% confidence interval
(‐0.15 to 0.69 SDs) includes zero. This means
that although the best estimate is 0.27 SDs,
the true difference may lie anywhere
between ‐0.15 and 0.69 SDs, including zero,
the estimate of no effect. An SMD of 0.2 is
said to be small, 0.5 to be moderate, and 0.8
to be large (Cohen 25).
The three meta‐analyses show the
individual estimates of each study plus the
overall pooled result. There are different
statistical methods for pooling. We chose a
method that accounts for random effects
between individual studies. This means that
not only did we expect subject‐to‐subject
variation, but also study‐to‐study variation,
such as differences due to type of computer
programs, teaching styles, evaluation
periods, and so on. Including random effects
in the overall estimate of effect widens the
confidence intervals, thereby increasing the
uncertainty of the estimate.
Traditional vs. No Instruction
Of the studies comparing TI to no
instruction, 4 provided sufficient
information to calculate SMD. Three of the
four studies (Cooper Moore, Lechner, and
Toifel) provided multiple TI vs. NI
comparison groups within their studies, and
these were included in the meta‐analysis as
separate comparisons. This information is
summarized in the meta‐graph in Figure 6.
The meta‐graph shows that for this group of
studies, there is a statistically significant
difference between the TI and no instruction
groups, suggesting that the TI method is
more effective than no instruction.
Additionally, since the confidence interval
stretches from 0.14 to 0.48, we would
conclude that the size of effect would be
somewhere between small and moderate,
using Cohen’s guidelines.
Evidence Based Library and Information Practice 2006, 1:3
17
Figure 6: Standardized Mean Difference of Traditional Instruction vs. No Instruction
Computer Assisted Instruction vs. Traditional
Instruction
The CAI subset of data included 8 studies
that were able to be compared using meta‐
analysis, since all 8 compared CAI with TI,
and they provided sufficient information to
calculate the standardized mean difference
(SMD). This information is summarized in
Figure 7. The pooled SMD estimate is ‐0.09
SDs, and the 95% confidence intervals, ‐0.47
to 0.29 SDs, cross the zero line, allowing for
the possibility of no difference between
groups. Note also that our best estimate, ‐
0.09 SDs, is very small. Thus, provided that
these studies are representative (of typical
and quality instruction methods), we can
say there is no evidence to support any
difference between CAI and TI methods.
However, we cannot say that there is
absolutely no possibility of a true difference,
since the confidence limits include estimates
(e.g. ‐0.47, 0.29) that Cohen (25) and other
researchers would consider important.
The meta‐graph shows that for this group of
studies, there is no difference between CAI
and TI, suggesting that CAI is just as
effective as traditional teaching methods.
Results of this research did not support the
hypothesis that, “Instruction that is taught
by a librarian face‐to‐face is more effective
than instruction that is computer‐based”.
Self‐Directed, Independent Learning vs. No
Instruction
There were 4 studies comparing SDIL to no
instruction and provided enough data to
calculate SMD. The meta‐graph in Figure 8
Figure 7: Standardized Mean Difference of CAI vs. Traditional studies
Evidence Based Library and Information Practice 2006, 1:3
18
Figure 8. Standardized Mean Difference of Self‐Directed Independent Learning vs. No Instruction.
shows the result of these studies. Note that
Broadway studied 1 SDIL group and 2 NI
groups. We divided Broadway’s SDIL group
(halving the sample size, so that each of the
no instruction groups could be included in
the meta‐analysis as a separate study.
Phipps and Dickstein studied 4 instruction
groups (2 were self‐directed, and 2 had no
instruction). The studies were included in
the meta‐analysis as 2 separate studies.
As the graph shows, the data in these
studies qualitatively favours SDIL over no
instruction, and all the studies individually
favour self‐directed teaching over no
instruction. However, since the size of effect
is heterogeneous (the confidence intervals
are non‐overlapping), we cannot conclude a
precise estimate of effect. Size of benefit will
vary based on the specific content of self‐
directed instruction. We can, however,
conclude that SDIL is more effective than no
instruction, since all studies are statistically
significant in favour of SDIL. Additionally,
since the lower overall confidence limit is
greater than 0.8 (as are most of the lower
confidence limits for the individual studies),
we can expect that SDIL will confer a larger
benefit compared to no instruction.
Discussion
The 122 studies included in this review
provide a broad picture of the state of the
research pertaining to effective library
instruction methods for improving the
information skills of students at an
introductory, undergraduate level, using
cognitive outcomes.
Most studies have been conducted on
traditional modes of instruction, however
these varied in terms of quality, and the
studies often lacked a comparison group.
When there was a comparison, it was
usually to ‘No Instruction’. The next most
frequent teaching area was ‘Computer
Assisted Instruction’. This group of studies
largely focused on a comparison with
‘Traditional Instruction’, perhaps an
indication of the way in which that type of
instruction has sought to prove its worth
against face‐to‐face teaching. ‘Self‐Directed,
Independent Learning’ had a number of
positive outcomes as a whole when
compared to either ‘No Instruction’ or
‘Traditional Instruction’ methods. Neither
‘Active Learning’ nor ’Learner‐Centred’
instruction yielded enough studies to make
any meaningful comparisons.
Looking at the studies as a whole, there
were a variety of research methods used,
some more rigorous than others. Of the 122,
79 (65%) were experimental or quasi‐
experimental studies. In trying to measure
effectiveness, such studies indicate more
reliable methods, if those studies are
conducted properly.
Evidence Based Library and Information Practice 2006, 1:3
19
There were several issues with methodology
and gaps in the information reported. One
apparent area was the lack of validated
research instruments. Of the 122, Only 9 (7%)
studies gave a detailed description of how
the research instrument was validated. A
validated research tool increases the
strength of the study, allowing us to trust
that the questions actually measure what
they intended to measure. A further 22 (18%)
studies indicated some attempt to pilot test
the instrument. The remaining studies did
not touch upon the importance of the
research instrument at all.
Sample size ranged in size from those with
less than 100 participants to very large with
more than 600 participants throughout the
122 studies. In 8 (6.5%) studies the sample
size was not mentioned, and in another
group of 5 (4%) studies the number of
participants was stated in only approximate
numbers. There were 37 (30%) studies with
100 or fewer participants, 35 (29%) studies
had 101‐200 participants, 22 (18%) studies
had 201‐600 participants, and 9 (7%) studies
had more than 600 participants.
More than 25 (20%) of the 122 studies did
not perform any statistical analysis of the
data they collected. A further 12 (10%) noted
that statistical tests were performed, but did
not report their data. Other methodological
issues with the studies were the lack of
reporting of loss to follow‐up between pre‐
test and post‐test, and biases that were
introduced, most noticeably selection bias of
participants.
Gaps in reporting were often a problem. In
many cases, it was difficult to determine
what a study was trying to measure, since
there was no clear description of the
research instrument or the learning
outcomes being measured. Sometimes there
was no description of what was being
taught or the learning environment. Such
elements are important to give the reader a
complete picture, in order to determine
whether the study may be applicable to the
reader’s own situation. With regard to
Bloom’s levels of learning, many studies did
not report sufficient information to
determine the outcomes being measured.
For the studies that could be measured,
focus rested on levels of learning that were
lower and easier to measure.
The 16 studies (13%) that met the criteria for
meta‐analysis were of higher quality, and
the meta‐analysis method accounted for
differences between studies. This analysis
suggests three key points, which could
affect practice:
• Computer assisted instruction is as
effective as traditional instruction.
• Traditional instruction is more
effective than no instruction.
• Self‐directed, independent learning
is more effective than no instruction.
The current research in our field does not
highlight any particular teaching method as
being more effective than any other teaching
method. While individual studies have
compared different teaching methods
directly, not enough of them have compared
those teaching methods in a meaningful
way. The only direct comparison between
two teaching methods that could be made
statistically using meta‐analysis was with
computer assisted instruction vs. traditional
instruction.
The neutral result showing that CAI is just
as effective as TI indicates that decisions to
implement CAI should not be disregarded
because of the belief that in‐person
instruction is more beneficial to the students.
As the research points to equal outcomes,
other factors such as time, costs, staffing,
and ability to reach greater numbers of
students need to be weighed in the decision
of using one teaching method over another.
Qualitative research designs may have more
to add to the discussion of which method
Evidence Based Library and Information Practice 2006, 1:3
20
may be preferable, based upon other factors
such as user preference.
The result showing that TI is more effective
than no instruction, provides a measure of
the worth of library instruction as it has
most commonly been performed in the past.
Teaching seems to provide some benefit,
over and above not teaching at all. While
this is positive, more research is needed to
determine whether or not TI is the best
method of instruction. Studies comparing
traditional teaching methods to AL and to
SDIL, for example, should be considered, so
that we can more fully understand the
effectiveness of the ways in which we teach
information literacy skills.
The positive results for SDIL versus no
instruction, should not be overlooked. As
with CAI, this method of instruction
provides a solution that does not require
instruction to be face‐to‐face. It is a possible
solution for those wanting to implement
information literacy initiatives but not
having the resources to teach in person.
Again, more research is required to test this
teaching method against other methods,
including traditional instruction and CAI.
Active learning and learner‐centred
instruction did not have enough studies to
reach any conclusions about their
effectiveness. These areas require further
research to create a body of studies
comparing these teaching methods to other
more established methods, such as
traditional and computer assisted
instruction.
Possible confounders of this systematic
review and meta‐analysis include the
dominance of U.S. studies in the evidence
base, which may impact transferability of
these results. Another factor is that only
studies reported in the English language
were included; results may not be able to be
generalized to practice in other countries.
The authors relied on study design as a
measure of quality, but study design in and
of itself does not constitute a good study.
Just because a study has a design that
should be more rigorous, does not mean
that it has been well conducted. However,
for the questions of effectiveness that the
authors were attempting to answer, the
comparative study design was an
appropriate filter. Finally, results were not
compared across subject areas, so results
may not necessarily be transferable from
one subject area to another. Readers should
look at the detail provided in Appendix D
for studies that apply specifically to their
subject areas.
Conclusion
The goal of conducting this systematic
review was to find evidence about
information literacy instructional methods
that may have a direct impact on the way
academic librarians approach teaching
information literacy to undergraduate
students. This review provides a general
picture of the research that has been done in
this area, highlights some of the better
research, draws together current evidence
on what teaching methods are most effective,
highlights problems with this body of
research, and outlines what we can learn
from the current evidence base.
To determine effectiveness of teaching
methods, more comparative research needs
to be done across different teaching methods
using sound research methodologies and
validated research tools. Careful
consideration should be given to the most
appropriate research method, and
researchers should determine possible areas
of comparison before beginning the study.
The norm is to compare with no instruction,
or to the standard traditional methods.
Studies comparing to active learning,
computer assisted instruction, and self‐
directed independent learning would
Evidence Based Library and Information Practice 2006, 1:3
21
greatly enrich the research literature.
Building upon existing studies with good
methodologies and validated research tools
would enrich our evidence base and
contribute to the growth of knowledge
about effectiveness of particular teaching
methods.
This systematic review of the literature
provides librarians who teach information
literacy skills with an overview of the
research literature in this area. Despite
certain flaws in the studies which have been
outlined, this review provides pragmatic
evidence of the effectiveness of several
methods of teaching information literacy.
The lack of comparative studies, however,
does not allow us to conclude anything
about the effectiveness of teaching methods
in areas such as active learning and learner‐
centred instruction. In the areas of
traditional instruction, CAI, and self‐
directed independent learning, only three
conclusions could be made. The systematic
method of arriving at those conclusions
provides us with clear evidence in those
specific areas. However, there is not enough
evidence to determine which teaching
method is best. We should approach
information literacy grounded in this
knowledge, and move forward to build
upon it, contributing to the evidence base in
the future.
Nevertheless, this systematic review will aid
librarians teaching information literacy
skills to undergraduate students to be more
informed about the teaching methods they
use. While there is not enough current
evidence to persuade instructional librarians
to change their teaching practices from one
method to another, librarians should know
that the evidence base for effectiveness
based on cognitive outcomes is weak, and
make their decisions with that in mind. This
systematic review should be a call for those
involved with information literacy
instruction to contribute to the research
knowledge base in their field of interest, so
that better decisions can be made in the
future. Researchers, particularly those in
countries other than the U.S., should
continue to build upon the research results
summarised here to provide more research
evidence of effective library instruction
methods.
Works Cited
Anderson, Lorin W., and David R.
Krathwohl, eds. A Taxonomy for
Learning, Teaching, and Assessing: A
Revision of Bloomʹs Taxonomy of
Educational Objectives. New York:
Longman, 2001.
Baruchson‐Arbib, Shifra, and Jenny
Bronstein. “A View to the Future of the
Library and Information Science
Profession: A Delphi Study.” Journal of
the American Society for Information
Science and Technology 53.5 (Mar.
2002): 397‐408.
Brettle, Alison. “Information Skills Training:
A Systematic Review of the
Literature.” Health Information and
Libraries Journal 20.Supp.1 (2003): 3‐9.
Chalmers I., L.V. Hedges, and H. Cooper.
“A Brief History of Research
Synthesis.” Evaluation & The Health
Professions 25.1 (2002):12‐37.
Cohen, J. Statistical Power Analysis for the
Behavioral Sciences, Rev. ed. Hillsdale,
NJ: Lawrence Erlbaum Associates, 1988.
Cooper, H., and L.V. Hedges, eds. The
Handbook of Research Synthesis. Sage
Publications, 1994.
Egger, M., G. Davey Smith, and D.G.
Altman. Systematic Reviews in Health
Care. London: BMJ Books, 2001.
Evidence Based Library and Information Practice 2006, 1:3
22
Garg, Anupama, and Kathleen M. Turtle.
“Effectiveness of Training Health
Professionals in Literature Search Skills
using Electronic Health Databases – A
Critical Appraisal.” Health Information
and Libraries Journal 20.1 (2003): 33‐41.
Godwin, Peter. “Learning and Teaching –
The Librarian’s Contribution: An
Introduction.” Vine 31.1 (2001): 3‐4.
Morrison, Jillian M., Frank Sullivan,
Elizabeth Murray, and Brian Jolly.
“Evidence‐Based Education:
Development of an Instrument to
Critically Appraise Reports of
Educational Interventions.” Medical
Education 33 (1999): 890‐3.
National Forum on Information Literacy.
“What is Information Literacy? “ 20
July 2006 .
Peacock, Judith. “Teaching Skills for
Teaching Librarians: Postcards from
the Edge of the Educational
Paradigm.” Australian Academic &
Research Libraries 32.1 (2001): 26‐42.
26 April 2006
.
Acknowledgements
The authors would like to thank the
anonymous peer reviewers whose
comments improved the quality of this
paper.
Denise Koufogiannakis would like to thank
the University of Alberta Libraries for
granting her a one year professional leave,
during which time she conducted this
research. She would also like to thank
Andrew Booth, who generously provided
invaluable advice and support during the
research process.
A preliminary version of this paper was
presented at the 3rd International Evidence
Based Librarianship conference in Brisbane,
Australia, October 2005. See:
.
http://www.infolit.org/
http://www.alia.org.au/publishing/aar
http://conferences.alia.org.au/ebl2005/Kouf
Evidence Based Library and Information Practice 2006, 1:3
23
Appendix A: Literature Search Process
Searches were conducted between September and November, 2004 and updated in May, 2005.
1. LISA (CSA) 1969 ‐ present
Search strategy:
((de=(user training) or (information literacy) or (computer assisted instruction) or (information
literacy tutorial) or (education activities)) or ʺlibrary instruct*ʺ or ʺlibrary educat*ʺ or ʺlibrar* teach*ʺ
or ʺuser train*ʺ or ʺuser educat*ʺ or ʺonline tutor*ʺ or ʺlibrary skill*ʺ or ʺinformation skill*ʺ) and
(de=students or undergrad* or ʺfirst yearʺ or bachelor* or ʺpost secondaryʺ)
Number of documents retrieved: 1,443.
2. Library Literature (Silverplatter) 1984 ‐ present
Search strategy:
#1 bibliographic‐instruction‐college‐and‐university‐students in DE (1,285 records)
#2 bibliographic‐instruction‐junior‐and‐community‐college‐students in DE (47 records)
#3 #1 or #2 (1,332 records)
#4 computer‐assisted‐instruction in DE (909 records)
#5 end‐user‐searching‐teaching in DE (750 records)
#6 bibliographic‐instruction in DE (207 records)
#7 internet‐teaching in DE (398 records)
#8 information literacy (593 records)
#9 library instruct* (535 records)
#10 librar* teach* (2,351 records)
#11 user train* (41 records)
#12 user educat* (213 records)
#13 online tutor* (16 records)
#14 library skill* (159 records)
#15 information skill* (206 records)
#16 #4 or #5 or #6 or #7 or #8 or #9 or #10 or #11 or #12 or #13 or #14 or #15 (5,708 records)
#17 undergrad* (508 records)
#18 first year (77 records)
#19 bachelor* (9 records)
#20 post secondary (8 records)
#21 postsecondary (12 records)
#22 #17 or #18 or #19 or #20 or #21 (605 records)
#23 #16 and #22 (108 records)
#24 #3 or #23 (1,383 records)
Number of documents retrieved: 1,383.
3. ERIC (OVID) 1966 ‐ July 2004
Search Strategy:
1 exp Information Literacy/ (984)
2 exp Course Integrated Library Instruction/ (409)
3 exp Library Instruction/ (3,082)
4 exp Information Skills/ (1,950)
5 exp Library Skills/ (1,542)
6 library instruct$.mp. [mp=abstract, title, headings word, identifiers, eric digests full text] (3,231)
7 library educat$.mp. [mp=abstract, title, headings word, identifiers, eric digests full text] (3,133)
8 librar$ teach$.mp. [mp=abstract, title, headings word, identifiers, eric digests full text] (1,068)
Evidence Based Library and Information Practice 2006, 1:3
24
9 user train$.mp. [mp=abstract, title, headings word, identifiers, eric digests full text] (255)
10 user educat$.mp. [mp=abstract, title, headings word, identifiers, eric digests full text] (224)
11 information literacy.mp. [mp=abstract, title, headings word, identifiers, eric digests full text]
(1,274)
12 online tutor$.mp. [mp=abstract, title, headings word, identifiers, eric digests full text] (55)
13 library skill$.mp. [mp=abstract, title, headings word, identifiers, eric digests full text] (1,749)
14 information skill$.mp. [mp=abstract, title, headings word, identifiers, eric digests full text]
(1,114)
15 or/1‐14 (9,234)
16 exp Undergraduate Students/ (5,855)
17 exp College Freshmen/ (5,672)
18 exp college freshmen/ or exp college juniors/ or exp college seniors/ or exp college
sophomores/ (6,057)
19 undergrad$.mp. [mp=abstract, title, headings word, identifiers, eric digests full text] (27,073)
20 (postsecondary or post‐secondary or post secondary).mp. [mp=abstract, title, headings word,
identifiers, eric digests full text] (38,914)
21 bibliographic instruct$.mp. (632)
22 15 or 21 (9,278)
23 bachelor$.mp. [mp=abstract, title, headings word, identifiers, eric digests full text] (3,905)
24 (first year or first‐year).mp. [mp=abstract, title, headings word, identifiers, eric digests full text]
(8,997)
25 or/16‐20 (70,037)
26 or/23‐25 (79,802)
27 26 and 22 (885)
28 exp Undergraduate Study/ (7,066)
29 26 or 28 (79,802)
30 29 and 22 (885)
31 from 30 keep 1‐885 (885)
Number of documents retrieved: 885
4. Inspec (Axiom/IOP) 1968 – present
Search Strategy:
((information literacy or information instruction or information searching skills or library instruct*
or library educat* or librar* teach* or user train* or user educat* or online tutor* or library skill* or
information skill* or bibliographic instruct*) and (undergrad* or first year or bachelor* or post
secondary or freshman or academic librar*)) (TI,AB,CI,UI)
Number of documents retrieved: 232
5. Academic Search Premier (EBSCO) 1975 ‐present
Search Strategy:
S1
SU (information literacy or library education or
information services ‐ user education or computer‐
assisted instruction or user education)
5,366
S2
(bibliographic instruct* or library instruct* or
library educat* or librar* teach* or user train* or
user educat* or online tutor* or library skill* or
information skill*)
2,868
S3
(undergrad* or first year or bachelor* or
postsecondary or academic librar*)
35,765
Evidence Based Library and Information Practice 2006, 1:3
25
S4 (S2 Or S1) 8,071
S5 (S4 And S3) 767
Number of documents retrieved: 767
6. Educational Research Abstracts 1995‐present
Search Strategy:
((keywords/ʺlibrary instruct*ʺ or keywords/ʺbibliographic instruct*ʺ or keywords/ʺinformation
literacyʺ or keywords/ʺuser train*ʺ or keywords/ʺlibrary educat*ʺ or keywords/ʺlibrary teach*ʺ or
keywords/ʺonline tutor*ʺ or keywords/ʺlibrary skill*ʺ or keywords/ʺinformation skill*ʺ) and
(keywords/undergrad* or keywords/ʺfirst yearʺ or keywords/bachelor* or keywords/postsecondary
or keywords/ʺpost secondaryʺ or keywords/ʺacademic librar*ʺ))
Number of documents retrieved: 6
7. CINAHL (OVID) 1982 – present
Search Strategy:
1 exp Library User Education/ (357)
2 (library instruct$ or library educat$ or librar$ teach$ or information literacy or library skill$ or
information skill$).mp. [mp=title, cinahl subject headings, abstract, instrumentation] (205)
3 1 or 2 (473)
4 exp Students, College/ (2,911)
5 (undergrad$ or first year or bachelor$ or postsecondary or post secondary).mp.
[mp=title, cinahl subject headings, abstract, instrumentation] (4,314)
6 exp Libraries, Academic/ (928)
7 or/4‐6 (7,719)
8 3 and 7 (124)
9 from 8 keep 1‐124 (124)
Number of documents retrieved: 124
8. Web of Science (all sections) 1945‐present
Search Strategy:
#1 1,665 TS=(librar* instruct* OR bibliographic instruct* OR library educat* OR librar*
teach* OR user train* OR user educat* OR information literacy OR online
tutor* OR library skill* OR information skill*)
DocType=All document types; Language=All languages; Databases=SCI‐
EXPANDED, SSCI, A&HCI; Timespan=1945‐2004
#2 56,264 TS=(undergrad* OR first year OR first‐year OR bachelor* OR postsecondary
OR post‐secondary OR academic librar* OR university student* OR college
student*)
DocType=All document types; Language=All languages; Databases=SCI‐
EXPANDED, SSCI, A&HCI; Timespan=1945‐2004
#3 185 #1 AND #2
DocType=All document types; Language=All languages; Databases=SCI‐
EXPANDED, SSCI, A&HCI; Timespan=1945‐2004
Number of documents retrieved: 185
9. Dissertation Abstracts / Digital Dissertations (ProQuest)
Search Strategy:
Evidence Based Library and Information Practice 2006, 1:3
26
1 KEY(library instruction) or KEY(bibliographic
instruction) or KEY(information literacy)
138
2 KEY(library education) OR KEY(user education) OR
KEY(library skill?) OR KEY(information skill?)
171
3 #2 or #1 296
4 KEY(undergrad?) OR KEY(first year) OR KEY(first‐
year) OR KEY(bachelor?) OR KEY(postsecondary)
OR KEY(post‐secondary) OR KEY(academic librar?)
OR KEY(university student?) OR KEY(college
student?)
40,957
5 #3 and #4 70
Number of documents retrieved: 70
10. Conference Papers Index (CSA)
Search Strategy:
(ʺinformation literacyʺ or ʺlibrary instruct*ʺ or ʺlibrary educat*ʺ or ʺlibrar* teach*ʺ or ʺuser train*ʺ or
ʺuser educat*ʺ or ʺonline tutor*ʺ or ʺlibrary skill*ʺ or ʺinformation skill*ʺ) and (undergrad* or ʺfirst
yearʺ or bachelor* or ʺpost secondaryʺ or academic librar*)
Number of documents retrieved: 2
11. SIGLE (System for Information on Grey Literature in Europe)) 1980‐present
Search Strategy:
(LIBRAR? INSTRUCT? OR BIBLIOGRAPHIC INSTRUCT? OR LIBRARY EDUCAT? OR LIBRAR?
(3A)TEACH? OR USER TRAIN? OR USER EDUCAT? OR INFORMATION LITERACY OR
ONLINE TUTOR? OR LIBRARY SKILL? OR INFORMATION SKILL?) AND (UNDERGRAD? OR
FIRST YEAR OR FIRST‐YEAR OR BACHELOR? OR POSTSECONDARY OR POST‐SECONDARY
OR ACADEMIC LIBRAR? OR UNIVERSITY STUDENT? OR COLLEGE STUDENT?)
Number of documents retrieved: 0
12. CERUK: Current Education Research in the UK
Browsed keyword terms: Librarians; Library and information services; Information skills
Number of documents retrieved: 15
13. Education‐Line
Query: (ʺLIBRARY INSTRUCT$ʺ OR ʺBIBLIOGRAPHIC INSTRUCT$ʺ OR ʺLIBRARY EDUCAT$ʺ
OR ʺLIBRAR$ TEACH$ʺ OR ʺUSER TRAIN$ʺ OR ʺUSER EDUCAT$ʺ OR ʺINFORMATION
LITERACYʺ OR ʺONLINE TUTOR$ʺ OR ʺLIBRARY SKILL$ʺ OR ʺINFORMATION SKILL$ʺ) AND
(UNDERGRAD$ OR ʺFIRST YEARʺ OR ʺFIRST‐YEARʺ OR BACHELOR$ OR POSTSECONDARY
OR ʺPOST‐SECONDARYʺ OR ʺUNIVERSITY STUDENT$ʺ OR ʺCOLLEGE STUDENT$ʺ OR
ʺACADEMIC LIBRAR$ʺ)
Number of documents retrieved: 11
14. British Education Index (Dialog)
Number of documents retrieved: 16
15. Australian Education Index (Dialog)
Number of documents retrieved: 30
http://www.leeds.ac.uk/educol
Evidence Based Library and Information Practice 2006, 1:3
27
Appendix B: Studies Included
1. Ackerson, Linda G., Jeanne G. Howard, and Virginia E. Young, ʺAssessing the Relationship
Between Library Instruction Methods and the Quality of Undergraduate Research.ʺ Research
Strategies 9.3 (1991): 139‐41. (Results also published in Ackerson et al., 1994.)
2. Ackerson, Linda G., and Virginia E. Young. “Evaluating the Impact of Library Instruction
Methods on the Quality of Student Research (Three‐year Study at the University of
Alabama).” Research Strategies 12 (1994): 132‐44. (Results also published in Ackerson et al.,
1991.)
3. Alexander, Linda B. LIBS 1000: A Credit Course in Library Skills at East Carolina University.
1994. ERIC Document Reproduction Service ED376818.
4. Alexander, Linda B. “Library Skills Instruction: A Comparison of Students in a Web‐Based
Course versus a Traditional Instruction Course.” Diss. Univ. of Louisville, May 2000.
(Results also published in Alexander and Smith, 2001.)
5. Alexander, Linda B., and Robert C. Smith. ʺResearch Findings of a Library Skills Instruction
Web Course.ʺ portal 1.3 (2001): 309‐28. (Results also published in Alexander, 2000.)
6. Andretta, Susie. “Legal Information Literacy: A Pilot Study.” New Library World
102.1166/1167 (2001): 255‐64.
7. Arnold, Julie, Robert Kackley, and Stephen Fortune. “Hands‐on Learning for Freshman
Engineering Students.” Issues in Science and Technology Librarianship 37 (Spring 2003). 19
May 2006 .
8. Axeen, Marina Esther. Teaching Library Use to Undergraduates‐‐Comparison of Computer‐
based Instruction and the Conventional Lecture. Final report. Urbana: Univ. of Illinois, 1967.
ERIC Document Reproduction Service ED014316.
9. Benefiel, Candace R., and Joe Jaros. ʺPlanning and Testing a Self‐Guided Taped Tour in an
Academic Library.ʺ RQ 29.2 (Winter 1989): 199‐208.
10. Benham, Frances. College Library Technology and Cooperation Grants Program Interim
Performance Report Higher Education Act, Title II‐D. Tuscaloosa: Alabama Univ.,1988. ERIC
Document Reproduction Service ED346853.
11. Bolt, Janice Ann Havlicek. “A Study of the Effects of a Bibliographic Instruction Course on
Achievement and Retention of College Students.” Diss. Florida State University, 1986.
12. Borgman, Christine L. “The User’s Mental Model of an Information Retrieval System: An
Experiment on a Prototype Online Catalog.” International Journal of Man‐Machine Studies
24.1 (1986): 47‐64. (Accessed via reprint in International Journal of Human‐Computer Studies
51.2 (1999): 435‐52).
13. Bostian, Rebecca, and Anne Robbins. ʺEffective Instruction for Searching CD‐Rom Indexes.ʺ
Laserdisk Professional 3.1 (1990): 14‐7.
http://www.istl.org/03%E2%80%90spring/article3.html%00%00
Evidence Based Library and Information Practice 2006, 1:3
28
14. Bradigan, Pamela S., and Carol A. Mularski. “End‐User Searching in a Medical School
Curriculum: An Evaluated Modular Approach.” Bulletin of the Medical Library Association
77.4 (1989 Oct.): 348‐56.
15. Breivik, Patricia Senn. Open Admissions and the Academic Library. Chicago: ALA, 1977.
“Brooklyn College: A Test Case” p.49‐66.
16. Bren, Barbara, Beth Hillemann, and Victoria Topp. ʺEffectiveness of Hands‐on Instruction of
Electronic Resources.ʺ Research Strategies 16.1 (1998): 41‐51.
17. Broadway, Marsha Denise. ʺSelf‐directed Instruction in Query Formation and Presentation
for College Students (Bibliographic Instruction, Reference Interview, Communication).ʺ Diss.
Florida State University, 1985.
18. Brown, Cecelia, and Lee R. Krumholz. “Integrating Information Literacy into the Science
Curriculum.” College and Research Libraries 63.2 (Mar. 2002): 111‐23.
19. Brown, Lyn S., and G. Jeremiah Ryan. The Relationship of Time to Effectiveness in Research
Skills Instruction for Students at Philadelphia College of Bible. Pennsylvania: Philadelphia
College of Bible, 1995. ERIC Document Reproduction Service ED402931.
20. Buchanan, Nancy L., Karen Rupp‐Serrano, and Johanne LaGrange. ʺThe Effectiveness of a
Projected Computerized Presentation in Teaching Online Library Catalog Searching.ʺ College
and Research Libraries 53.4 (July 1992): 307‐18.
21. Bushong, Sara. Utilization of Powerpoint Presentation Software in Library Instruction of
Subject Specific Reference Sources. Kent State University, 1998. ERIC Document
Reproduction Service ED423914.
22. Cameron, Lynn, and James Hart. “Assessment of PsycLit Competence, Attitudes and
Instructional Methods.” Teaching of Psychology 19.4 (1992): 239‐42.
23. Champion, Brian. Computer Assisted Instruction and Bibliographic Instruction: Preliminary
Data on the Use of Plato in the BI Program of the Humanities and Social Sciences Library,
University of Alberta. University of Alberta, 1986. ERIC Document Reproduction Service
ED284567.
24. Cherry, Joan M., Weijing Yuan, and Marshall Clinton. “Evaluating the Effectiveness of a
Concept‐based Computer Tutorial for OPAC Users.” College and Research Libraries 55.4
(July 1994): 355‐64.
25. Cherry, Joan M., and Marshall Clinton. “An Experimental Investigation of Two Types of
Instruction for OPAC Users.” Canadian Journal of Information Science 16 (Dec. 1991): 2‐22.
26. Churkovich, Marion, and Christine Oughtred. ʺCan an Online Tutorial Pass the Test for
Library Instruction? An Evaluation and Comparison of Library Skills Instruction Methods for
First Year Students at Deakin University.ʺ Australian Academic and Research Libraries 33.1
(2002): 25‐38.
Evidence Based Library and Information Practice 2006, 1:3
29
27. Colaric, Susan M. ʺInstruction for Web Searching: An Empirical Study.ʺ College & Research
Libraries 64.2 (Mar. 2003): 111‐22.
28. Cooper Moore, Anne. ʺThe Impact of Hands‐on Information Literacy Instruction on
Learning/Knowledge of Information Literacy Concepts and Mastery of the Research Process
in College Courses: A Quasi‐Experimental Study.ʺ Diss. New Mexico State University, 2001.
29. Cudiner, Shelley, and Oskar Harmon. ʺComparing the Effectiveness of Different Presentation
Formats for Workshops on Introductory Library Skills.ʺ Research Strategies 18.1 (2001): 49‐61.
30. Currie, Margaret, Elaine Goettler, and Sandra McCaskill. “Evaluating the Relationship
Between Library Skills and Library Instruction.” Canadian Library Journal 39.1 (Feb. 1982):
35‐7.
31. Damron, James A. ʺIn‐Coming Provisional Students: Comparing Pre‐ and Post‐Test Scores
for a Summer Library Program.ʺ Virginia Libraries 49.2 (2003): 21‐2.
32. Daugherty, Timothy K., and Elizabeth W. Carter. “Assessment of Outcome‐Focused Library
Instruction in Psychology.” Journal of Instructional Psychology 24.1 (1997): 29‐33.
33. Davis, Dorothy F. “A Comparison of Bibliographic Instruction Methods on CD‐ROM
Databases.” Research Strategies 11.3 (1993): 156‐63.
34. Davis, H. Scott. A Comparison of Three Instructional Approaches to Online Catalog
Instruction: What Students Prefer May Work Best. 1989. ERIC Document Reproduction
Service ED314083.
35. DeLong, Edward James. An Evaluative Report of the Richmond College Freshman Library
Instruction Program. Virginia: Richmond Univ., 1978. ERIC Document Reproduction Service
ED157547.
36. Dodgen, Lynda, Sarah Naper, Olia Palmer, and Adrian Rapp. “Not so SILI: Sociology
Information Literacy Infusion as the Focus of Faculty and Librarian Collaboration.”
Community & Junior College Libraries 11.4 (2003): 27‐33.
37. Donegan, Patricia Morris, Ralph E. Domas, and John R. Deosdade. ʺThe Comparable Effects
of Term Paper Counseling and Group Instruction Sessions.ʺ College and Research Libraries
50 (1989): 195‐205.
38. Dorsch, Josephine L., Meenakshy K. Aiyer, and Lynne E. Meyer. “Impact of an Evidence‐
Based Medicine Curriculum on Medical Students’ Attitudes and Skills.” Journal of the
Medical Library Association 92.4 (2004): 397‐406.
39. Edwards, Sherri. “Effects of a Self‐Paced Workbook on Students’ Skills and Attitudes.”
Research Strategies 9.4 (1991): 180‐8.
40. Ellsbury, Susan H. Feasibility Study: Library Instruction in Specific Science Disciplines Using
the Self‐paced Workbook Adapted to Departmental Needs, Mitchell Memorial Library, Fall
1981. Mississippi: Mississippi State Univ., 1981. ERIC Document Reproduction Service
Evidence Based Library and Information Practice 2006, 1:3
30
ED225589.
41. Emmons, Mark, and Wanda Martin. ʺEngaging Conversation: Evaluating the Contribution of
Library Instruction to the Quality of Student Research.ʺ College and Research Libraries 63.6
(2002): 545‐60.
42. Eyman, David H., and Alven C. Nunley, Jr. The Effectiveness of Library Science 1011 in
Teaching Bibliographical Skills. Tahlequah: Northeastern Oklahoma State Univ., 1977. ERIC
Document Reproduction Service ED150962.
43. Facinelli, Jaclyn. Music Students and Bibliography Instruction: A Study. Ohio: University of
Akron, 1982. ERIC Document Reproduction Service ED242322.
44. Fox, Lynne M., Judith M. Richter, and Nancy E. White. “A Multidimensional Evaluation of a
Nursing Information‐Literacy Program.” Bulletin of the Medical Library Association 84.2
(Apr. 1996): 182‐90.
45. Franklin, Godfrey, and Ronald C. Toifel. ʺThe Effects of BI on Library Knowledge and Skills
Among Education Students.ʺ Research Strategies 12.4 (Fall 1994): 224‐37.
46. Frasca, Michael A., Josephine L. Dorsch, Jean C. Aldag, and Richard G. Christiansen. “A
Multidisciplinary Approach to Information Management and Critical Appraisal Instruction:
A Controlled Study.” Bulletin of the Medical Library Association 80.1 (Jan.1992): 23‐8.
47. Fry, Thomas K., and Joan R. Kaplowitz. ʺThe English 3 Library Instruction Program at UCLA:
A Follow‐up Study.ʺ Research Strategies 6.3 (1988): 100‐8.
48. Germain, Carol Anne, Trudi E. Jacobson, and Sue A. Kaczor. ʺA Comparison of the
Effectiveness of Presentation Formats for Instruction: Teaching First‐year Students.ʺ College
and Research Libraries 61.1 (Jan. 2000): 65‐72.
49. Gutierrez, Carolyn, and Jianrong Wang. ʺA Comparison of an Electronic vs. Print Workbook
for Information Literacy Instruction.ʺ The Journal of Academic Librarianship 27.3 (2001): 208‐
12.
50. Hales, Celia. ʺBasic BI at the University of North Carolina at Charlotte: Results of an
Experiment.ʺ Southeastern Librarian (Fall 1985): 76‐7.
51. Hall, Virginia B., Daniel Krautheim, and Baljit S. Hansra. “A Slide‐Tape Program for
Beginning Pharmacy Students: Effect on Learning.” Bulletin of the Medical Library
Association 65.4 (Oct. 1977): 443‐5.
52. Hardesty, Larry, Nicholas P. Lovrich, Jr., and James Mannon. ʺEvaluating Library‐Use
Instruction.ʺ College and Research Libraries 40.4 (July 1979): 309‐17.
53. Hardesty, Larry, Nicholas P. Lovrich, Jr., and James Mannon. “Library Use Instruction:
Assessment of the Long‐term Effects.” College & Research Libraries 43.1 (1982): 38‐46.
54. Hausrath, Don, Shuk‐Chun Auyeung, Jo Anne Howell, and Kaye Bedell. “Integrating
Evidence Based Library and Information Practice 2006, 1:3
31
Information Competencies into the Allied Health Curriculum at Gavilan College.”
Community & Junior College Libraries 11.2 (2003): 13‐49.
55. Holman, Lucy. ʺA Comparison of Computer‐Assisted Instruction and Classroom
Bibliographic Instruction.ʺ Reference and User Services Quarterly 40.1 (2000): 53‐60.
56. Holt, Joan S., and Steven Falk. “Evaluation of Library Workbooks in a Community College
Setting.” Reference Librarian 11 (Fall‐Winter 1984): 321‐34.
57. Hooks, James D. Teaching Library Skills to Academically Unprepared College Freshmen.
1986. ERIC Document Reproduction Service ED296740.
58. Howze, Philip C., and Dana E. Smith. “Library Instruction as Independent Study: The
Summer Enrichment Program Experiment at Iowa State University.” Reference Services
Review; 23.4 (1995): 75‐82.
59. Jacobsen, Gertrude N., and Michael J. Albright. “Motivation via Videotape: Key to
Undergraduate Library Instruction in the Research Library.” Journal of Academic
Librarianship 9.5 (Nov. 1983): 270‐5.
60. Jennerich, Elaine Zaremba, and Bessie Hess Smith. ʺA Bibliographic Instruction Program in
Music.ʺ College and Research Libraries 40.3 (May 1979): 226‐33.
61. Johnson, Kathleen A., and Barbara S. Plake. “Evaluation of PLATO Library Instructional
Lessons: Another View.” Journal of Acadmic Librarianship 6.3 (1980): 154‐8.
62. Julien, Heidi, and Stuart Boon. ʺAssessing Instructional Outcomes in Canadian Academic
Libraries.ʺ Library & Information Science Research 26.2 (2004): 121‐39.
63. Kaplowitz, Joan, and Janice Contini. “Computer‐Assisted Instruction: Is it an Option for
Bibliographic Instruction in Large Undergraduate Survey Classes?” College and Research
Libraries 59.1 (Jan. 1998): 19‐27.
64. Kaplowitz, Joan R. ʺA Pre‐ and Post‐Test Evaluation of the English 3‐Library Instruction
Program at UCLA.ʺ Research Strategies 4 (1986): 11‐7.
65. Keever, Ellen H., and James C. Raymond. “Integrated Library Instruction on the University
Campus: Experiment at the University of Alabama.” Journal of Academic Librarianship 2.4
(Sept. 1976): 185‐7.
66. Kimsey, Mary B., and S. Lyn Cameron. “Teaching and Assessing Information Literacy in a
Geography Program.” Journal of Geography 104.1 (Jan.‐Feb. 2005): 17‐23.
67. Kirk, Thomas. “A Comparison of Two Methods of Library Instruction for Students in
Introductory Biology.” College and Research Libraries 32 (1971): 465‐74.
68. Kohl, David, and Lizabeth Wilson. “Effectiveness of Course‐Integrated Bibliographic
Instruction in Improving Coursework.” RQ 27 (1986): 206‐11.
Evidence Based Library and Information Practice 2006, 1:3
32
69. Kochtanek, Thomas R. Measuring the Effectiveness of a Library Skills Program. Arlington
Educational Resources Information Center. 1980. ERIC Document Reproduction Service
ED190157.
70. Koufogiannakis, Denise, Jeanette Buckingham, Arif Alibhai, and David Rayner. “Impact of
Librarians in First Year Medical and Dental Student Problem‐Based Learning (PBL) Groups:
A Controlled Study.” Health Information and Libraries Journal 22.3 (Sept. 2005): 189‐95.
71. Langley, Linda. “The Effects of a Credit Course in Bibliographic Instruction.” Technicalities 2
(1987): 3‐7.
72. Lawson, Mollie D. ʺAssessment of a College Freshman Course in Information Resources.ʺ
Library Review 48.2 (1999): 73‐8.
73. Lawson, V. Lonnie. ʺUsing a Computer‐Assisted Instruction Program to Replace the
Traditional Library Tour: An Experimental Study.ʺ RQ 29 (Fall 1989): 71‐9.
74. Lechner, Judith V. ʺBibliographic Instruction Evaluation: A Study Testing the Correlations
Among Five Measures of the Impact of a Bibliographic Instruction Program on
Undergraduatesʹ Information Searching Behavior in Libraries.ʺ Diss. University of California,
1989.
75. Linton, Anne M., Patricia H. Wilson, Alexandra Gomes, Laura Abate, and Matthew
Mintz. ”Evaluation of Evidence‐Based Medicine Search Skills in the Clinical Years.” Medical
Reference Services Quarterly 23.2 (Summer 2004): 21‐31.
76. Mackey, Neosha, Charlotte Dugan, Willa Garrett, and C. Lynne Freeman. “Teaching with
Hypercard in Place of a Textbook.” Computers in Libraries 12 (Oct. 1992): 22‐6.
77. MacPherson, Karen. “An Information Processing Model of Undergraduate Electronic
Database Information Retrieval.” Journal of the American Society for Information Science
and Technology 55.4 (2004): 333‐47. (Results also published in MacPherson, 2004, Australian
Academic and Research Libraries.)
78. MacPherson, Karen. “Undergraduate Information Literacy: A Teaching Framework.”
Australian Academic and Research Libraries 35.3 (2004): 226‐42. (Results also published in
MacPherson, 2004, Journal of the American Society for Information Science and Technology.)
79. Madland, Denise, and Marian A. Smith. “Computer Assisted Instruction for Teaching
Conceptual Library Skills to Remedial Students.” Research Strategies 6 (1988): 52‐64.
80. Magi, Trina J. “Whatʹs Best for Students? Comparing the Effectiveness of a Traditional Print
Pathfinder and a Web‐based Research Tool.” portal: Libraries and the Academy 3.4 (Oct.
2003): 671‐86.
81. Marcus, Sandra, and Sheila Beck. ʺA Library Adventure: Comparing a Treasure Hunt with a
Traditional Freshman Orientation Tour.ʺ College and Research Libraries 64.1 (2003): 23‐44.
82. Marfleet, B. Gregory, and Brian J. Dille. Information Literacy and the Undergraduate
Evidence Based Library and Information Practice 2006, 1:3
33
Methods Curriculum. 2003. ERIC Document Reproduction Service ED479704.
83. Meehan‐Black, Elizabeth C. The Effects of Two Methods of Course‐Related Library
Instruction on Undergraduatesʹ Library Skills and Attitudes. 1981. ERIC Document
Reproduction Service ED246918.
84. Nichols, James, Barbara Shaffer, and Karen Shockey. ʺChanging the Face of Instruction: Is
Online or In‐Class More Effective?ʺ College and Research Libraries 64.5 (2003): 378‐88.
85. Nielsen, Brian, and Betsy K. Baker. ʺEducating the Online Catalog User: A Model Evaluation
Study.ʺ Library Trends 35.4 (1987): 571‐85.
86. Orme, William A. “A Study of the Residual Impact of the Texas Information Literacy Tutorial
on the Information‐Seeking Ability of First Year College Students.” College and Research
Libraries 65.3 (May 2004): 205‐15.
87. Paglia, Alison, and Annie Donahue. “Collaboration Works: Integrating Information
Competencies into the Psychology Curricula.” Reference Services Review 31.4 (2003): 320‐8.
88. Palmer, Stuart, and Barry Tucker. “Planning, Delivery and Evaluation of Information
Literacy Training for Engineering and Technology Students.” Australian Academic and
Research Libraries 35.1 (Mar. 2004): 16‐34.
89. Parang, Elizabeth, Melinda Raine, and Trisha Stevenson. “Redesigning Freshman Seminar
Library Instruction Based on Information Competencies.” Research Strategies 17.4 (2000):
269‐80.
90. Pearson, Penelope, and Virginia Tiefel. “Evaluating Undergraduate Library Instruction at the
Ohio State University.” Journal of Academic Librarianship 7.6 (Jan. 1982): 351‐7.
91. Phipps, Shelley, and Ruth Dickstein. “The Library Skills Program at the University of
Arizona: Testing, Evaluation and Critique.” Journal of Academic Librarianship 5 (Sept. 1979):
205‐14.
92. Phillips, Linda L., and E. Ann Raup. ʺComparing Methods for Teaching Use of Periodical
Indexes.ʺ Journal of Academic Librarianship 4.6 (Jan. 1979): 420‐3.
93. Popa, Opritsa, Deborah A. Metzger, and James A. Singleton. ʺTeaching Search Techniques on
the Computerized Catalog and on the Traditional Card Catalog: A Comparative Study.ʺ
College and Research Libraries 49.3 (1988): 263‐74.
94. Portman, Chris A., and Adrienne Julius Roush. “Assessing the Effects of Library
Instruction.” Journal of Academic Librarianship 30.6 (Nov. 2004): 461‐5.
95. Prorak, Diane, Tania Gottschalk, and Michael Pollastro. ʺTeaching Method and Psychological
Type in Bibliographic Instruction: Effect on Student Learning and Confidence.ʺ RQ 33.4
(Summer 1994): 484‐95.
96. Reidelbach, Marie A., Dorothy B. Willis, Joan Latta Konecky, Ruth J. Rasmussen, and Joan
Evidence Based Library and Information Practice 2006, 1:3
34
Stark. ʺAn Introduction to Independent Learning Skills for Incoming Medical Students.ʺ
Bulletin of the Medical Library Association 76.2 (1988): 159‐63.
97. Ren, Wen Hua. ʺLibrary Instruction and College Student Self‐Efficacy in Electronic
Information Searching.ʺ Journal of Academic Librarianship 26.5 (2000): 323‐8.
98. Riddle, John S., and Karen A Hartman. ʺBut Are They Learning Anything?ʺ Designing an
Assessment of First Year Library Instruction.ʺ College and Undergraduate Libraries 7.2 (2000):
59‐69.
99. Ridgeway, Trish. Library Orientation Methods, Mental Maps, and Public Services Planning.
1983. ERIC Document Reproduction Service ED247942.
100.Robinson, Andrew M., and Karen Schlegl. ʺStudent Bibliographies Improve when Professors
Provide Enforceable Guidelines for Citations.ʺ portal: Libraries and the Academy 4.2 (2004):
275‐90.
101.Salisbury, Fiona, and Jenny Ellis. “Online and Face‐to‐Face: Evaluating Methods for Teaching
Information Literacy Skills to Undergraduate Arts Students.” Library Review 52.5 (2003):
209‐17.
102.Schilling, Katherine Lynne. “Information‐literacy Skills Development in Undergraduate
Medical Education: A Comparison Study of the Impact of Training Methodologies on
Learning Outcomes.” Diss. Boston University, 2002.
103.Schipper, Rachel Ann. ʺComputer‐assisted Instruction, Learning Style, Field Orientation,
Time Measurement, and Citizen Status: Bibliographic Instruction and College Freshmen.ʺ
Diss. Florida Institute of Technology, 2000.
104.Selegean, John Cornell, Martha Lou Thomas, and Mary Louise Richman. “Long‐range
Effectiveness of Library‐Use Instruction.” College and Research Libraries 44 (Nov. 1983): 477‐
80.
105.Steele, Godfrey A., and Ernesta Greenidge. “Integrating Medical Communication Skills with
Library Skills Curricula Among First Year Medical Students at the University of the West
Indies, St. Augustine.” Health Information and Libraries Journal 19.4 (Dec. 2002): 206‐13.
106.Stevens, Robert. AIMLO: Auto‐instructional Media for Library Orientation. Final Report. Ft.
Collins: Colorado State Univ., 1974. ERIC Document Reproduction Service ED105882.
107.Stewart, Linda, and Jan Olsen. “Compact Disk Databases: Are They Good for Users?” Online
12 (May 1988): 48‐52.
108.Sugranes, Maria, and James A. Neal. “Evaluation of a Self‐paced Bibliographic Instruction
Course.” College & Research Libraries 44 (Nov. 1983): 444‐57.
109.Suprenant, Thomas T. ʺLearning, Lecture, and Programmed Instruction Text: An Experiment
in Bibliographic Instruction.ʺ College and Research Libraries 43.1 (Jan. 1982): 31‐7.
Evidence Based Library and Information Practice 2006, 1:3
35
110.Swanson, Troy A. “A Radical Step: Implementing a Critical Information Literacy Model.”
portal: Libraries and the Academy 4.2 (Apr. 2004): 259‐73.
111.Tabur, Susanne, Rea Devakos, Sylvia Newman, Jennifer Bayne, Joan Bartlett, Elizabeth A.
Reid, Linda Devore, M. Lynn Russell, and Elizabeth Uleryk. “Peer Informatics Tutoring for
Medical Students: The Toronto Experience.” Bibliotheca Medica Canadiana 22.3 (Spring 2001):
121‐7.
112. Thaxton, Lyn, Mary Beth Faccioli, and Anne Page Mosby. “Leveraging Collaboration for
Information Literacy in Psychology.” Reference Services Review 32.2 (2004): 185‐9.
113.Tiefel, Virginia May. ʺEvaluating a Library User Education Program: A Decade of
Experience.ʺ College and Research Libraries 50 (1989): 249‐59.
114.Tierno, Mark J., and Joann H. Lee. “Developing and Evaluating Library Research Skills in
Education: A Model for Course‐integrated Bibliographic Instruction.” RQ 22 (1983): 284‐91.
115.Toifel, Ronald C., and Godfrey Franklin. “Using Technology to Teach Preservice Students
About Locating Information in the Academic Library.” Journal of Educational Technology
Systems 27.2 (1998‐99): 133‐45.
116.Vander Meer, Patricia F., and Galen E. Rike. “Multimedia: Meeting the Demand for User
Education with a Self Instructional Tutorial.” Research Strategies 14 (1996): 145‐58.
117.Vertrees, Robert L., and Marjorie E. Murfin. “Teaching the Legislative Process: An Evaluation
of Classroom and Library Instruction and a Legislative History Exercise.” Government
Publications Review 7A.6 (1980): 505‐15.
118.Wallace, Margaret C., Allison Storten, and Patrick A. Crookes. ʺTeaching Information
Literacy Skills: an Evaluation.ʺ Nurse Education Today 20.6 (2000): 485‐9.
119.Ware, Susan A., and Deena J. Morganti, ʺA Competency‐based Approach to Assessing
Workbook Effectiveness.ʺ Research Strategies 4 (Winter 1986): 4‐10.
120.Webster, Janet, and Loretta Riely. “A Library Instruction Case Study: Measuring Success
from Multiple Perspectives.” Research Strategies 19.1 (2003): 16‐32.
121.Wendt, Paul. A Study to Determine the Extent to Which Instruction to University Freshmen
in the Use of the University Library Can be Turned Over to Teaching Machines. 1963. ERIC
Document Reproduction Service ED003559.
122.Wiggins, Marvin E. Hands‐On Instruction in an Electronic Classroom. A Final Report to the
U.S. Dept. of Education of a Research and Development Grant Awarded To Establish a Fully‐
Equipped Electronic Training Room and Test the Effectiveness of Hands‐On Instruction in
Learning the NOTIS OPAC and Silver‐Platter ERIC. Provo, Utah: Brigham Young Univ., 1994.
ERIC Document Reproduction Service ED369391.
123.Williams, Mitsuko, and Elisabeth B. Davis. “Evaluation of Plato Library Instructional
Lessons.” Journal of Academic Librarianship 5.1 (Mar. 1979): 14‐19.
Evidence Based Library and Information Practice 2006, 1:3
36
124.Wood, Richard J. ʺThe Impact of a Library Research Course on Students at Slippery Rock
University.ʺ The Journal of Academic Librarianship 10.5 (1984): 278‐84.
125.Zahner, Jane Elizabeth. ʺA Cognitive Strategies Framework for Domain‐integrated Process‐
oriented Library Instruction: The Effects on Research Process Orientation, Library Anxiety,
Attitudes, and Research Products of College Students.ʺ Diss. Florida State University, 1992.
Evidence Based Library and Information Practice 2006, 1:3
37
Appendix C: Data Extraction Elements
Author – names of authors on paper
Date – date paper was published
Country and Institution – country and institution where research took place
Study Objective – the stated objective of the study, as relayed by the authors
Subject area – at the departmental level (i.e., Sociology, Medicine, Biology)
Participants – number of participants in study, other factors that are known about the
participants (university level of students, average age, prior educational level)
Evaluation Period – duration of the research project (i.e., 4 years, 6 weeks)
Learning Objectives – aims of teaching related to information literacy
Faculty Collaboration – Was the instruction based on a partnership between librarians and
faculty (collaboration/partnership/consultation), or was it solely librarian initiated. (yes/no) Did
the librarian and faculty member work together?
Integrated or Independent – Was the instruction integrated as part of an existing subject‐based
course, or was it an independent course or session? (integrated/independent)
Tie to student need – Was the instruction tied directly to student need, wherein the instruction
had a direct tie to a student assignment; or was the instruction indirectly tied to student need,
wherein the instruction gave examples and assignments that were similar to what they would be
expecting in their course; or was the instruction not tied to student need, using examples and
assignments which were of supposed interest to the student for future need. (direct/indirect/no
tie)
Contact time ‐‐ number of hours of instruction included within the study
Teaching method used – way of presenting instructional materials or conducting instructional
activities
Mode of Delivery – specific elements regarding how the instruction was carried out (i.e., lecture,
small‐group, study guide, computer lab exercises, face‐to‐face, WebCT module, online tutorial)
Instructional Topic – specific focus of the instruction (i.e., library orientation, using bibliographic
databases, doing research for a term paper)
Research methods used – type of study design. Also note details of the research methodology.
Learning Outcomes Measured – information literacy learning outcomes the researchers were
testing (i.e., improved ability to search online database, increased knowledge of appropriate
Evidence Based Library and Information Practice 2006, 1:3
38
information resources, and improved understanding of where to search for information). List the
outcomes stated and their categories, based on Bloom’s revised taxonomy (Anderson and
Krathwohl) of educational objectives for the Cognitive domain (‘Remember’, ‘Understand’,
‘Apply’, ‘Analyze’, ‘Evaluate’, ‘Create’).
ACRL Information Literacy Standard / Performance Indicator – cite the Standard and
Performance Indicator numbers as given in the ALA document: Information Literacy
Competency Standards for Higher Education
.
Point at which Learning Outcomes were Measured – when the test of learning outcomes took
place (i.e., immediately following instruction, 1 week after instruction, 2 years after instruction)
Results – results of research (include all numerical data)
Reported Outcome – outcome of research (Positive, Mixed, Neutral, or Negative), as reported by
authors (i.e., students using X method of instruction had higher test scores).
Implications of research – as suggested by the authors (i.e., X method of teaching should be used;
more research needs to be done on X).
Critical Appraisal Elements – using the Glasgow checklist (Morrison et al. 891).
http://www.ala.org/ala/acrl/acrlstandards/standards.pdf
Evidence Based Library and Information Practice 2006, 1:3
39
Appendix D: Characteristics of studies meeting quality filter
Quality filter: 1) comparative studies, 2) compared 2 different teaching methods, 3) outcomes
based on data assessed for statistical significance.
**Shaded rows are studies included in meta‐analysis
Abbreviations
AL = Active Learning SDIL = Self‐Directed Independent Learning
CAI = Computer‐Assisted Instruction TI = Traditional Learning
Study Teaching
Method
Used
Subject Area Research methods
used
Outcome
Measure
Study
Outcome
Sample
size
Active Learning
Bren 1998 AL vs. TI English
Literature
Post‐intervention,
single time point, with
control group.
Score on
post‐test
Positive 86
Cudiner 2001 AL vs. TI
with an AL
component
First Year
Experience
Controlled before‐
and‐after.
Score on
post‐test
Negative 13
Frasca 1992 AL with a
TI
component
vs. No
instruction
Medicine Post‐intervention,
single time point, with
control group.
Score on
post‐test
Positive 92
Koufogiannakis
2005
AL vs. no
instruction
Medicine and
Dentistry
Randomized
controlled trial
(before‐and‐after).
Score on
post‐test;
score on final
exam
Mixed 164
Prorak 1994 AL vs. TI English
Composition
Controlled before‐
and‐after.
Score on
post‐test
Neutral 246
Computer Assisted Learning
Alexander 2000
& 2001
CAI vs. TI Library Science Post‐intervention,
single time point, with
control group.
Final exam
grade
Neutral 88
Axeen 1967 CAI vs. TI Library Science Controlled before‐
and‐after.
Score on
post‐test
Neutral 66
Cherry 1991 CAI vs. TI
vs. No
instruction
English Controlled before‐
and‐after.
Score on
post‐test
search tasks
Positive 53
Cherry 1994 CAI vs. No
instruction.
N/A Randomized
controlled trial
(before‐and‐after).
Score on
post‐test
search tasks
Neutral 30
Churkovich
2002
CAI vs.
CAI with
mediation
vs. TI with
an AL
component
Sociology Randomized
controlled trial
(before‐and‐after).
Score on
post‐test
Negative 174
Germain 2000 CAI vs. TI
with an AL
component
First Year
Experience
Controlled before‐
and‐after.
Score on
post‐test
Neutral 303
Evidence Based Library and Information Practice 2006, 1:3
40
Gutierrez 2001 CAI with
a TI
component
vs. SDIL
with a TI
component.
Not stated Randomized
controlled trial
(before‐and‐after).
Score on
post‐test
Neutral 134
Holman 2000 CAI vs. TI
vs. No
instruction
English
Composition
Controlled before‐
and‐after.
Score on
post‐test
Neutral 125
Hooks 1986 CAI vs. TI Library Science Post‐intervention,
single time point, with
control group.
Score on
post‐test
Positive 19
Johnson 1980 CAI vs. TI
(2 different
modes) vs.
No
instruction
English Post‐intervention,
single time point, with
control group.
Score on
post‐test
Neutral 68
Kaplowitz 1998 CAI with a
SDIL
component
vs. TI with
a SDIL
component
Biology Controlled before‐
and‐after with follow‐
up.
Score on the
post‐test
Neutral 423
Lawson 1989 CAI vs. TI English Controlled before‐
and‐after.
Score on
post‐test
Positive 172
Madland 1988 CAI vs. TI
vs. No
instruction
Remedial
English
Controlled before‐
and‐after.
Score on
post‐test
Negative not stated
Nichols 2003 CAI vs. TI English
composition
Controlled before‐
and‐after.
Score on
post‐test
Neutral 64
Orme 2004 CAI vs. TI
vs. CAI
and TI vs.
No
instruction
Business Post‐intervention,
single time point, with
control group.
Scores on
post‐test;
searching
task; and
transcription
task
Neutral 128
Schilling 2002 CAI vs. TI Medicine Randomized
Controlled Trial
(before‐and‐after).
Score on
post‐test
Neutral 128
Vander Meer
1996
CAI vs.
SDIL with
a TI
component
University 101 Randomized
controlled trial
(before‐and‐after).
Score on
post‐test
Neutral 186
Learner‐centred Instruction
Donegan 1989 Learner‐
centred vs. TI
vs. No
instruction
Management Post‐intervention,
single time point,
with control
group.
Score on
post‐test
Neutral 156
Tabur 2001 Learner‐
centred vs. No
instruction
Medicine Before‐and‐after
study.
Score on
post‐test
Neutral 137
Self‐directed, Independent Learning
Broadway 1985 SDIL vs. No
instruction
Library science Controlled
before‐and‐after
with follow‐up.
Score on
post‐test
Positive 48
Evidence Based Library and Information Practice 2006, 1:3
41
Ellsbury 1981 SDIL vs. No
instruction
Agricultural and
Biological
Engineering,
Biological
Sciences,
Entomology
Controlled
before‐and‐after.
Score on
post‐test
Positive 155
Holt 1984 SDIL vs. TI vs.
No instruction
English
Composition
Controlled
before‐and‐after.
Score on
post‐test
Neutral approximately
112
Kirk 1971 SDIL vs. TI Biology Post‐intervention,
multiple time
points, with
control group.
Score on
bibliography;
essay grades
Neutral approximately
190
Marcus 2003 SDIL vs. TI Introduction to
College Life
Randomized
controlled trial
(after‐only).
Score on
post‐test
Neutral 203
Meehan‐Black
1981
SDIL vs. TI vs.
No instruction
Educational
Psychology and
Human
Development
Controlled
before‐and‐after.
Scores on
two post‐
tests
Positive 50
Phillips 1979 SDIL vs. TI Communications Randomized
controlled trial
(after‐only).
Score on
post‐test
Neutral 161
Phipps 1979 SDIL vs. No
instruction
English
Composition
Controlled
before‐and‐after.
Score on
post‐test
Positive 487
Ridgeway 1983 SDIL vs. TI vs.
No instruction
Writing Randomized
controlled trial
(after‐only).
Score on
post‐test
Neutral 149
Stevens 1974 SDIL,
comparing two
different
modes of
delivery vs. No
instruction
Psychology Post‐intervention,
single time point,
with control
group.
Score on
post‐test
Positive 502
Suprenant 1982 SDIL vs. TI vs.
No instruction
English Randomized
controlled trial
(before‐and‐
after).
Score on
post‐test
Positive 1,234
Wendt 1963 SDIL vs. TI vs.
No instruction
English Randomized
controlled trial
(before‐and‐
after).
Score on
post‐test
Neutral 193
Wood 1984 SDIL with a TI
component vs.
No instruction
Library Science Controlled
before‐and‐after.
Score on
post‐test
Positive 86
Traditional Instruction
Bolt 1986 TI with an AL
component vs.
No instruction
Library Science Longitudinal
retrospective
cohort with
control group.
Grade point
average
Neutral 572
Bostian 1990 TI, comparing
3 different
modes vs. No
instruction
Introduction to
the Academic
Community;
Business
Post‐intervention,
single time point,
with control
group.
Assessment
of search
strategies
Positive 56
Breivik 1977 TI with an AL
component vs.
TI vs. No
instruction
English Controlled
before‐and‐after.
Score on
final term
paper
Positive 130
Evidence Based Library and Information Practice 2006, 1:3
42
Buchanan 1992 TI vs. No
instruction
English
Composition
Post‐intervention,
single time point,
with control
group.
Score on
searching
exercises
Positive 137
Cooper Moore
2001
TI with an AL
component vs.
No instruction
English courses
accounted for
75% of the
sample.
Controlled
before‐and‐after.
Score on
post‐test and
bibliography
assessment
Positive 820
Currie 1982 TI (instruction
plus
compulsory
assignment vs.
instruction
only) vs. No
instruction
Biology;
Sociology
Post‐intervention,
single time point,
with control
group.
Score on
post‐test
Positive 406
Davis 1993 TI (compares 3
different
modes of
delivery), vs.
CAI
Freshman
Orientation;
English
Composition;
Education
Randomized
controlled trial
(after‐only).
Score on
post‐test
Mixed 220
Dodgen 2003 TI with an AL
component vs.
No instruction
Sociology Controlled
before‐and‐after.
Score on
post‐test
Mixed 294
Eyman 1977 TI vs. No
instruction
Library Science Controlled
before‐and‐after.
Score on
post‐test
Neutral 242
Fox 1996 TI with an AL
component vs.
No instruction
Nursing Multidimensional
program
evaluation.
Scores on
post‐tests;
graded
assignments
Positive A) not stated;
B) 276;
C) 112
Hardesty 1979 TI with a SDIL
component vs.
No instruction
English
Composition
Controlled
before‐and‐after.
Score on
post‐test
Positive 162
Hardesty 1982 TI with a SDIL
component vs.
No instruction
English
Composition
Longitudinal
cohort with
control group.
Scores on
skills tests
Positive 403
Lechner 1989 TI with a SDIL
component vs.
No instruction
Education Randomized
controlled trial.
Score on
post‐test;
score on
performance
test
Neutral 199
Nielsen 1987 TI with a SDIL
component,
comparing the
order in which
the modes of
delivery were
given, vs.
Placebo
N/A Randomized
controlled trial
(before‐and‐
after).
Score on
post‐test
Mixed 90
Robinson 2004 TI comparing
different
motivations
for an
assignment vs.
No instruction
Political Science Post‐intervention,
single time point,
with control
group.
Analysis of
bibliography
citations
Positive 84
Selegean 1983 TI with an AL
component vs.
No instruction
Library Science Longitudinal
cohort study with
control group.
Grade point
average;
student
persistence;
and
Positive 468
Evidence Based Library and Information Practice 2006, 1:3
43
graduation
rates
Toifel 1999 TI and AL vs.
TI with an AL
component vs.
No instruction
Education Controlled
before‐and‐after.
Score on
post‐test
Positive 145
Wallace 2000 TI with an AL
component vs.
No instruction
Nursing Controlled
before‐and‐after.
Score on
post‐test
Positive 127