Microsoft Word - ES 1460_Haglund Evidence Based Library and Information Practice 2008, 3:2 48 Evidence Based Library and Information Practice Evidence Summary Students with Non-Proficient Information Seeking Skills Greatly Over-Estimate Their Abilities A Review of: Gross, Melissa, and Don Latham. "Attaining Information Literacy: An Investigation of the Relationship between Skill Level, Self-Estimates of Skill, and Library Anxiety." Library & Information Science Research 29.3 (2007): 332-53. Reviewed by: Lotta Haglund Head of Information and Public Relations Karolinska Institutet, University Library, Stockholm, Sweden Email: lotta.haglund@ki.se David Herron Scholarly Developer Karolinska Institutet, University Library, Stockholm, Sweden Email: david.heron@ki.se Received: 25 February 2008 Accepted: 23 April 2008 © 2008 Haglund and Herron. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Abstract Objective – The objective of this study is an investigation of the relationship between students’ self-assessment of their information literacy skills and their actual skill level, as well as an analysis of whether library anxiety is related to information skill attainment. Design – Quantitative research design (Information Literacy Test (ILT), Library Anxiety Scale (LAS), pre and post surveys). Setting – Florida State University, United States. Subjects – Students, incoming freshmen. Methods – Information literacy skills were measured using the Information Literacy Test (ILT), presenting subjects with 65 multiple choice items designed around four of the five ACRL information literacy standards, in which students were expected to: 1) determine the nature and extent of the information needed; 2) access needed information effectively and efficiently; 3) 49 evaluate information and its sources critically and incorporates selected information into his/her knowledge base system; 4) understand many of the economic, legal and social issues surrounding the use of information and accesses and uses information ethically and legally. The ILT categorized participant scores as non- proficient (<39), proficient (30-53), or advanced (54 or higher). The test had earlier been validated and tested for reliability (Cameron, Wise & Lottridge). Library anxiety was measured using the Library Anxiety Scale (LAS). The LAS test asks participants to rate 43 statements using a Likert-type scale, with ratings from 1 to 5. The scale can also be used to produce five subscales, identified as 1) barriers with staff; 2) affective barriers; 3) comfort with the library; 4) knowledge of the library; 5) mechanical barriers. Fifty percent of the students were given the ILT first, and the other half of the respondents was given the LAS first, in order to neutralize the effect of the ILT itself. The two tests were sandwiched between pre- and post-surveys that were designed to collect demographic information, measure self-assessment of skills, and gather information about exposure to information literacy skills instruction. The students were asked to estimate their performance on the ILT both before and after taking the test, in terms of the expected percent of questions they would be able to answer correctly, the number of questions they would answer correctly, and how their performance would compare to the scores of the other incoming freshmen (in percentages). The students participating in the study represented the top and the bottom 25 % of the incoming class (as assessed by GPA and SAT/ACT scores) who were participating in summer session. They were contacted via e- mail and asked to participate in the study. There were 51 participants in the study, 33 in the top quartile and 18 in the bottom quartile. They were given the incentive of a gift certificate for the university bookstore, and that those scoring in the top 15 % of all participants would be included in a raffle for an additional 4 gift certificates. Also, a response time effort analysis was performed, data produced being consistent with times suggesting all participants were engaged in the test. All data were coded and analyzed using SPSS, for all tests alpha was set at p=0.05. Main Results – The main aim of the study was to test the hypothesis that students who test non-proficient on an information literacy test tend to overestimate their competency to a higher degree than proficient and advanced students. In the pre- and post-surveys, the students were asked to estimate their performance on the ILT in terms of the expected percentage of questions they would answer correctly, the number of questions they expected to answer correctly, and how their performance on the ILT would compare to others taking the test (in percentage). The results of the study show that all students overestimate their abilities, both in terms of performance and relative performance, in the pre-survey. The estimated percentage correct answers for the whole group was 75%, but the actual percentage correct was 65%. The estimated score was 50 and the actual 39, and the estimated comparison with their peers was 77% and the actual 53%. All three measures demonstrated a significant difference between estimated and actual values. On the ILT, the mean score for the bottom tier of the students was 34, and the mean score for the top tier was 42, showing a significant difference between the top and 50 bottom tier in a t-test. On the ILT, 23 students scored less than 39 (= non- proficient), 27 scored between 39-53 (proficient). Only one student, a top quartile participant, showed advanced information literacy skills (a score above 53). In the post-survey, the students still overestimated their performance, but to a lesser degree. All three groups adjusted their self-estimates in the post-survey in response to information skills testing, but the non-proficient group overestimated their skills to a higher degree, on both pre-and post-surveys. The estimated percentage correct answers for the whole group was 69%, but the actual was 65%, the estimated score was 44 and the actual 39, and the estimated comparison with their peers was 70% and the actual 53%. All three measures demonstrated a significant difference. All results show that the original hypothesis holds: non-proficient students overestimate their abilities to a greater degree, both in terms of score and performance relative to their peers. The LAS was used to see if there was a relationship between student scores on the ILT and library anxiety. Bi-variate analysis was performed on the ILT scores and student total scores on the LAS, and the results show that library anxiety tended to decrease with higher scores on the ILT. This result was not expected from the theory. In the pre-survey, the students were asked how they had obtained their information literacy skills. The top tier indicated a reliance on more formal sources (e.g. school library media center, classroom, and/or public library), while the bottom tier relied on less formal sources (e.g. friends and self- teaching). This suggested that library instruction was not effective reaching the weakest group of students. In addition, nearly 75% of all participants responded that they were largely self-taught. Conclusions – The results show a difference between the level of information literacy skills between lower-performing and higher-performing freshmen, but the study as a whole gives some evidence that many students could be information illiterate when entering university (45% tested non- proficient). There is an association between scores on an information literacy test and students’ estimates of their information literacy skills. Students who demonstrate low-level skills hold even more inflated views of their abilities, and the very competent may tend to underestimate their performance. However, the limited number of subjects in this study makes further analysis impossible. No correlation was found between performance on the ILT and the experience of library anxiety as measured by the LAS, although the subscale “knowledge of the library” demonstrated a negative relationship with information literacy skills. That is, the higher the score, the less the anxiety. There are several limitations of the study (which the authors acknowledge). One limitation that is important to note the small sample size, which limits the possibility to generalize the findings to the broader population of incoming students. More research is needed on how to best design, market, and deliver information literacy education to students with non- proficient information literacy skills since they seem to be unaware of their deficiencies. Commentary This article was exceptionally well written making it a pleasure for the reviewers to read. The study design seemed to be very rigorous. However, there are nevertheless 51 some minor shortcomings of the study, described in the following sections. A good case for the design of the study is made, providing good evidence for the validity of the study. Although the study can be judged as reliable, the assessment of reliability would benefit from additional information about the total student population. One also wonders on which grounds the authors have decided on the preferred 60 respondents (n=51). To enhance the understanding of the reader, the tables and figures of a text should complement each other, and this is not always the case in this article. The titles to the figures should be more descriptive, making them easier to interpret. The presentation of the study results is sometimes repetitive, in text and figures, especially noted in relation to Figure 1 and 2 and Tables 4-6. This somewhat confusing presentation is a pity since here lies the climax of the results, and the conclusion of the study would have been clearer had the results been presented in a less confusing way. Another confusing factor is that the authors discuss the figures in the text, jumping from one to the other without referring to them directly by name. In relation to Figure 2, one wishes that the authors had explained the calculations behind it, since this would have made it easier to understand. On Page 345, the mean estimate comparing performance to other incoming freshmen by percentile is given as 73 (SD=11.853, n=21). This must be a mistake since the number cannot be found in any of the tables, and neither 73 nor n=21 can be found in Table 6. On a deeper pedagogical level, one can wonder about the usefulness of testing information literacy skills out of subject context, since this study describes assessment of cognitive skills in a decontextualized and non-subject specific environment. It would also have been interesting to see how the ILT relates to a taxonomy of knowledge, for example, Bloom’s taxonomy, or the now popular (in Europe) SOLO taxonomy. However, these issues are probably beyond the scope of the study as well as this evidence summary. Some of the authors’ conclusions also seem somewhat outside of the span of the article. For example, the authors suggest that the study “presents a new way of thinking about how to improve student learning in both traditional and distance learning environments,” and this assertion is unsupported. However, this does not influence the overall assessment of the article. The authors also conclude that this study, together with other similar recent studies, might form a basis for additional research on how to set up information literacy education in innovative ways. This may well be the case, and library practice would benefit from this; hitherto, no studies have been able to give an answer to the question of which method of user education works best and for which user group (Brettle). Works Cited Brettle, Alison. “Information Skills Training: A Systematic Review of the Literature.” Health Information and Libraries Journal 20.S1 (2003): 3-9 Cameron, L. and S.L. Wise and S.M. Lottridge. “The Development and Validation of the Information Literacy Test.” College and Research Libraries 68 (2007): 229- 36.