Evidence Based Library and Information Practice 2011, 6.3 1 Evidence Based Library and Information Practice Evidence Summary Students are Confident Using Federated Search Tools as much as Single Databases A Review of: Armstrong, A. (2009). Student perceptions of federated searching vs. single database searching. Reference Services Review, 37(3), 291-303. doi:10.1108/00907320910982785 Reviewed by: Deena Yanofsky Liaison Librarian Walter Hitschfeld Geographic Information Centre McGill University Montreal, Quebec, Canada Email: deena.yanofsky@mcgill.ca Received: 4 Mar. 2011 Accepted: 15 July 2011 © 2011 Yanofsky. This is an Open Access article distributed under the terms of the Creative Commons-Attribution-Noncommercial-Share Alike License 2.5 Canada (http://creativecommons.org/licenses/by-nc-sa/2.5/ca/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly attributed, not used for commercial purposes, and, if transformed, the resulting work is redistributed under the same or similar license to this one. Abstract Objective – To measure students’ perceptions of the ease-of-use and efficacy of a federated search tool versus a single multidisciplinary database. Design – An evaluation worksheet, employing a combination of quantitative and qualitative questions. Setting – A required, first-year English composition course taught at the University of Illinois at Chicago (UIC). Subjects – Thirty-one undergraduate students completed and submitted the worksheet. Methods – Students attended two library instruction sessions. The first session introduced participants to basic Boolean searching (using AND only), selecting appropriate keywords and searching for books in the library catalogue. In the second library session, students were handed an evaluation worksheet and, with no introduction to the process of searching article databases, were asked to find relevant articles on a research topic of their own choosing using both a federated search tool and a single multidisciplinary database. mailto:deena.yanofsky@mcgill.ca� Evidence Based Library and Information Practice 2011, 6.3 54 The evaluation worksheet was divided into four sections: step-by-step instructions for accessing the single multidisciplinary database and the federated search tool; space to record search strings in both resources; space to record the titles of up to five relevant articles; and a series of quantitative and qualitative questions regarding ease-of-use, relevancy of results, overall preference (if any) between the two resources, likeliness of future use and other preferred research tools. Half of the participants received a worksheet with instructions to search the federated search tool before the single database; the order was reversed for the other half of the students. The evaluation worksheet was designed to be completed in one hour. Participant responses to qualitative questions were analyzed, codified and grouped into thematic categories. If a student mentioned more than one factor in responding to a question, their response was recorded in multiple categories. Main Results – Participants indicated a slight preference for using a federated search tool over a single multidisciplinary database. Of the 31 students who completed the evaluation worksheet, 16 (51.6%) found that their search results were more relevant in the federated search tool; 10 (32.3%) students reported that the articles they found were more relevant in the single database. Three students stated that both search tools produced equally relevant results and two students responded that neither resource produced relevant results on their topic. When asked to state which resource they would be likely to use in the future, 22 students (71%) indicated that they would use the federated search tool and 21 (67.7%) students answered that they would use the single multidisciplinary database. Of the participants who expressed potential use of the single database to look for articles in the future, 43% referred to the ease of use or efficiency of the search tool in their responses. Similarly, more than half of the students who stated that they would use the federated search tool in the future (54.5%) also cited ease of use or efficiency in their answers. In total, 11 students (35.5%) stated that they would be unlikely to use the federated search tool for future research. In their responses, students referred to the inefficiency or complexity of the research tool. Of the 12 participants (38.7%) who stated that they would be unlikely to use the single multidisciplinary database, 50% cited a lack of relevant results and 42% referred to the overall complexity and inefficiency of the database. Conclusions – The results of this study do not support a significant preference among undergraduate students for either search tool. Though some participants struggled with terminology or various features of each resource, more students expressed confidence and satisfaction with the search process no matter which tool they opted to use. Given student confidence and comfort level in both research environments, the author suggests that librarians should place equal weight on both types of resources in library instruction. Commentary The aim of this study was to measure undergraduate student search preferences for using a federated search tool compared to navigating a single multidisciplinary database. Unlike other usability studies of metasearch tools, interpretations of effectiveness, efficiency and user satisfaction were based solely on student first impressions; the students themselves determined the usefulness and quality of search results after only initial and brief exposure to both resources. Although the author does acknowledge some of the limitations of this approach, the overall quality of the study is uneven and does not provide a significant contribution to the literature on metasearch environments. Evidence Based Library and Information Practice 2011, 6.3 55 While the study was designed to measure the functional abilities of students using two different metasearch tools for the first time, it is unclear how many students were in fact first- time users or whether students were indeed novice researchers. Participants were enrolled in two sections of English 161, a required first-year course at the UIC. The author states that “most” of the participants were first-year college students; she does not provide data to support this statement or clarify how many students fall into this category. The total response rate is also not reported. Given that the underlying assumption of this study is that first experiences indicate future use of a search tool, the fact that the study does not make clear each student’s level of education or verify their previous experience with each search tool is a substantial omission. The author reports that of the 31 students who took part in this study, 21 indicated that they might use the single multidisciplinary database to look for articles in the future and 22 students answered that they would be likely to use the federated search tool. While it is clear that several students responded with confidence in both resources, the study does not separate the number of students who responded positively to both search tools from the number of students who experienced difficulties or frustrations with one or both tools. A comparison would have given more weight to the study’s findings, making it possible to determine whether the responses indicated an actual preference for one tool or feature over the other. It is worth noting that although the results of this study are limited by several significant methodological errors, the goal was simply to discover whether subjects would be as comfortable, confident and functional using a federated search tool as using individual databases. It is safe to conclude that students feel self-sufficient and confident regardless of which search tool they opt to use. These findings indicate that information literacy models should focus on transferability, one of the desired outcomes of information literacy instruction and a core component of digital literacy. The tendency for students to self-report educational success and confidence using online information and communication tools is well documented in the literature. In order to provide adequate information literacy skills for today's students, librarians must encourage the development of critical thinking skills through activities that emphasize the nature of information and the information-seeking process as a whole, regardless of the tools being used or the topic being studied. / Evidence Based Library and Information Practice Evidence Summary Students are Confident Using Federated Search Tools as much as Single Databases A Review of: Armstrong, A. (2009). Student perceptions of federated searching vs. single database searching. Reference Services Review, 37(3), 291-303. doi:10.1108/00907320910982785 Received: 4 Mar. 2011 Accepted: 15 July 2011 Abstract Objective – To measure students’ perceptions of the ease-of-use and efficacy of a federated search tool versus a single multidisciplinary database. Design – An evaluation worksheet, employing a combination of quantitative and qualitative questions. Setting – A required, first-year English composition course taught at the University of Illinois at Chicago (UIC). Subjects – Thirty-one undergraduate students completed and submitted the worksheet. Methods – Students attended two library instruction sessions. The first session introduced participants to basic Boolean searching (using AND only), selecting appropriate keywords and searching for books in the library catalogue. In the second librar... The evaluation worksheet was divided into four sections: step-by-step instructions for accessing the single multidisciplinary database and the federated search tool; space to record search strings in both resources; space to record the titles of up to... Participant responses to qualitative questions were analyzed, codified and grouped into thematic categories. If a student mentioned more than one factor in responding to a question, their response was recorded in multiple categories. Main Results – Participants indicated a slight preference for using a federated search tool over a single multidisciplinary database. Of the 31 students who completed the evaluation worksheet, 16 (51.6%) found that their search results were more rele... When asked to state which resource they would be likely to use in the future, 22 students (71%) indicated that they would use the federated search tool and 21 (67.7%) students answered that they would use the single multidisciplinary database. Of the ... In total, 11 students (35.5%) stated that they would be unlikely to use the federated search tool for future research. In their responses, students referred to the inefficiency or complexity of the research tool. Of the 12 participants (38.7%) who sta... Conclusions – The results of this study do not support a significant preference among undergraduate students for either search tool. Though some participants struggled with terminology or various features of each resource, more students expressed con... Given student confidence and comfort level in both research environments, the author suggests that librarians should place equal weight on both types of resources in library instruction. Commentary The aim of this study was to measure undergraduate student search preferences for using a federated search tool compared to navigating a single multidisciplinary database. Unlike other usability studies of metasearch tools, interpretations of effectiv... While the study was designed to measure the functional abilities of students using two different metasearch tools for the first time, it is unclear how many students were in fact first-time users or whether students were indeed novice researchers. Par... The author reports that of the 31 students who took part in this study, 21 indicated that they might use the single multidisciplinary database to look for articles in the future and 22 students answered that they would be likely to use the federated s... It is worth noting that although the results of this study are limited by several significant methodological errors, the goal was simply to discover whether subjects would be as comfortable, confident and functional using a federated search tool as us...