Evidence Based Library and Information Practice 2011, 6.3 1 Evidence Based Library and Information Practice Evidence Summary Millennial Students’ Online Search Strategies are Associated With Their Mental Models of Search A Review of: Holman, L. (2011). Millennial students’ mental models of search: Implications for academic librarians and database developers. Journal of Academic Librarianship, 37(1), 19-27. doi:10.1016/j.acalib.2010.10.003 Reviewed by: Leslie Bussert Research & Instruction Librarian University of Washington Bothell, Cascadia Community College Campus Library Bothell, Washington, United States of America Email: lbussert@uwb.edu Received: 01 June 2011 Accepted: 02 Aug. 2011 © 2011 Bussert. This is an Open Access article distributed under the terms of the Creative Commons-Attribution-Noncommercial-Share Alike License 2.5 Canada (http://creativecommons.org/licenses/by-nc-sa/2.5/ca/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly attributed, not used for commercial purposes, and, if transformed, the resulting work is redistributed under the same or similar license to this one. Abstract Objective – To examine first-year college students’ information seeking behaviours and determine whether their mental models of the search process influence their ability to effectively search for and find scholarly materials. Design – Mixed methods including contextual inquiry, concept mapping, observation, and interviews. Setting – University of Baltimore, a public institution in Maryland, United States of America, offering undergraduate, graduate, and professional degrees. Subjects – A total of 21 first-year undergraduate students, ages 16 to 19 years, undertaking research assignments for which they chose to use online resources. Methods – First-year students were recruited in the fall of 2008 and met with the researcher in a university usability lab for about one hour over a three week period. The researcher observed mailto:lbussert@uwb.edu� Evidence Based Library and Information Practice 2011, 6.3 78 and videotaped the students as they conducted research in their chosen search engines or article databases. The searches were captured using software, and students were encouraged to think aloud about their research process, search strategies, and anticipated search results. Observation sessions concluded with a 10- question interview incorporating a review of the keywords the student used, the student’s reflection on the success of his or her searches, and possible alternate keywords. The interview also offered prompts to help the researcher learn about students’ conceptualizations of search tools’ utilization of keywords to generate results. The researcher then asked the students to provide a visual diagram of the relationship between their search terms and the items retrieved in the search tool. Data were analyzed by identifying the 21 different search tools used by the students and categorizing all 210 searches and student diagrams for further analysis. A scheme similar to Guinee, Eagleton, and Hall’s (2003) characterized the student searches into four categories: simple single-term searches, topic plus focus searches, phrase searches, and advanced searches employing multiple Boolean operators. Students’ diagrams were put into three different groups: process view, hierarchical view, and network view. The researcher then analyzed the relationships between the students’ search behaviours and their mental models to develop further conclusions. Main Results – Analysis revealed that this population of students had a limited mental model of the search process and used narrow sets of fairly simple search strategies for retrieving information online. Search engines were used for the majority (61.9%) of total searches and 72.3% of those conducted in search engines were in Google. The majority of students (76%) began their search process with a search engine while other students began searching in online encyclopedias (10%) or online databases (14%). Academic Search Premiere was used for 73.8% of the database searches. Some students (5%) also performed searches in individual websites (6.3%), for an overall total of 224 searches conducted. Students performed four varieties of searches: simple searches using short phrases conveying a single concept (34% of total searches); topic plus focus searches using a single Boolean AND (30%); phrase searches consisting of multiple- word descriptive phrases or sentence fragments (17.4%); and advanced Boolean searches combining two or more distinct concepts (13.8%). Generally, students used the same search terms and structure whether they were in a search engine or database, particularly with phrase searches. Nearly 71% of the advanced Boolean searches were inappropriately formed, particularly when used in the databases. Of the few students employing Boolean logic beyond a single AND, only two used it correctly, and only one with successful results. Students were unable to recognize or explain why a search failed or why they got the results they did. They made frequent incorrect use of punctuation, spelling, and syntax, leading to limited or no search results. Students assumed that obtaining few results indicated a problem of keyword choices rather than search query structure. When faced with no results in the databases, they assumed there were no articles on their topics and did not re-evaluate their search queries. Those with unsuccessful Boolean searches did not recognize that their errors were due to logic, and instead changed their keywords or began a new search altogether. Several students understood keywords as concepts versus literal strings of letters, yet thought the search tools determined search results based solely on what was typed into the search box. Of those employing phrase searches, some believed that each word was queried, while others thought only the “primary terms” were queried. Most students (61%) offered Evidence Based Library and Information Practice 2011, 6.3 79 analogies to print resources to explain how search engines process queries, and all the students’ descriptions included their ideas about what a search tool contained, rather than how the search tools organized information. Attempts to expand or narrow searches were haphazard. While most students (57%) employed the strategy of adding keywords to narrow searches, only a few (11%) recognized the function of this technique and used it regularly, while others tended to return to their original broad searches in a different tool. Some had a limited understanding that adding terms narrows and reducing terms broadens the search, but their Boolean errors negated the use of synonyms or alternative terms for those purposes. Other strategies included using the search tool’s “advanced search” features or quotes, although all who used the latter did so incorrectly and some mistakenly thought parentheses served the same purpose. All subjects drew representations of their views of the relationship between keywords used and search results retrieved, though few were able to clearly visualize how a search engine processes a query, or address ideas such as expanding or narrowing searches or synonym use. Three categories of diagrams emerged: the process view, hierarchical view, and network view. The process view displayed a task flow diagram. These students demonstrated the least formed mental models and experienced the search tool as a “black box” that gives results, showing little understanding of how they are generated. They performed the fewest overall searches (11.6%), the majority (79%) of which were simple or phrase searches with no use of Boolean operators. The hierarchical view displayed a broad subject with subtopics, or results highlighting specific aspects of the subject. These students performed nearly 30% of the total searches, 17% of which included the use of Boolean logic. The network view displayed models of interconnected terms. These students performed the majority of the searches (58.7%), and also constructed the most sophisticated queries. Many of their searches employed Boolean logic (83%), and 65% were either Boolean or topic plus focus searches. Students with this mental model tended to focus more on the queries themselves than the results received. Students indicated feelings of success in their searching and were comfortable relying on simple searches retrieving large results sets. While not central to the research design or driving questions, students’ evaluation of search results was observed and found to be weak. Students displayed rapid searching, scanning, and evaluation processes which may have played a role in many of their mistakes when repeating or attempting to correct faulty searches. Conclusion – The results show students did not have strong conceptual models of the search process or how search queries impacted results, and were often unable to recognize or troubleshoot problems with searches in order to improve results. Students displaying stronger mental models used more complex search strategies, but still performed unsuccessful searches and demonstrated challenges in remedying defective searches. Students skimmed search results quickly, rarely looking beyond the first two pages, and did not take time to evaluate them for topic relevance. The findings suggest that librarians should rethink how Millennial students are taught search strategies and evaluation, to focus more on problem solving or critical thinking. They also suggest that database developers should continue developing search algorithms and tools, considering this population’s conceptualizations of search. Further research on Millennials’ information processing, critical thinking, and evaluation skills in the context of academic work is needed. Commentary This study builds upon others investigating search conceptualization and behaviour in the Evidence Based Library and Information Practice 2011, 6.3 80 fields of library and information science and computer science. The literature review offers a context for Millennials’ information and technology behaviours and a foundation for understanding the role of mental models in information retrieval. The identified relationships between mental models and search skills can inform librarians’ approaches to teaching Millennials how to search, as well as the design or modification of user interfaces and functionality of search tools. The mixed methods approach facilitated the collection of multiple data from which the researcher could draw conclusions. The small sample size was admitted, yet other limitations or variables remain unaddressed, such as: how subjects were recruited; whether they received library instruction in prior educational settings; whether subjects were researching for the same assignment, warranting the same types of sources; whether the interview data was formally coded for analysis; or the possibility that subjects modified typical search behaviour due to being observed in close proximity. Some confusion is introduced through the organization of information presented in the article. Findings on narrowing searches are discussed across multiple sections when it would be clearer if it was all contained within one section. The author’s definition of “search engine” is also hard to track. The majority of the article uses this term to describe search tools freely available online, such as Google, and does not include library databases, yet in later sections the author appears to use it more broadly to incorporate both. This introduces a lack of clarity regarding the type of search tools around which the students’ mental models were constructed, and consequently a reader’s understanding of the findings may be confused. The provided recommendations for instruction librarians and database developers are valuable, yet not groundbreaking. Instruction librarians have emphasized critical thinking since the late 1980’s, and database vendors have been developing search discovery tools better suited for simple searching since 1998 (Bodi, 1988; WebFeat, 2011). It would have been helpful if instead the author had shared suggestions for instruction librarians on ways to teach students about relationships between search queries and results, or discussed how these findings might apply to other user populations or search behaviour in discovery and federated search tools. The author’s findings can also be illuminated when placed within the broader study of human information seeking behaviours. In particular, Mansourian and Ford (2007a, 2007b) have identified theoretical frameworks to help understand web-based information seeking behaviours such as satisficing, persistence in searching, and decision making. They utilized Simon’s (1955, 1956) bounded rationality theory, Cooper’s (1976) utility theory, and Kraft and Lee’s (1979) three stopping rules. They also framed their work around prior research relevant to this study and the practice of instruction librarians. For example, Ren (2000) identified a link between self-efficacy and search performance and found that search training can increase self-efficacy, while Thompson, Meriac, and Cope (2002) identified links between self- efficacy and the number of items correctly retrieved, as well as increased searcher persistence when specific task instructions (versus general directions) were provided. References Bodi, S. (1988). Critical thinking and bibliographic instruction. Journal of Academic Librarianship, 14(3), 150-153. Cooper, W. S. (1976). The paradoxical role of unexamined documents in evaluation of retrieval effectiveness. Information Processing and Management, 12(6), 367- 375. Evidence Based Library and Information Practice 2011, 6.3 81 Guinee, K., Eagleton, M. B., & Hall, T. E. (2003). Adolescents’ internet search strategies: Drawing upon familiar cognitive paradigms when accessing electronic information sources. Journal of Education Computing Research, 29(3), 363-374. Kraft, D. H., & Lee, T. (1979). Stopping rules and their effect on expected search length. Information Processing and Management, 15(1), 47-58. Ren, W. H. (2000). Library instruction and college student self-efficacy in electronic information searching. Journal of Academic Librarianship, 26(5), 323-328. Mansourian, Y. and Ford, N. (2007). Web searchers’ attributions of success and failure: An empirical study. Journal of Documentation, 63(5), 659-679. doi: 10.1108/00220410710827745 Mansourian, Y. and Ford, N. (2007). Search persistence and failure on the Web: A bounded rationality and satisficing analysis. Journal of Documentation, 63(5), 680-701. doi: 10.1108/00220410710827754 Simon, H. A. (1955). A behavioral model of rational choice. Quarterly Journal of Economics, 69(1), 99-118. Simon, H. A. (1956). Rational choice and the structure of the environment. Psychological Review, 63(2), 129-138. Thompson, L. F., Meriac, J.P., & Cope, J. G. (2002). Motivating online performance: The influences of goal setting and internet self-efficacy. Social Science Computer Review, 20(2), 149-160. WebFeat. WebFeat: A brief history. Retrieved 29 May 2011 from http://www.webfeat.org/about.htm / Evidence Based Library and Information Practice