Evidence Based Library and Information Practice Evidence Based Library and Information Practice 2010, 5.4 39 Evidence Based Library and Information Practice Article Letting Students Take the Lead: A User-Centred Approach to Evaluating Subject Guides Kimberley Hintz Humanities & Social Sciences Librarian University of British Columbia Library Vancouver, British Columbia Email: kimberley.hintz@ubc.ca Paula Farrar Art + Architecture + Planning Librarian University of British Columbia Library Vancouver, British Columbia Email: paula.farrar@ubc.ca Shirin Eshghi Japanese Language Librarian University of British Columbia Library Vancouver, British Columbia Email: shirin.eshghi@ubc.ca Barbara Sobol Learning Services Librarian (Research) University of British Columbia Library Kelowna, British Columbia Email: barbara.sobol@ubc.ca Jo-Anne Naslund Education Librarian University of British Columbia Library Vancouver, British Columbia Email: joanne.naslund@ubc.ca Teresa Lee Health Sciences Librarian University of British Columbia Library Vancouver, British Columbia Email: teresa.lee@ubc.ca Tara Stephens cIRcle 2010 Olympics and Paralympics Librarian University of British Columbia Library Vancouver, British Columbia Email: tara.stephens@ubc.ca Aleha McCauley Community Business Services Librarian Irving K. Barber Learning Centre at the University of British Columbia Vancouver, British Columbia Email: aleha.mcauley@ubc.ca Received: 21 June 2010 Accepted: 23 Oct. 2010 2010 Hintz et al. This is an Open Access article distributed under the terms of the Creative Commons- Attribution-Noncommercial-Share Alike License 2.5 Canada (http://creativecommons.org/licenses/by-nc- sa/2.5/ca/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly attributed, not used for commercial purposes, and, if transformed, the resulting work is redistributed under the same or similar license to this one. Abstract Objective – What do students need and want from library subject guides? Options such as Web 2.0 enhancement are now available to librarians creating subject-specific web pages. Librarians may be eager to implement these new tools, but are such add-ons a priority for mailto:kimberley.hintz@ubc.ca� mailto:paula.farrar@ubc.ca� mailto:shirin.eshghi@ubc.ca� mailto:barbara.sobol@ubc.ca� mailto:joanne.naslund@ubc.ca� mailto:teresa.lee@ubc.ca� mailto:tara.stephens@ubc.ca� mailto:aleha.mcauley@ubc.ca� Evidence Based Library and Information Practice 2010, 5.4 40 students? This paper aims to start a dialogue on this issue by presenting the findings of the University of British Columbia (UBC) Library’s Subject Guides Working Group (SGWG), which was tasked with assessing current library subject guides in order to make recommendations for the update and future development of UBC Library subject guides. Methods – The working group solicited feedback through a questionnaire that was distributed to undergraduate and graduate students from a variety of disciplines at UBC. The questionnaire included an evaluation of UBC subject guides, as well as guides from other academic libraries that used various platforms such as LibGuides and SubjectsPlus. Results – Respondents to the student questionnaire indicated that a simple and clean layout was of primary importance. Students also desired succinct annotations to resources and limited page scrolling. Meanwhile, few students identified Web 2.0 features such as rating systems and discussion forums as being important for their needs. The working group used the questionnaire data to create a “Top Ten” list of student recommendations. Conclusions – The “Top Ten” list of student recommendations was combined with stakeholder feedback from faculty, liaison librarians and Library Systems and Information Technology representatives to create the SGWG’s final recommendation for subject guide revision and enhancement. For the SGWG these findings called into question the necessity of Web 2.0 technologies within subject guide pages and highlighted the need for further research on the topic of subject guide usability and effectiveness. Introduction Academic libraries often employ the term “subject guides” to describe web pages created by liaison librarians that include subject- specific content meant to assist students or faculty in their research. Subject guides often contain links to online resources, information about print resources, research tips, citation style guides, as well as other information relevant to a particular research area. Subject guides may also potentially include Web 2.0 interactive multimedia tools such as video tutorials, tagging and user polls, and incorporate blogs and social networking applications. Although the content of a subject guide is of primary concern to liaison librarians, as with any website or online resource, the presentation of that content both positively and negatively impacts a student’s ability to access the information at their point of need. At the University of British Columbia (UBC) Library, subject guides have been considered an important element of liaison librarian work, and a critical means by which the library communicates information about its collections and services to its users. Despite this perceived importance, the layout and functionality of UBC Library subject guides had not been reviewed for at least 10 years, and many of the emerging Web 2.0 technologies were not readily implementable within the existing platform. In the spring of 2008, UBC librarians expressed a desire to refresh the look of the Library’s subject guides, improve their functionality and take advantage of recent Web 2.0 developments that were key features of commercial subject guide platforms such as LibGuides. A LibGuides Review Working Group was formed to assess LibGuides and its suitability for use in the UBC Library system. This working group quickly came to the conclusion that in order to make an informed decision on the future direction of the subject guides, it was necessary to compare a variety of subject guide platforms, and more importantly, consult with our key stakeholders. Believing that the assessment should be user-driven, the LibGuides Review Working Group was Evidence Based Library and Information Practice 2010, 5.4 41 disbanded, and the Subject Guides Working Group (SGWG) was formed. In July 2008, the newly formed SGWG developed a revised task list: review the current subject guide literature, investigate the various platforms available, gather feedback from the key stakeholders, and, finally, make a recommendation for the future development of UBC Library’s subject guides. At the initial meeting of the SGWG much of the discussion revolved around how to gather feedback from key stakeholders. It was decided that input from liaison librarians and systems staff should be solicited through invitational meetings. However, the group determined that a more wide-reaching and evidence based approach was necessary for the student feedback component, as students have been and will continue to be the primary target audience for subject guides. With this in mind, the SGWG decided that a questionnaire distributed to a large sampling of students from various disciplines would be appropriate. Using questionnaire data in conjunction with a "wish list" of needs from the liaison librarian feedback sessions, the SGWG met with systems staff to explore platform options, including proprietary subject guide software systems, open source tools and in-house solutions. At the end of this process the SGWG proposed a series of recommendations for the future directions of UBC Library’s subject guides. These recommendations addressed the needs of library stakeholders, while at the same time providing the best possible resource for library users over the long term. This article provides a review of the literature, an explanation of the student questionnaire methodology, and an analysis and discussion of the data, including a “Top Ten” list of priorities. How this information was used towards the SGWG’s final recommendations is then discussed. Literature Review The research literature on subject guides can be divided into three prominent categories that demonstrate those areas which have received the most attention: first, the history or evolving purpose of subject guides; second, the challenges of subject guides, both technical and administrative; third, a focus on subject guide use. Finally, the topic of Web 2.0 technologies has emerged as a trend that crosses these categories and is something our research addresses. The history or purpose of subject guides is well covered within the literature. Smith (2008) provides a comprehensive overview of the evolution of guides from the late nineteenth-century to the present, including descriptions of the many iterations subject guides have taken along the way. The origin of subject guides lies in descriptive bibliographies designed to facilitate the use of resources (Smith, 2008, pp. 512-514). Tchangalova and Feigley (2008) list a plethora of terms for what basically falls into the category of subject guides: research guides, research tools, pathfinders, electronic library guides, e-guides, webliographies, subject portals, etc. While the style of delivery has changed substantially, the basic purpose of providing a starting point for the researcher remains unchanged. Much of the literature on subject guides concerns the challenges that they pose to librarians. Issues of creation (policy, technology), maintenance (division of labour, technology, technical expertise) and overall administration are common in the literature (Darby, 2006; Jackson, Blackburn, & McDonald, 2007; Prentice, 2009; Staley, 2007; Wales, 2005). A trend within this category of the literature concerns the technical requirements of staff to create and maintain subject guides (Buczynski, 2009). This issue is tied to the frequency with which guides need to be updated, and the common practice in most libraries of migrating print guides to the online environment through static webpages. Recent technological developments, such as server side include (SSI) statements, allow for new or revised content to be “pushed” to web pages alleviating the need for manual updates. As these types of technologies are Evidence Based Library and Information Practice 2010, 5.4 42 implemented in libraries, many of the technical concerns surrounding the updating of subject guides should be mitigated (Buczynski, 2009; Goans 2006; Greene, 2008; Northrup & Ashmore, 2006). The administering of guides also emerges as a dominant trend in the literature with a focus on issues regarding standardization (Tchangalova & Feigley, 2008), time and consistency (Jackson, Blackburn, & McDonald, 2007, p. 20, p. 33). The emergence of commercial and open source subject guide products such as LibGuides and Library à la Carte, respectively, directly address these types of concerns as they streamline the technical and administrative aspects of subject guides (Griggs 2009; Montgomery, 2009; Smith, 2008). Given that subject guides are innately user- focused, the literature indicates that subject guide use is less studied than one might expect. This sentiment is echoed by Staley, who describes the state of research on subject guide usage as “still at an exploratory stage” (Staley 2007, p. 122). An overview of the literature reveals that three types of questions are often asked: How are subject guides used? Who uses them? And, are they useful? Staley focuses on “actual use and perceived usefulness of subject guides” amongst specific groups of students (Staley, 2007, p. 130). Griggs details how Library à la Carte was created with user input integrated into its development (Griggs, 2009, p. 14). Courtois, Higgins and Kapur (2005) surveyed user satisfaction with subject guides. Reeb and Gibbons (2004) recommend delivering subject guide content at the course level to better match students’ mental models of information. These studies on subject guide use are insightful, but there is clearly more work to be done on these issues. Woven through each of these dominant categories of subject guide research is the recurrent theme of incorporating emerging Web 2.0 technologies within libraries (Kellam, Cox, & Winkler, 2009, p. 350). Various approaches to integrating Web 2.0 features have been explored and these are frequently referred to in the literature as “subject guide 2.0”. While no official definition of this term exists, Yang (2009) takes her meaning from the “unspoken consensus that subject guides 2.0 are those created with Web 2.0 technologies” (p. 91). However, Yang is quick to observe that other writers on the subject, such as Meredith Farkas and Ellyssa Kroski, place more emphasis on usage and interactivity than on the technologies themselves (p. 91). The diverse means of distributing librarian- produced content, as well as facilitating user- generated content, therefore, take precedence in much of the subject guide 2.0 literature. Yang (2009) provides an extensive catalogue of subject guide 2.0 characteristics, ranging from multimedia features to statistics reporting, and a comparison chart that lists those features included in common subject guide content management systems. Allan (2009) offers a two page guide to increasing the “functionality and depth” of your subject guide with Web 2.0. Others advocate using Web 2.0 tools as the very basis of subject guides (Dodson, 2008; Jackson, Blackburn, & McDonald, 2007). The appeal of platforms such as LibGuides is that these Web 2.0 features are built-in (Judd, 2009; Moses, & Richard, 2008). Web 2.0 technologies are extensively discussed in the professional literature and the assumption is that by adding Web 2.0 features to library resources, specifically subject guides, there will be a perceptible improvement viewed as such by students. Curious about the validity of these assumptions, we asked students to assess an array of subject guides, some with Web 2.0 features, and some without. The purpose of our questionnaire, therefore, was to find out what students really want from subject guides. Methods A questionnaire was used to gather data from students as to what subject guide features, content, and design would be most helpful in serving their research and study needs (see Appendix A). The SGWG obtained a minimal risk certificate of approval from the Evidence Based Library and Information Practice 2010, 5.4 43 Behavioural Research Ethics Board at UBC’s Office of Research Services before proceeding with the project. Prior to distributing the questionnaire, it was piloted in two trial sessions: one with six participants and the other with four. These trial sessions provided an opportunity to test the length and format of the questionnaire, as well as to determine venue requirements. A total of 55 questionnaires were administered in the computer labs of six branch libraries during Winter Session Term II. The six branches chosen serve different disciplines (humanities, social sciences, and sciences) and user groups (UBC Vancouver and UBC Okanagan students). Handbill flyers in each branch were used to advertise the study and $5 UBC AMS Food Outlets gift certificates were offered as incentives for participants. These sessions were conducted during lunch hour and early afternoon time periods to take advantage of peak student traffic in the branch libraries. Most respondents completed the questionnaire in twenty to thirty minutes. Completion of the questionnaire required respondents to read a consent form and anonymously record their personal background information including: • student status (undergraduate, graduate, unclassified) • number of years attending UBC (0-5+) • program, department, or faculty • frequency of use of the UBC library website (never, sometimes, frequently, always). The first section of the questionnaire asked students the following open-ended question: If you could create a webpage that would help you with your research or course work, what are the kinds of things that you would put on it? The respondents could list, sketch, or draw what they would like to see on such a web page. The second section of the questionnaire involved three parts. The first part asked respondents to examine three sample subject guides selected from different libraries. To ensure that participants were exposed to a wide variety of features available on different subject guides while giving each participant a manageable number of samples to view in the time allotted, the subject guides were grouped and administered as two sets (28 of Set A and 27 of Set B). Each set included the following: 1. A UBC subject guide (static web page) 2. A LibGuides subject guide 3. A subject guide created using open source software (eg. Library à la Carte or SubjectsPlus) The second part asked students to record their comments on printed copies of the guides. The third part asked students to rate each subject guide using a Likert Scale of 1-5 on the following: • Comprehension (from 1-Very unclear to 5-Very clear) • Visual Appearance (from 1-Boring to 5-Gets my interest) • Content (from 1-Not Useful to 5-Very Useful). The final question in this section had students select which subject guide they would most likely use if available in their subject area. They were also asked to provide a reason for their choice. The purpose of the data analysis of the completed questionnaires was to identify common elements in content, design and functionality, not to compare specific platforms. To decrease potential bias in the analysis of the qualitative components of the questionnaires, two members of the SGWG coded the data independently. The results were then compared and discrepancies in coding between the reviewers were discussed until reviewers came to a consensus. In the end a total of 40 separate codes were used to quantify the data. Findings Demographics From the 55 completed questionnaires we learned that undergraduate students represented 78% of the participants and the remaining 22% were graduate students. The Evidence Based Library and Information Practice 2010, 5.4 44 surveyed population showed a relatively even distribution in terms of years spent at UBC, with 9% at less than 1 year; 28% at 1 year; 16% at 2 years; 16% at 3 years; 20% at 4 years; and 11% at 5 years. Participants were asked to indicate their subject areas and again, a relatively even distribution was achieved with 42% self-identifying as students in the Arts, 29% from the Sciences, and 20% from Education. A significant percentage of the students (46%) used the library website at least occasionally and only 9% reported that they never consulted the website for research or coursework. Student Preferences Student feedback on the open-ended question ranged from scribbled keywords to more elaborate commentary about specific features. For example, respondent #19 commented: “Highlight seminal articles in the field of research. i.e. The most cited, most discussed, on which other papers follow.” Although students were encouraged to list or sketch ideas, nearly all chose textual description rather than visual representation. Given the similarity between comments in this section and other narrative commentary provided in parts two and three of the second section, the reviewers decided to use one coding scheme to analyze all sections. The second section of the questionnaire asked participants to evaluate three separate subject guides. One of the most telling findings of the study showed that when asked which guide they would use if it were available in their subject area, 20 of the 55 respondents (36%) chose a UBC guide, a much higher number than expected. These results suggest that the UBC subject guides were not as outdated and unusable as we had assumed. We reviewed the responses of the 35 participants who chose other guides as their preferred format and found an interesting correlation. When respondents were asked to compare three subject guides based on comprehension, visual appearance, and content using the 1-5 Likert Scale, UBC’s guides rated high for content and comprehension but significantly low on visual appeal (Fig. 1). 0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 UBC Guide Open Source LibGuides UBC Guide 3.97 2.58 4.02 Open Source 3.82 3.02 3.58 LibGuides 4.04 3.84 4.11 Comprehension Visual Appearance Content Fig. 1. The mean values for comprehension, visual appearance and content are represented, indicating UBC’s guide received significantly lower marks for visual appearance. Evidence Based Library and Information Practice 2010, 5.4 45 When asked to choose their favourite subject guide, participants generally chose the one they had ranked as highest in visual appearance. This relationship showed that while content and comprehension are important, visual appeal can be a deciding factor in determining which guides students would most likely use. To learn more about what constitutes visual appeal, we analyzed the commentary provided and responses to other parts of the questionnaire. The data analysis used a total of 40 codes to represent the general themes expressed in the narrative comments provided by participants. While all of the data was useful, importance was placed on ideas or features that received numerous comments. In the end, the working group decided to focus on the ten codes that had received the most comments—the resulting “Top Ten” list would form the basis for future development recommendations (Fig. 2). Student Priorities In first place, by a large margin, students mentioned the importance of a simple or clean layout. A total of 47 of the 55 respondents made comments on the simple structure, layout and use of white space on the guide. As an example, when questionnaire respondent # 30 (a senior undergraduate student) was asked to choose his/her favourite guide and tell us why, the response was: “The format is very clear and neat. The first two webpages just make me feel dizzy. Too many words on it – very messy.” The overwhelmingly positive response for this characteristic showed that basic principles of good web design must be followed. Analysis also revealed students’ desire for more succinct, jargon-free resource descriptions. We also noted from our results that navigation was extremely important as three of the top ten features included the use of tabs, section headings and keeping pages to a manageable page length to limit the amount of scrolling—all pointing to common usability considerations. Fig. 2. “Top Ten” list of student priorities indicates the number of times (frequency) a particular element was given positive feedback on the questionnaires. 47 31 31 23 22 22 21 19 18 16 Codes Fr eq ue nc y simple layout/clean annotations search feature embedded instruction content easy to understand librarian contact info tabs citation info section headings length Evidence Based Library and Information Practice 2010, 5.4 46 At the same time, participants noted that short annotations describing resources were highly desirable. Participants on 31 questionnaires responded positively to this feature. Additional feedback regarding length of the page and length of annotations suggested that students are not willing to read an annotation more than a sentence or two long. Participants also expressed a desire to see embedded instruction linked to resources in the form of brief tutorials. Students wanted direction and help with these resources, indicating that they did not want to simply be pointed to a resource; they wanted to be told how best to make use of it. Students also liked seeing a search box, the inclusion of librarian contact information, and help with citing their sources on the guides. They wanted immediate access to search results and assistance in finding someone to help them with their questions. Their interest in having citation style information available on the subject guide is a reflection of students’ desire for a resource that provides guidance in each stage of the research process. This theme was also evident in their comments regarding embedded instruction features, including chat and online tutorials, each of which was coded separately. Interestingly, while several codes were used to record comments about Web 2.0 elements, none of them appeared in the “Top Ten” list. Students appeared skeptical about rating systems, discussion forums, student recommendations, and they showed little interest in personalization features. Discussion While the results of the student questionnaires provided valuable insight into the preferences and opinions of our primary stakeholders, the limitations of the study must be addressed. Constraints in terms of staff time, resources, and recruiting participants resulted in a small sample size that could not be considered sufficiently representative of the diverse student population at UBC. However, the SGWG employed several strategies to increase participation from students across disciplines and levels of study: • Six questionnaire sessions were conducted in six of the Library’s busiest 21 branches, including the UBC Okanagan Library and in the Irving K. Barber Learning Centre, which houses a café, computer lab, and several academic support departments including the library. • Sessions were conducted on six separate dates and at 3 separate times (11am, 12pm or by appointment) to encourage participation from students with disparate schedules and, therefore, different programs and levels of study. Despite these measures to capture a variety of responses across the student body, it was not possible to guarantee accurate representation from all student groups, nor was it possible to determine whether participants were more experienced than the general student population as library users, and therefore more familiar with UBC Library and other subject guides. Another possible instance of familiarity introducing bias is the fact that though participants were encouraged to evaluate the merit of each guide independently of its institutional context, students were likely to be best acquainted with the UBC guides. With this in mind, the SGWG members conducting questionnaire sessions were provided with a standardized script instructing participants to consider the overall usefulness of the subject guide in terms of style, format and types of content rather than who developed the guide or its applicability to their area of study (see Appendix B). Given the limitations of our study, the data collected from the questionnaires were interpreted not as a definitive representation of what our students need or want from subject guides, but rather as a way to broaden our understanding of what elements appeal to students and why. Not surprisingly, the questionnaire results confirm the importance of simplicity and clarity in the design and content of subject guides. Students Evidence Based Library and Information Practice 2010, 5.4 47 consistently emphasized a need for a clean and simple layout, with tabs, section headings, and other clear navigational signposts. Respondents also expressed a desire for easy- to-understand content that is comprehensible both in terms of navigational structure and clarity of language. Additionally they emphasized a desire for careful selection over quantity. Generally, the findings from our study confirm Strutin's intuitions that students' compulsion to rely on Google and Wikipedia as starting points for research has to do with "speed and simplicity"; "familiarity and consistency"; and "know[ing] how to navigate quickly" (2008). On the other hand, our findings suggest that we have yet to fully understand users' needs with respect to subject guides. In the current literature, the inclusion of Web 2.0 technologies in subject guides tends to be viewed uncritically as a positive development. Yang noted, for example, that user-generated content is needed: "Collaboration among faculty, students, and librarians is necessary in creating and maintaining subject guides; therefore subject guide 2.0 must have the capability to receive and incorporate input from users" (2009, p. 92). Further, Yang suggests that in being able to enter into online student territory, LibGuides, the most complete CMS in terms of Web 2.0 functionality, can only be beneficial: "By integrating with FaceBook, Delicious.com, and Twitter, LibGuides advertises and distributes the contents of the subject guides to internet social networking sites, places that students frequent" (p. 94). However, there appears to be little evidence to support the efficacy of this kind of exposure and integration. In fact, our findings suggest that students do not welcome or even understand all the Web 2.0 features that can possibly be embedded into subject guides. Despite our small sample, data from the questionnaires suggests that Web 2.0 features need to be examined more critically. A pattern that appeared to emerge from our findings was the conceptualization of Web 2.0 features into two broad categories: 1) features that promote peer-to-peer interaction and learning (student recommendations, forums and personalization/customization) and 2) features that enable students largely to receive authoritative advice (librarian chat window, embedded tutorials/handouts giving advice on how to use resources). Although we initially suspected that collaborative elements such as forums and popularity functions would rank highly as enhanced features, students were largely unreceptive to these tools. In fact, responses often indicated that students found these features confusing. The following comments were received regarding the rating system functionality: “Don’t understand its intention.”, “What do these mean? Who rates them?”, and “Stars lead researchers in certain ways which might be misleading.” Both the quantitative data and the subtext of many of the participant comments revealed that students are more interested in finding authoritative information from accepted experts (librarians and faculty members) rather than in using subject guides as a site for their own knowledge production and interaction with peers. In short, students come to subject guides expecting to be firmly guided towards the materials and conventions of accepted scholarly practice. Future research, therefore, needs to move beyond simply advocating for the integration of Web 2.0 features into subject guides to a more critical and selective approach for adopting features and functionalities based on appropriate evaluation. These findings provided the SGWG with much to consider in revising UBC Library’s subject guides. The working group’s recommendation to implement a locally- developed Content Management System (CMS) rather than LibGuides, a popular commercial product, emerged directly from and reflects student feedback, internal contexts and priorities, as well as the needs expressed by librarians as primary content creators. Evidence Based Library and Information Practice 2010, 5.4 48 Conclusions The Subject Guide Working Group began this study with the goal of understanding what our primary and secondary stakeholders needed from subject guides with regard to both content and mode of delivery. In addition to a review of the literature and available subject guide platforms, we solicited feedback from librarians and library staff and developed a questionnaire for students. The insights gained from these questionnaires proved invaluable to the SGWG as we investigated future directions for UBC subject guides. While many of the comments and viewpoints expressed by students reinforced our own perceptions of what needed to be improved or revised on our subject guides, we could not adequately predict which available features would appeal most to students and why. Once these findings were compared to the needs and preferences articulated by librarians and library staff, it became evident that an in- house subject guide CMS could be built with the features and functionality outlined in our “Top Ten” list. Moreover, the potential for greater control over customization and integration with existing local systems provided by an in-house CMS added an additional level of appeal over a commercial or open-source product despite the extensive list of subject guide 2.0 features currently available. Yang’s comprehensive checklist includes an impressive selection of options for tools and functionality such as multimedia, multi-formats, ease of use, global change, search boxes, link checking, integration with social bookmarking sites, RSS feed, tagging, interactivity and real time chat, user input, blogs, wikis, and statistics reporting (2009, p. 97). Despite the vast possibilities for content production and delivery offered by these features, we discovered that the students who completed the questionnaire favoured those that enabled the provision of authoritative advice and instruction, such as librarian contact information and embedded tutorials. This unexpected response raised a number of questions about the benefits of Web 2.0 features in subject guides and the need for a better understanding of appropriate contexts in which these features may best be used. There are an increasing number of platforms that are capable of integrating various Web 2.0 features into library subject guides. We would like to see more studies that assess the effectiveness of subject guide 2.0 features in academic libraries from users’ perspectives. Rigorous research is needed to help us learn more about the usage of subject guides, the resources and features utilized within them, and to better understand how to guide our users to resources that enable them to meet their research goals. As libraries re-evaluate and recreate subject guides to incorporate new technologies —many for the first time since transitioning to online from static print —we encourage decision makers to carefully consider local users’ perspectives, goals, needs, and real usage of subject guides before investing additional resources, money, and time into the direction of subject guide 2.0. References Allan, C. (2009). Evolve your research guides into the web 2.0 ecosystem. Texas Library Journal, 85(1), 20-21. Buczynski, J. A. (2009). Online web development platforms enable all reference staff to work on subject guides. Internet Reference Services Quarterly, 14(3), 61-66. Courtois, M. P., Higgins, M. E., & Kapur, A. (2005). Was this guide helpful? Users' perceptions of subject guides. Reference Services Review, 33(2), 188-196. Darby, A. (2006). Implementing an open source application in a college library: ECU's pirate source. College & Undergraduate Libraries, 13(1), 41-52. Farkas, M. (2007). Subject guide 2.0. American Libraries, 38(5), 33. Evidence Based Library and Information Practice 2010, 5.4 49 Goans, D., Leach, G., & & Vogel, T. (2006). Beyond HTML: Developing and re- imagining library web guides in a content management system. Library Hi Tech, 24(1), 29-53. Greene, A. (2008). Managing subject guides with SQL server and ASP.net. Library Hi Tech, 26(2), 213-231. Griggs, K. (2009). Library information made to order: An open source project built for and with librarians. Computers in Libraries, 29(2), 12-47. Jackson, M., Blackburn, J. D., & McDonald, R. H. (2007). Media wiki open-source software as infrastructure for electronic resources outreach. Reference Librarian, 48(1), 19-36. Judd, C., & Montgomery, N. M. (2009). LibGuides and librarians: Connecting content and community. Kentucky Libraries, 73(3), 14-17. Kellam, L. M., Cox, R., & Winkler, H. (2009). Hacking blackboard: Customizing access to library resources through the blackboard course management system. Journal of Web Librarianship, 3(4), 349-363. Moses, D., & Richard, J. (2008). Solutions for subject guides. Partnership: The Canadian Journal of Library & Information Practice & Research, 3(2), 1-9. Northrup, L. A., & Ashmore, B. (2006). Creating easy to update subject guides without using a database. College & Undergraduate Libraries, 13(1), 53-57. Prentice, K., Gaines, J., & Levy, L. (2009). New "starting points" for resources by subject. Medical Reference Services Quarterly, 28(1), 88-97. Reeb, B. &. Gibbons, S. (2004). Students, librarians, and subject guides: Improving a poor rate of return. Portal: Libraries and the Academy, 4(1), 123-130. Smith, M. M. (2008). 21st century readers' aids: Past history and future directions. Journal of Web Librarianship, 2(4), 511-523. Staley, S. M. (2007). Academic subject guides: A case study of use at San Jose State University. College & Research Libraries, 68(2), 119-139. Strutin, M. (2008, Fall). Making research guides more useful and more well used. Issues in Science & Technology Librarianship, 55. Tchangalova, N., & Feigley, A. (2008). Subject guides: Putting a new spin on an old concept. Electronic Journal of Academic & Special Librarianship, 9(3), 1-1. Wales, T. (2005). Library subject guides: A content management case study at the Open University, UK. Program: Electronic Library & Information Systems, 39(2), 112- 121. Wrosch, J. (2007, June 28). Open source software options for any library. MLA Forum, 5(3). Yang, S. Q. (2009). Subject guide 2.0: A dream or reality? Journal of Library and Information Science, 35(1), 90-98. Evidence Based Library and Information Practice 2010, 5.4 50 Appendix A Student Questionnaire UBC Library Subject Guides Questionnaire A A) Background Information 1. Student status: Undergraduate Graduate Unclassified 2. How many years have you been at UBC? 0 1 2 3 4 5+ 3. What program, department or faculty are you in? ________________________ 4. How often do you use the UBC Library website for your research or course work? Never Sometimes Frequently Always B) Questions 1. [Estimated time: 3-5 minutes] If you could create a webpage that would help you with your research or course work what are the kinds of things that you would put on it? (Hint: List ideas and/or sketch what the page would look like). 2. [Estimated Time: 10-15 minutes] A) Take a few minutes to look at the three sample subject guides listed in Group A at: http://www.library.ubc.ca/finearts/sgwg/samples.html Evidence Based Library and Information Practice 2010, 5.4 51 B) Using the printed copies of the guides, let us know what you like or dislike about each guide. Feel free to highlight, circle, mark up, make notes, etc... C) Please rate each subject guide by circling the appropriate number ranking: ------------------------------------------------------------------------------------------------- 1. University of British Columbia – Environmental Studies/Sciences Comprehension: Very unclear 1 2 3 4 5 Very clear Visual Appearance: Boring 1 2 3 4 5Gets my Interest Content: Not useful 1 2 3 4 5 Very useful ------------------------------------------------------------------------------------------------- 2. University of Victoria – History Comprehension: Very unclear 1 2 3 4 5 Very clear Visual Appearance: Boring 1 2 3 4 5 Gets my Interest Content: Not useful 1 2 3 4 5 Very useful ------------------------------------------------------------------------------------------------- 3. Dalhousie University – Biology Comprehension: Very unclear 1 2 3 4 5 Very clear Visual Appearance: Boring 1 2 3 4 5 Gets my Interest Content: Not useful 1 2 3 4 5 Very useful ------------------------------------------------------------------------------------------------- D) Of the subject guides you have just reviewed, which one would you be most likely to use, if available in your subject area? 1 2 3 None of the above Please tell us why you chose this answer: Appendix B Standardized Script for Administering the Student Questionnaire Introduction Script for Student Questionnaires Preamble Thanks for coming today to participate in this UBC Library Research Project. My name is ____________ and joining me today are librarians from across campus: (introduce everyone present) The reason we’ve invited you here is to get your input on how to make the library’s subject guides better. Subject guides are webpages created by the subject librarians to guide you in your research or course work. Evidence Based Library and Information Practice 2010, 5.4 52 Before we get started, we’d like to ask you to read the consent form. (2 min.) Now you have the option to either complete the questionnaire here or take it with you, complete it and return it to _____________ at __________ Library within the next three days (date: __________). If you would like to continue now, please fill out Section A of the form. (2 min.) ________________________________________________________________________ Brainstorming The session today consists of two main parts. In this first part, we’d like you to brainstorm a bit about what you would consider an ideal subject guide (and this can be about any research topic or subject). Please sketch out or jot down how you would envision this guide. We’ll give you about 5 minutes to do this. (6 min.) Responding to examples In this second part, we’re going to ask you to respond to three examples of subject guides, one from UBC and two from other university libraries. We’ve chosen subject guides in __________________________. Although these topics may not be directly relevant for you, we’re interested in getting your feedback about how generally usefully they are. Take a look at each webpage online and at the print outs we’ve given you. Feel free to mark up the print outs by circling, highlighting or noting things that you find helpful or interesting or not useful. Then, complete section C and D, using the rating scales provided. You have about 10 min. for this section. (12 min.) / Evidence Based Library and Information Practice/ / /