Redesigning Research Guides: Lessons Learned from Usability Testing at the University of Memphis ARTICLE Redesigning Research Guides Lessons Learned from Usability Testing at the University of Memphis Jessica McClure, Carl Hess, and David Marsicano INFORMATION TECHNOLOGY AND LIBRARIES | SEPTEMBER 2023 https://doi.org/10.5860/ital.v42i3.15535 Jessica McClure (jmcclre3@memphis.edu) is the Virtual Instruction Librarian, University of Memphis. Carl Hess (carl.hess@memphis.edu) is the Undergraduate Success Librarian, University of Memphis. David Marsicano (dmmrscno@memphis.edu) is a Library Assistant II, University of Memphis. © 2023. ABSTRACT At the University of Memphis, a team of librarians and library staff formed the Research Guides Redesign Team (RGRT) to redesign, organize, and evaluate the University Libraries’ (UL) research guides. The purpose of the project was to ensure that the new design of the research guides homepage was intuitive to use. While it is impossible to ensure absolute usability for every user, this usability study attempts to eradicate the most common interface issues in community experiences at the University of Memphis. The RGRT conducted usability testing to evaluate the effectiveness of the new standardized format, grouped headings, and the appearance of the interface. The RGRT worked within the limitations of Springshare’s software to create the design and then chose five users to complete various task scenarios. Upon analysis of the users’ ability to complete the tasks, the RGRT discovered that overall, the design was effective, but they did make a few minor changes. This study describes the process and includes the original design, the new design, edits made after usability testing was conducted, and plans for future testing. INTRODUCTION In the spring of 2021, the Research Guides Redesign Team (RGRT) assembled to establish a new workflow for maintaining and designing research guides at the University of Memphis. Previously, all University Library (UL) faculty (hereafter “librarians”) were tasked with creating and maintaining research guides for academic programs and courses in their liaison areas in addition to their other duties. All librarians took part in the liaison program but did not have extensive training in guide creation. As a result, many guides consisted of lists of resources without instructions on how to use them, did not encourage information-seeking behavior, or did not cover basic information literacy skills.1 While some librarians had administrative privilege over the research guides software (Springshare’s LibGuides product), no one was tasked with reviewing or evaluating the guides holistically. Consequently, librarians were creating guides for their liaison departments without considering whether the information they included was covered in existing guides, and many were not regularly updated. The RGRT was created to solve these issues. It allowed for a more centralized workflow for creating and maintaining guides. It comprised a smaller group of librarians and library staff volunteering to take the baton from library liaisons in creating, updating, and deleting guides based on the needs of all patrons at the University of Memphis. The role of liaison librarians in the process changed to providing content expertise for their assigned departments’ guides as needed. mailto:jmcclre3@memphis.edu mailto:carl.hess@memphis.edu mailto:dmmrscno@memphis.edu INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2023 REDESIGNING RESEARCH GUIDES 2 MCCLURE, HESS, AND MARSICANO The RGRT selected the redesign of the research guides homepage as the first project. The homepage (illustrated in figs. 1, 2, and 3) had become an unwieldy list, organized by a mix of topic headings and headings reflecting University college and school names. Users had to scroll through this list to find guides, which the RGRT decided was clunky and inefficient. There was a search bar for locating guides, but it was in the upper right of the page away from the guide list (see fig. 1–3). Despite this long list of guides, users were often choosing between very similar guides with redundant content and long lists of undifferentiated resources, and if a u ser found a guide, it was not guaranteed to be up to date and often did not follow a consistent format. Improving this experience became the focus of the RGRT. While the RGRT considered starting with usability testing on the old homepage, it determined this was not necessary. The guides included many of the common usability issues in LibGuides as described by Thorngate and Hoden including “inconsistent design from page to page and from guide to guide,” “cluttered pages lacking a focal point,” and “too much content, not appropriately scoped to the task at hand”; and anecdotally, librarians and staff in the past had expressed frustrations with trying to find useful guides to share with patrons while navigating through the clutter of the homepage.2 Further, the RGRT began the project of homepage and guide redesign early in the COVID-19 pandemic. This would have required recruiting and running usability testing remotely, since the campus was primarily engaging in virtual instruction, adding difficulty to the project. The pandemic virtual instruction environment also gave the RGRT an added sense of urgency to quickly design a more usable research guide environment, since the UL’s digital learning objects and digital services had become the primary tool for patrons to interact with the UL. Performing virtual usability testing was a less efficient use of time when a wide variety of usability issues with the old setup was already known. The homepage redesign project took place simultaneously with a project to reorganize the subject guides, consolidate related academic programs to limit the amount of redundant content, focus more on teaching users how to use the resources recommended in the guides, and create a more manageable number of guides both for users and for the RGRT to update. All guides now have a consistent format and include instructional materials (videos, tutorials, etc.) to teach users how to use the resources in each guide. This study describes the creation and testing conducted on the new design of the r esearch guides homepage and the redesigned subject guides. To assess the success of the RGRT’s work on the new homepage, the researchers’ objectives for this study were to: • Identify whether users could effectively navigate the new homepage structure, inclu ding categorization and search features. • Determine whether the new structure allowed users to identify useful resources with a minimum of attempts and clicks. • Demonstrate that the updated design facilitated and encouraged users’ information- seeking behavior. INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2023 REDESIGNING RESEARCH GUIDES 3 MCCLURE, HESS, AND MARSICANO Figure 1. The research guides homepage before the redesign, pt. 1. INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2023 REDESIGNING RESEARCH GUIDES 4 MCCLURE, HESS, AND MARSICANO Figure 2. The research guides homepage before the redesign, pt. 2. INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2023 REDESIGNING RESEARCH GUIDES 5 MCCLURE, HESS, AND MARSICANO Figure 3. The research guides homepage before the redesign, pt. 3. LITERATURE REVIEW Online learning is an established format of higher education and there is an increase in the number of students attending classes online.3 This growth has implications for the use of library digital materials. A Primary Research Group study of online library services revealed that 10 out of 37 academic libraries reported that distance students utilized research guides and tutorials more than traditional students.4 Another stated the use of research guides increased by 56 percent in 2020.5 Colorado Christian University also used the COVID-19 pandemic as an opportunity to redesign its websites and research guides.6 Ghaphery and White’s study states that “LibGuides are as commonplace as books” in academic libraries, reiterating a need for gathering more usage statistics on them.7 Based on this data, increasing the effectiveness and usefulness of these library digital materials is imperative to expand the scope of the library’s reach to all users, not just those on ground. It is clear research guides are one of the primary ways academic libraries share their resources with the community and that ongoing maintenance and attention is essential, particularly to online users. The authors of this study wanted to know how topic-based navigation would work for the University of Memphis and wanted to conduct usability testing based on this design. The value of usability testing in libraries is well documented.8 At California State University, Vargas discovered that using a topic-based navigation design is user-friendly and helps patrons “get used to” the site layout.9 Furthermore, Polger observed that the website and not the building is the user’s first interaction with the space and is often deemed the “face” of the library.10 Designers of library web pages must organize information in a way that is intuitive for all users. This includes faculty, staff, undergraduate and graduate students, visiting researchers, and those with disabilities. While it is an impossible task to ensure absolute usability for every user, this usability study attempts to eradicate the most common interface issues in community experiences at the University of Memphis. INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2023 REDESIGNING RESEARCH GUIDES 6 MCCLURE, HESS, AND MARSICANO Redesigning research guides as a more holistic way of sharing resources with faculty and students is an effective way of designing. In the past, many research guides were lists of key reference sources; catalogs for finding books, theses, and dissertations; and periodical databases for finding journal articles and news articles—without much guidance on how to use them.11 Similarly, the authors of a five-step usability study with research guides discovered that a significant challenge was that the “purpose and nature of Research Guides was not readily evident to users” and the library jargon used was confusing to users as well.12 An example of a more holistic design would be one guiding students through the research process.13 Replacing library vocabulary with more commonly used language is also recommended.14 Bowen et al. stated, “Consequently, the questions of what to put on a guide and how best to arrange that material have driven an entire research agenda for the better part of a decade.” Their study has similar objectives to this one regarding the ease of use and navigation menus. They elected to use an A/ B method of usability testing, presenting students with two different versions of the same page. Their study was an excellent example of providing a format that would assist users with comprehending the content instead of having to understand the design itself.15 For this study, the RGRT adopted the five-user assumption commonly practiced among user experience professionals. The five-user assumption posits that, as you run usability testing, each additional user after the first will find usability problems already discovered by an earlier user in addition to discovering new problems. After the fifth user, the amount of repeat problems found will be so much greater than the number of new problems discovered as to render further testing an inefficient use of time and resources.16 Borsci et al. addressed and tested the debate concerning the five-user assumption and its established adoption. They observed that, while it is widely accepted, human-computer interaction professionals are split into two camps: those who apply and accept the model and those who are critical of it but apply it nevertheless. They proposed an alternative model called the “grounded procedure,” but still recommended the five-user assumption as a starting point.17 Additionally, the RGRT adopted the use of task scenarios for usability testing to gain valuable insights from users. Rather than setting goals such as “Find a website on citation,” it is more helpful to design a task that presents context. Therefore, a task scenario lets the user know why they need to find something and not simply how. McCloskey recommended designing tasks based on what a user would realistically do when visiting the website. She recommended making the task “actionable” to set up the context of why a user would need to visit the site.18 The success of McCloskey’s recommended actionable tasks are observed in many other usability studies.19 REDESIGN PROCESS To create the design, the RGRT began by independently investigating other universities ’ research guides homepage designs and reporting back to the group on their findings. The RGRT was looking for a model for improving categorization and guides, limiting the use of long lists, and foregrounding search. After investigating a few candidates, the group unanimously agreed that the University of Arizona’s research guides homepage (see fig. 4) should be the model.20 The design was effective because the content was not excessive, scrolling was unnecessary, and headings were descriptive. The visual aspects were not purely decorative and helped the patron understand the purpose of the content. The RGRT was given approval from the University of Arizona to model the new research guides homepage after theirs.21 In this model, a topic-based navigation design would organize sections into Research Help Guides, Subject Guides, Topic Guides, and Course Guides. Every guide would only be included in one of these four categories. Research Help Guides covered information literacy and library use skills, Subject Guides covered major academic https://libguides.library.arizona.edu/library-guides INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2023 REDESIGNING RESEARCH GUIDES 7 MCCLURE, HESS, AND MARSICANO programs, and Course Guides were for specific classes. Topic Guides was a catch -all area for guides that did not easily fit in the other categories. For each category, a smaller list of the most highly used guides in that category, based on Springshare user statistics and to be updated regularly, would be displayed on the homepage, followed by a link to a complete list of guides in the category. At the request of librarians at the University of Memphis’s three branch libraries (Health Sciences, Lambuth, and Music), guides connected to these branches were also organized together. The bottom of the homepage was dedicated to an Ask a Librarian help box. Figure 4. University of Arizona research guides homepage. To create the redesigned subject guides, the RGRT identified a course guide created by the University of Memphis’s instruction curriculum coordinator as the design standard. It included tutorials, videos, and other instructional materials guiding students on how to use the linked content.22 Each guide would have a standardized set of content pages based on this model, including redirect pages for Introduction to Research and Writing Help guides. The RGRT created a list of subject guides and used it to identify redundant guides that could be consolidated into these new subject guides. Each guide would be a broad subject area (e.g., Business & Economics, Social Sciences), and the main page would cover commonly used resources such as major databases for that area. More specialized resources would be included on topic pages for subareas reflecting specific academic programs (e.g., Accountancy, Psychology). In the spring of 2021, the RGRT completed the design for the homepage (see fig. 5). It was populated with the new subject guides (see fig. 6 for an example) that followed a standard format (see fig. 7 for the page-level organization). INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2023 REDESIGNING RESEARCH GUIDES 8 MCCLURE, HESS, AND MARSICANO Figure 5. Initial redesigned research guides homepage. Figure 6. Business & Economics subject guide homepage. INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2023 REDESIGNING RESEARCH GUIDES 9 MCCLURE, HESS, AND MARSICANO Figure 7. Business & Economics subject guide navigation. USABILITY TESTING This study was exempt from IRB (Institutional Review Board) approval and qualitative data using task scenarios was gathered to gain insights on the new research guides homepage and its linked pages. Task scenarios 1, 2, 3, and 5 were designed to reflect real user needs and required access to a variety of different guides, which would involve either using the guides homepage structure or using the LibGuides search. Task scenario 4 was designed to see if users identified the Ask a Librarian section at the bottom. With a laptop and a series of task scenarios (see table 1), a usability testing center was set up at the University of Memphis’s Starbucks. For recruitment, two of the authors asked various students in line and socializing at tables to participate. A $5 Starbucks gift card was awarded to participants. Data from five recruited participants was gathered. Before beginning the testing session, one of the authors acted as facilitator, explaining the testing was to ensure the website was intuitive and set up in a way that is helpful. She assured the participants that their knowledge was not being tested. Instead, the goal was to make sure that the web page was user-friendly and accessible. The participants were encouraged to use a think-aloud method when completing tasks, so the author acting as note-taker could transcribe the thought process of each participant.23 INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2023 REDESIGNING RESEARCH GUIDES 10 MCCLURE, HESS, AND MARSICANO Table 1. Task scenarios Task #1 You need to find information on MLA citation. How would you use this page to find that information? Task #2 You are in a course titled SOCI 4420: Racial Inequality. How would you use this page to find information on that course? Task #3 How would you use this page to find historical newspapers to use in a research project? Task #4 If you were struggling to access a library resource, how would you use this page to solve the problem? Task #5 If you wanted a video explaining how to find articles to use in your research assignment, how would you find it? The same series of task scenarios was given to each participant to complete. The authors began each testing session on a laptop with the research guides homepage already open. The main goal of each testing session was to observe if each participant could successfully complete task scenarios using the homepage and its links to external and internal web pages. The secondary goal was for participants to complete the task scenario with minimal attempts and limited clicks. When a task was completed, the facilitator observed how many attempts were made and if the participant was successful or unsuccessful. A successful attempt was documented if the participant completed the task using the information on the homepage and its links. A successful attempt was also documented if the user did not complete a task but was able to use the page to demonstrate adequate information-seeking behavior (e.g., using Chat or the Ask a Librarian box to seek answers). If a participant did not complete a task, while also not utilizing the homepage to seek answers, then the task was marked as unsuccessful. RESULTS Out of the 25 total task scenarios conducted during usability testing, 20 were successfully completed and 5 failed (see table 2). A quick summary of each participant’s experiences with completing the task scenarios is provided below. More detailed tables and notes for each participant are included in appendix A. Table 2. Task scenarios completed or failed by participant Task scenario 1 Task scenario 2 Task scenario 3 Task scenario 4 Task scenario 5 Participant 1 Completed Completed Completed Failed Completed Participant 2 Completed Completed Completed Completed Completed Participant 3 Completed Completed Completed Completed Completed Participant 4 Completed Completed Failed Completed Completed Participant 5 Failed Failed Completed Failed Completed INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2023 REDESIGNING RESEARCH GUIDES 11 MCCLURE, HESS, AND MARSICANO Participant #1 successfully completed four out of five task scenarios. They used the topic-based navigation to complete tasks 1, 2, and 5, while they initially failed task scenario 3 using the topic - based navigation before completing it using the search bar. They failed task scenario 4, failing to understand the “access a resource” language in the task scenario and stating that they would email someone for help. Participant #2 completed all five of the task scenarios. They used topic-based navigation to complete tasks 1 and 2. For task scenario 3, the participant used the search bar. Though there was again some confusion over the “access a resource” language used in task 4, the participant stated they would use chat or contact using the information in the Ask a Librarian box to complete the task. To complete task 5, the participant scanned the homepage and found the library help videos link. Participant #3 successfully completed all five task scenarios but required multiple attempts to complete tasks 3 and 4. Ultimately tasks 1, 2, 3, and 5 were completed using topic-based navigation. When unsuccessfully clicking various links in an attempt to complete task 4, the participant stated they would contact using the information in the Ask a Librarian box. Participant #4 successfully completed four out of five task scenarios. Tasks 1, 2, and 5 were completed using the topic-based navigation, while task scenario 4 was completed by scrolling to the Ask a Librarian section of the homepage. The participant tried to use the topic-based navigation for task 3, but they found a guide on contemporary, not historical, newspapers and failed the task. Participant #5 completed two out of the five task scenarios. Task 3 was completed using the search bar and finding an FAQ on historical newspapers, and task 5 was completed using the topic-based navigation. For task scenario 1, the participant left the research guides homepage using the logo in the banner at the top of the page, which took them to the University Libraries website. They had to be returned to the guides homepage for the following tasks. The participant tried to use the topic-based navigation three times to complete task 2 before giving up, and they also tried topic-based navigation for task scenario 4 before giving up. DISCUSSION Out of the 20 successfully completed task scenarios during the usability testing, 15 task scenarios were completed by users browsing the research guides homepage and making use of the headings and layout. These results demonstrate that the topic-based navigation design separating each section by Research Help, Subject Guides, Topic Guides, and Course Guides proved effective. Users were regularly able to intuit the correct area to look under with minimal errors. Scrolling was limited and clicks were few. The search bar was used less frequently, to the surprise of the RGRT. Only three task scenarios were successfully completed by using the search. Participants often clicked on random pages instead of using the search bar, and even when the search bar was used, it was not always immediately identified by the users. The authors noted that users would skip directly to the four category headings. The success of the Ask a Librarian section was mixed. Three of the five users successfully completed task scenario 4. Part of this was that some users were not familiar with the phrase “report access issues,” though once one of the authors explained further, many of them were still INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2023 REDESIGNING RESEARCH GUIDES 12 MCCLURE, HESS, AND MARSICANO unsure of who to ask or where to report issues. Additionally, only two out of the five participants asked for help when they could not solve other task scenarios. The remaining three gave up or moved onto the next question regardless of the availability of Chat or the Ask a Librarian box. Limitations Four out of five participants were undergraduate students. An ideal study would have had participants that included an additional graduate student and perhaps a faculty/staff member from the University of Memphis. An additional limitation was that the disability status of the participants was unknown, so the accessibility of the design was not formally tested beyond running the design through a web accessibility checker. Lastly, the original design was not tested for user errors and assumptions were made based on the RGRT’s observations and usage statistics in Springshare. Edits were made after usability testing occurred; therefore, improvements in usability between the original homepage design and the revision cannot be shown directly. POST-TEST REVISIONS Since most of the task scenarios were completed with successful results, the new design for the research guides homepage proved usable overall. Still, when the authors presented the findings to the rest of the RGRT at one of their regular meetings, the group discussed ideas for addressing issues identified in the testing. As a result, some minor changes were made: • The four categories (Research Help, Topic Guides, Course Guides, and Subject Guides) were moved to the top of the page with the search bar below it. • A graphic was created to draw attention to the search bar. • The Music, Health Sciences, and Lambuth Campus guides were made more visible by giving each link a University of Memphis brand-approved blue background with white lettering. The final product after usability testing was completed is shown in figure 8. The minor changes were not subject to usability testing at the time to test whether they improved the usability of the homepage. Further usability testing on the homepage is in the planning stages as the authors write this article. INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2023 REDESIGNING RESEARCH GUIDES 13 MCCLURE, HESS, AND MARSICANO Figure 8. Post-testing redesigned research guides homepage. CONCLUSION These results report one of the three usability testing results conducted on the research guides homepage. The RGRT elected to do three small tests with five users each. These tests were conducted in the same manner as the first, with the authors asking users to complete tasks in a Starbucks on a laptop. The only difference was the task scenarios. The RGRT designed new task scenarios for each test to reflect changes made to the homepage based on previous user experiences. For example, there were questions designed to test the relocation of the search bar and the Branch Library (Music, Lambuth, and Health Sciences) headings. A list of task scenarios from the other two tests are included in appendix B and appendix C. This study proves it is imperative that library web page designers familiarize themselves with the users’ information-seeking behavior in their community to design pages to be intuitive and easy to use. While perhaps the most valuable data gleaned from this study was ensuring an effective library web page design, the RGRT also learned that streamlining a workflow and making INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2023 REDESIGNING RESEARCH GUIDES 14 MCCLURE, HESS, AND MARSICANO maintenance of research guides a priority is essential for the success of enhancing an academic library’s online presence. Furthermore, it is a goal of the RGRT to conduct usability testing of its websites on a quarterly basis, not just on the homepage, but many of its subpages and the design of the guides themselves. The value of usability testing to the University Library is exemplified in this study. Now, anytime a crucial change is made to research guides, the RGRT is equipped to conduct usability testing to verify modifications and introduce new designs. INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2023 REDESIGNING RESEARCH GUIDES 15 MCCLURE, HESS, AND MARSICANO APPENDIX A. SUMMARY OF PARTIPANT TESTS Participant #1 (undergraduate student) Task scenario 1: You need to find information on MLA citation. Where would you go? Attempt 1: Citation Resources Guide → MLA Subsection. Completed. The participant quickly found it without hesitation. Participant stated when talking aloud that they did not understand what “MLA” or “citation” meant. But the participant was able to find the information by scanning for the words provided. Task scenario 2: If you were in the class SOCI 4420: Racial Inequality, where would you find information on that course? Attempt 1: All Course Guides → SOCI 4420: Racial Inequality Guide. Completed. The participant quickly found it without hesitation. Task scenario 3: Where would you go to find historical newspaper articles to use in a research paper? Attempt 1: Open Educational Resources Guide. Failed. Attempt 2: Used the search bar → Historical Newspapers Guide. Completed. The participant paused to work through this task. The participant did not notice the search bar during the first attempt. Task scenario 4: If you were struggling to access a resource for the library, where would you go? Attempt 1: Participant did not know where to click. Failed. Participant did not appear to understand phrase “access a resource.” Participant stated, “I don’t know … I would just email someone.” Task scenario 5: If you needed to find articles to use in your research paper, where would you go to find a video explaining how to do that? Attempt 1: Library Help Videos → Articles Subsection. Completed. The participant found the link to report an access issue while performing this task. Participant #2 (undergraduate student) Task scenario 1: You need to find information on MLA citation. Where would you go? Attempt 1: Citation Resources → MLA Subsection. Completed. The participant quickly found it without hesitation. Task scenario 2: If you were in the class SOCI 4420: Racial Inequality, where would you find information on that course? Attempt 1: All Course Guides → SOCI 4420: Racial Inequality Guide. Completed.  The participant quickly found it without hesitation. Task scenario 3: Where would you go to find historical newspaper articles to use in a research paper? Attempt 1: Used the search bar → Historical Newspapers Guide. Completed. The participant took more time with this task. Ultimately, they relied on the search bar to complete the task. Task scenario 4: If you were struggling to access a resource for the library, where would you go? Attempt 1: Chat and Ask a Librarian. Completed. The participant gave up and said they would rely on Chat and the Ask a Librarian function on the web page to report an access issue. This attempt was documented as successful since they used the page to exhibit sufficient information-seeking behavior. Task scenario 5: If you needed to find articles to use in your research paper, where would you go to find a video explaining how to do that? Attempt 1: Library Help Videos → Articles Subsection. Completed. The participant quickly found it without hesitation. INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2023 REDESIGNING RESEARCH GUIDES 16 MCCLURE, HESS, AND MARSICANO Participant #3 (graduate student and staff member) Task scenario 1: You need to find information on MLA citation. Where would you go? Attempt 1: Citation Resources → MLA Subsection. Completed. The participant quickly found it without hesitation. Task scenario 2: If you were in the class SOCI 4420: Racial Inequality, where would you find information on that course? Attempt 1: All Course Guides → SOCI 4420: Racial Inequality Guide. Completed.   The participant quickly found it without hesitation. Task scenario 3: Where would you go to find historical newspaper articles to use in a research paper? Attempt 1: Library Help Videos. Failed. Attempt 2: All Research Help Guides → Historical Newspapers Guide. Completed. When not finding what they were looking for, they switched tactics and selected the All Research Help Guides link. From there, they found the Historical Newspapers guide from an alphabetical list. The only user who completed this task using the navigation menu headings on the Homepage. Task scenario 4: If you were struggling to access a resource for the library, where would you go? Attempt 1: Could not find it by clicking various links. Failed. Attempt 2: Ask a Librarian. Completed. The student clicked on various links on the Homepage looking for the phrase “access a resource” (it was impossible for the note-taker to keep up with the various links the participant tried). Eventually the student stated they would rely on the Ask a Librarian box Task scenario 5: If you needed to find articles to use in your research paper, where would you go to find a video explaining how to do that? Attempt 1: Online Tutorials and Help Guides → Library Help Videos → Articles Subsection. Completed. “I would use the Libraries search bar to look for articles. I wouldn’t need a video.” Participant #4 (undergraduate student) Task scenario 1: You need to find information on MLA citation. Where would you go? Attempt 1: Citation Resources → MLA Subsection. Completed. The participant quickly found it without hesitation. Task scenario 2: If you were in the class SOCI 4420: Racial Inequality, where would you find information on that course? Attempt 1: All Course Guides → SOCI 4420: Racial Inequality Guide. Completed. The participant quickly found it without hesitation. Task scenario 3: Where would you go to find historical newspaper articles to use in a research paper? Attempt 1: All Topic Guides → News Literacy & News Sources Guide. Failed. Task scenario 4: If you were struggling to access a resource for the library, where would you go? Attempt 1: Scrolled to the Ask a Librarian box → Report Access Issue. Completed. Task scenario 5: If you needed to find articles to use in your research paper, where would you go to find a video explaining how to do that? Attempt 1: Library Help Videos → Journals subsection. Completed. INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2023 REDESIGNING RESEARCH GUIDES 17 MCCLURE, HESS, AND MARSICANO Participant #5 (undergraduate student) Task scenario 1: You need to find information on MLA citation. Where would you go? Attempt 1: University Libraries Homepage → Searched Kinesiology → Selected an eBook on Kinesiology. Failed. Immediately left the Research Guides homepage by selecting banner at the top. The participant did not understand the question. Task scenario 2: If you were in the class SOCI 4420: Racial Inequality, where would you find information on that course? Attempt 1: Library Help Videos. Failed. Attempt 2: Writing Help Guide. Failed. Attempt 3: Humanities Guide → Courses and Topics subsection → Books and eBooks. Failed. The participant gave up and wanted to move to the next question. Task scenario 3: Where would you go to find historical newspaper articles to use in a research paper? Attempt 1: Used the search bar → FAQ: Can I access historical issues of The New York Times? → ProQuest Historical Newspapers: The New York Times with Index. Completed. The participant did not notice the links to the Historical Newspapers Guide once using the search bar. Task scenario 4: If you were struggling to access a resource for the library, where would you go? Attempt 1: Open Educational Resources Guide. Failed. The participant gave up quickly and appeared exasperated. Task scenario 5: If you needed to find articles to use in your research paper, where would you go to find a video explaining how to do that? Attempt 1: Library Help Videos → Articles Subsection. Completed. INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2023 REDESIGNING RESEARCH GUIDES 18 MCCLURE, HESS, AND MARSICANO APPENDIX B. TASK SCENARIOS FOR SECOND USABILITY TEST Task #1 You are enrolled in a course where you are tasked with writing an informative paper on the musical instrument of your choice; how would you use this page to locate that information? Task #2 You have heard about a service where you can borrow books and articles from other libraries. You remember it’s called Interlibrary Loan. How would you use this page to use that service? Task #3 Where would you go to find historical newspapers? Task #4 Where would you go to find support for LGBTQIA+ students? Task #5 In which category (Research Help, Subject Guides, Topic Guides, Course Guides) would you look for: A) a guide to the natural and physical sciences? (subject) B) a guide on news literacy and news sources? (topic) INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2023 REDESIGNING RESEARCH GUIDES 19 MCCLURE, HESS, AND MARSICANO APPENDIX C. TASK SCENARIOS FOR THIRD USABILITY TEST Task #1 Where would you look to find information on the history of the library’s Lambuth branch? Task #2 What would you search through information about the Health Sciences Library? Task #3 How would you use this page to find primary sources? Task #4 You are in a class where the instructors ask you to cite in APA format. You are unfamiliar with this citation style and seek assistance from the library. How would you use this page to find information on APA citation? Task #5 You are in a criminology course and need to find crime data to include in a project. How would you use this page to find that information? ENDNOTES 1 Nigel Ford, “What Is Information Behaviour and Why Do We Need to Know About It?,” in Introduction to Information Behaviour (London: Facet Publishing, 2015), 14. 2 Sarah Thorngate and Allison Hoden, “Exploratory Usability Testing of User Interface Options in LibGuides 2,” College & Research Libraries 78, no. 6 (2017): 846, https://doi.org/10.5860/crl.78.6.844. 3 Peggy C. Holzweiss, Barbara Polnick, and Fred C. Lunenberg, “Online in Half the Time: A Case Study with Online Compressed Courses,” Innovative Higher Education, 44 no. 4 (2019): 299– 315, https://doi.org/10.1007/s10755-019-09476-8; Andrew J. Magda, David Capranos, and Carol B. Aslanian, Online College Students 2020: Comprehensive Data on Demands and Preferences (Louisville, KY: Wiley Education), https://universityservices.wiley.com/wp- content/uploads/2020/06/OCS2020Report-ONLINE-FINAL.pdf; Julia E. Seaman, I. Elaine Allen, and Jess Seaman, Grade Increase: Tracking Distance Education in the United States (Babson Park, MA: Babson Survey Research Group, 2018), https://eric.ed.gov/?id=ED580852. 4 Primary Research Group, The Survey of Library Services for MOOCS: Blended and Distance Learning Programs, 2016 ed. (New York: Primary Research Group Inc, 2015), 24, https://digital.auraria.edu/files/pdf?fileid=149c1b4a-a35d-48f1-81d8-5f47bb757ecf 5 Ruth Sara Connell, Lisa C. Wallis, and David Comeaux, “The Impact of COVID-19 on the Use of Academic Library Resources,” Information Technology & Libraries 40, no. 2 (2021): 8, https://doi.org/10.6017/ital.v40i2.12629. 6 Oliver Schulz, “Library Support for Distance Learning at Colorado Christian University,” Theological Librarianship 13, no. 2 (2020): 20–22, https://doi.org/10.31046/tl.v13i2.1938. 7 Jimmy Ghaphery and Erin White, “Library Use of Web-Based Research Guides,” Information Technology & Libraries 31, no. 1 (2012): 23, https://doi.org/10.6017/ital.v31i1.1830. 8 Amy E. G. Barker and Ashley T. Hoffman, “Student-Centered Design: Creating LibGuides Students Can Actually Use,” College and Research Libraries 82, no. 4 (2021): 75–91, https://doi.org/10.5860/crl.82.1.75; Danielle A. Becker and Lauren Yannotta, “Modeling a Library Website Redesign Process: Developing a User-Centered Website through Usability https://doi.org/10.5860/crl.78.6.844 https://doi.org/10.1007/s10755-019-09476-8 https://universityservices.wiley.com/wp-content/uploads/2020/06/OCS2020Report-ONLINE-FINAL.pdf https://universityservices.wiley.com/wp-content/uploads/2020/06/OCS2020Report-ONLINE-FINAL.pdf https://eric.ed.gov/?id=ED580852 https://digital.auraria.edu/files/pdf?fileid=149c1b4a-a35d-48f1-81d8-5f47bb757ecf https://doi.org/10.6017/ital.v40i2.12629 https://doi.org/10.31046/tl.v13i2.1938 https://doi.org/10.6017/ital.v31i1.1830 https://doi.org/10.5860/crl.82.1.75 INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2023 REDESIGNING RESEARCH GUIDES 20 MCCLURE, HESS, AND MARSICANO Testing,” Information Technology & Libraries 32, no. 1 (2013): 6–22, https://doi.org/10.6017/ital.v32i1.2311; Suzanna Conrad and Christy Sevens, “Am I on the Library Website?: A LibGuides Usability Study,” Information Technology & Libraries 38, no. 3 (2019): 49–81, https://doi.org/10.6017/ital.v38i3.10977; Amanda Donovan, “Building the Website of the Future In-House,” Computers in Libraries 42, no. 2 (2022): 4–8, https://www.infotoday.com/cilmag/mar22/index.shtml. 9 Isabel Vargas Ochoa, “Navigation Design and Library Terminology: Findings from a User- Centered Usability Study on a Library Website,” Information Technology & Libraries 39, no. 4 (2020): 12, https://doi.org/10.6017/ital.v39i4.12123. 10 Mark Aaron Polger, “Student Preferences in Library Website Vocabulary,” Library Philosophy & Practice (2011): 1, https://digitalcommons.unl.edu/libphilprac/618/. 11 Jon C. Giullian and Ernest A. Zitser, “Beyond LibGuides: The Past, Present, and Future of Online Research Guides,” Slavic and East European Information Resources 16, no. 4 (2015): 170–80, https://doi.org/10.1080/15228886.2015.1094718. 12 Ashley Lierman, Bethany Scott, Mea Warren, and Cherie Turner, “Testing for Transition: Evaluating the Usability of Research Guides around a Platform Migration,” Information Technology and Libraries 38, no. 4 (2019): 83, https://doi.org/10.6017/ital.v38i4.11169. 13 Yoo Young Lee and M. Sara Lowe, “Building Positive Learning Experiences through Pedagogical Research Guide Design,” Journal of Web Librarianship 12, no. 4 (2018): 205–31, https://doi.org/10.1080/19322909.2018.1499453. 14 Kimberly L. O’Neill and Brooke A. Guilfoyle, “Sign, Sign, Everywhere a Sign: What Does ‘Reference’ Mean to Academic Library Users?,” The Journal of Academic Librarianship 41, no. 4 (2015): 386–93, https://doi.org/10.1016/j.acalib.2015.05.007. 15 Aaron Bowen, Jake Ellis, and Barbara Chaparro, “Long Nav or Short Nav?: Student Responses to Two Different Navigational Interface Designs in LibGuides Version 2,” The Journal of Academic Librarianship 44, no. 3 (2018): 391–403, https://doi.org/10.1016/j.acalib.2018.03.002. 16 Jakob Nielsen, “Why You Only Need to Test with 5 Users,” accessed May 22, 2023, https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/ 17 Simone Borsci et al., “Reviewing and Extending the Five-User Assumption: A Grounded Procedure for Interaction Evaluation,” ACM Transactions on Computer-Human Interaction (TOCHI) 20, no. 5 (2013): 11–18, https://doi.org/10.1145/2506210. 18 Marieke McCloskey, “Turn User Goals into Task Scenarios for Usability Testing,” accessed May 22, 2023, https://www.nngroup.com/articles/task-scenarios-usability-testing/. 19 Olga Torres-Hostench, Joss Moorkens, Sharon O’Brien, and Joris Vreeke, “Testing Interaction with a Mobile MT Post-Editing App,” Translation & Interpreting 9, no. 2 (2017): 138–50, https://doi.org/10.12807/ti.109202.2017.a09; Jure Trilar, Tjaša Sobočan, and Emilija Stojmenova Duh, “Family-Center Design: Interactive Performance Testing and User Interface Evaluation of the Slovenian eDavki Public Tax Portal,” Sensors 21, no. 15 (2021): 5161, https://doi.org/10.3390/s21155161; Scott Uhl, “Applying User-Centered Design to Discovery https://doi.org/10.6017/ital.v32i1.2311 https://doi.org/10.6017/ital.v38i3.10977 https://www.infotoday.com/cilmag/mar22/index.shtml https://doi.org/10.6017/ital.v39i4.12123 https://digitalcommons.unl.edu/libphilprac/618/ https://doi.org/10.1080/15228886.2015.1094718 https://doi.org/10.6017/ital.v38i4.11169 https://doi.org/10.1080/19322909.2018.1499453 https://doi.org/10.1016/j.acalib.2015.05.007 https://doi.org/10.1016/j.acalib.2018.03.002 https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/ https://doi.org/10.1145/2506210 https://www.nngroup.com/articles/task-scenarios-usability-testing/ https://doi.org/10.12807/ti.109202.2017.a09 https://doi.org/10.3390/s21155161 INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2023 REDESIGNING RESEARCH GUIDES 21 MCCLURE, HESS, AND MARSICANO Layer Evaluation in the Law Library,” Legal Reference Services Quarterly 38, no. 1/2 (2019): 30–63, https://doi.org/10.1080/0270319X.2019.1614373. 20 Jeremiah Paschke-Wood, Ellen Dubinsky, and Leslie Sult, “Creating a Student-Centered Alternative to Research Guides: Developing the Infrastructure to Support Novice Learners,” In the Library with the Lead Pipe (October 2020), https://www.inthelibrarywiththeleadpipe.org/2020/student-centered-alternative-research- guides/. 21 University Libraries, “Research: By Course, Subject, or Topic,” accessed May 22, 2023, https://libguides.library.arizona.edu/library-guides. 22 University Libraries, “ENGL 1020: English Composition 1020,” accessed May 22, 2023, https://libguides.memphis.edu/engl1020. 23 Jakob Nielsen, “Thinking Aloud: The #1 Usability Tool,” accessed May 22, 2023, https://www.nngroup.com/articles/thinking-aloud-the-1-usability-tool/. https://doi.org/10.1080/0270319X.2019.1614373 https://www.inthelibrarywiththeleadpipe.org/2020/student-centered-alternative-research-guides/ https://www.inthelibrarywiththeleadpipe.org/2020/student-centered-alternative-research-guides/ https://libguides.library.arizona.edu/library-guides https://libguides.memphis.edu/engl1020 https://www.nngroup.com/articles/thinking-aloud-the-1-usability-tool/ Abstract Introduction Literature Review Redesign Process Usability Testing Results Discussion Limitations Post-Test Revisions Conclusion Appendix A. Summary of Partipant Tests Participant #1 (undergraduate student) Participant #2 (undergraduate student) Participant #3 (graduate student and staff member) Participant #4 (undergraduate student) Participant #5 (undergraduate student) Appendix B. Task Scenarios for Second Usability Test Appendix C. Task Scenarios for Third Usability Test Endnotes