Navigation Design and Library Terminology: Findings from a User-Centered Usability Study on a Library Website COMMUNICATION Navigation Design and Library Terminology Findings from a User-Centered Usability Study on a Library Website Isabel Vargas Ochoa INFORMATION TECHNOLOGY AND LIBRARIES | DECEMBER 2020 https://doi.org/10.6017/ital.v39i4.12123 Isabel Vargas Ochoa (ivargas2@csustan.edu) is Web Services Librarian, California State University, Stanislaus. © 2020. ABSTRACT The University Library at California State University, Stanislaus is not only undergoing a library building renovation, but a website redesign as well. The library conducted a user-centered usability study to collect data in order to best lead the library website “renovation.” A prototype was created to assess an audience-based navigation design, homepage content framework, and heading terminology. The usability study consisted of 38 student participants. It was determined that a topic- based navigation design will be implemented instead of an audience-based navigation, a search-all search box will be integrated, and the headings and menu links will be modified to avoid ambiguous library terminology. Further research on different navigation and content designs, and usability design approaches, will be explored for future studies. INTRODUCTION The University Library at California State University, Stanislaus is currently undergoing a much anticipated and necessary redesign of the library website. Website redesigns are crucial and a part of website maintenance to acclimate with modern technology and meet accessibility standards. “If librarians are expected to be excellent communicators at the reference desk and in the classroom, then the library website should complement the work of a librarian.”1 In this case, a library website prototype was created, using a Springshare LLC product, LibGuides CMS, as the testing subject for our user-centered usability study. The usability study was completed with 38 student participants belonging to different academic years and areas of study. The library website prototype tested was designed using a user-based design framework and an audience-based navigation. This study found issues reported from users based on navigation design and ambiguous library terminology. An audience-based navigation was chosen in order to best organize and group the information and services offered to best make them accessible for users. However, an audience-based navigation will directly affect users and their search behaviors.2 The prototype, like the current library website, did not have a search-all search box during the study. A catalog search box was utilized to test whether or not the catalog was enough for student participants to find information. This also forced the participants to utilize the menu navigation. LITERATURE REVIEW The design and approach of usability studies, preference for types of search boxes, navigation design, and library terminology evolve over time in parallel with technology changes. Most recent usability studies use screen and audio recording tools as opposed to written observation notes. Participants in recent studies are also more adapted to learning how to navigate websites, as mailto:ivargas2@csustan.edu INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 NAVIGATION DESIGN AND LIBRARY TERMINOLOGY | OCHOA 2 opposed to participants in usability studies twenty years ago. Regardless, it’s crucial to compare the results from previous usability studies to analyze differences and similarities. Different types of usability studies include user-centered usability studies and heuristic usability studies. This study chose a user-centered approach because of the library's desire to collect data and feedback from student users. The way in which the usability study is presented is also detrimental to the approach. Website usability studies are meant to test the website, although participants may unconsciously believe they are being tested. In Tidal’s library website case study (2012) researchers assured the participants that “the web site was being tested and not the participants themselves.”3 This unconscious belief may also affect the data collected from the participants and “influence user behavior, including number of times students might attempt to find a resource or complete a given task.”4 The features tested were the navigation design and homepage elements. The navigation design in the prototype was developed to test an audience-based navigation design (see figure 1). An audience-based navigation design organizes the navigation content by audience type. 5 That is to say, the user will begin their search by identifying themselves first. Although this design can organize content in a more efficient manner, especially for organizations that have specific, known audiences, critics argue that this design forces users to identify themselves before searching for information, thus taking them out of their task mindset.6 For this usability study, I wanted to test this navigation design and compare the results to our current navigation design which is a topic- based navigation design. A topic-based navigation design is developed to present topics as navigation content.7 This design is our current library website navigation design (see figure 2) Figure 1. Screenshot of the audience-based navigation design developed for the library website prototype. Figure 2. Screenshot of the current content-based navigation design in the library website. INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 NAVIGATION DESIGN AND LIBRARY TERMINOLOGY | OCHOA 3 Designing the navigation and homepage also means choosing accessible terms that are relevant to all users. Unfortunately, over the course of many decades, library terminology has been a hindrance for student users. Terms such as “catalog,” “reference,” and “research guides” are still difficult for users to understand. As Conrad states (2019), “students are not predisposed to think of a ‘research guide’ as a useful tool to help them get started.”8 A research guide isn’t necessarily a self-explanatory term. In many ways, the phrase is ambiguous. Augustine’s case study in 2002 had similar difficulties. Students “lack of understanding and awareness of library resources impacted their ability more than the organization of the site did.”9 It’s unsettling to know that our own terminology has been deterring users from accessing library resources for decades. Librarians use library terminology to such an extent that it’s part of our everyday language, but what is common knowledge to us may be completely alien to our very own audience. Not only should libraries be aware of confusing library terms, but content should also not overwhelm the user with an abundance of information. Most students who visit the library are looking for something specific and easy to find. It’s important for librarians to condense their information on guides or website pages to not frustrate the user or make them search elsewhere, like Google. “Students scan. . . rather than [read] material.” 10 This is also something that has been noted from our Crazy Egg statistics. Heatmaps of our website’s pages prove that users are not scrolling to the bottom of the pages. This also applies to the use of large images, or unnecessary flashy or colorful content that covers most of the desktop or mobile screen. These images should be reduced in size so that users can find information swiftly. For this reason, any large design on the homepage should also be included in menu links, in case large flashy content is ignored.11 The search box is also another fundamental element I analyzed. In this case study, our search box was the catalog search box for Ex Libris Primo. If a page, particularly the homepage, has two search boxes—search-all and catalog search—the user can be confused. Search boxes are primarily placed at the center of the page. Depending on how these search boxes are labeled and identified, users may not know which one to use. Students approach library search boxes as if searching Google.12 In our case, neither the current website nor the prototype has a general search-all box. We have a catalog search box placed on the top center of the homepage for both sites. If we were to add a general search-all box, it would be placed away from the catalog search box and preferably in the header where it is visible in all pages. METHODOLOGY The usability study was conducted by the author, the web services librarian at California State University, Stanislaus, who also worked with a computer science instructor in order to recruit participants. Not only is the University Library redesigning its website, but the University Library building is also undergoing a physical renovation. Due to this project, the library has relocated to the library annex, a collection of modular buildings providing library services to the campus community. The usability study was conducted in a quiet study room in one of these modular sites. I reserved this study area and borrowed eight laptops for the sessions. The usability study employed two different methods to get students to participate. The first offered an extra credit incentive, which was offered when I collaborated with the computer science instructor. This instructor was teaching a course on human-centered design for websites. She offered her students an extra credit incentive, since several of her learning objectives centered on website design and usability studies. The second approach was an informal one. This approach INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 NAVIGATION DESIGN AND LIBRARY TERMINOLOGY | OCHOA 4 was promoted by scouting students who were already at the library annex during the usability study scheduled sessions. this enabled students to participate without having to sign-up or remember to participate. The students were recruited in-person during the usability session and through flyers posted in study rooms on the days of the study. An incentive of snacks for students to take home was also included. I created questions and seven tasks to be handed out to the participants during the study. The tasks were created to test the navigation design of the main menu and content on the homepage. I also added a task to test the research skills of the student. After these tasks, students were asked to rate the ease of access, answer questions about their experience navigating the prototype and to provide feedback. All students were given the same tasks, however if the student was taking the human-centered design course, they were also given specific web design questions for feedback (see Appendices A and B). The tasks were piloted before the study with three library student workers who provided feedback on how to better word the tasks for students. The following tasks are the final seven tasks used for the usability study: 1. Find research help on citing legal documents—a California statute— in APA style citation. 2. Find the library hours during spring break. 3. Find information on the library study spaces hours and location. 4. You’re a student at the Stan State campus and you need to request a book from Turlock to be sent to Stockton. Fill out the request-a-book form. 5. You are a graduate student and you need to submit your thesis online. Fill out the thesis submission form. 6. For your history class, you need to find information on the university’s history in the University Archives and Special Collections. Find information on the University Archives and Special Collections. 7. Find any article on salmon migration in Portland, Oregon. You need to print it, email it to yourself, and you also need the article cited. The usability study sessions took place from 11am to 2pm on February 10, 12, and 14, 2020. These days and times were chosen because the snack incentive would attract students during lunch hour and I wanted to accommodate the start and end times of the human-centered design course on Mondays, Wednesdays, and Fridays. The total time it took for students to complete the 7 tasks averaged 15 minutes. In total, there were 38 student participants. The student’s experience was recorded anonymously. I asked students to provide their academic year and major. Students ranged from freshman (5), sophomore (2), junior (12), senior (17), graduate (1), and unknown (1). Areas of study included computer science (16), criminal justice (2), business (2), psychology (3), communications (1), sociology (1), English (3), nursing (1), Spanish (1), biology (3), geology (1), history (2), math (1), gender studies (1), and undeclared (1). The subject tested was the library website prototype created and executed using a Springshare LLC Product, LibGuides CMS. The tools I used were eight laptops and a screen recording tool, Snagit. Snagit is a recording tool made accessible through a campus subscription. The laptops were borrowed from the library for the duration of the sessions. During the session, students navigated and completed the tasks on their own with no direct interference, including no direct observations. I planned to create a space where my presence didn’t directly influence or intimidate their experience with the website. My findings were based solely on their written responses and screen recordings. I also explained to the students that their screen recorded video INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 NAVIGATION DESIGN AND LIBRARY TERMINOLOGY | OCHOA 5 will not be linked to their identity, since they had to sign-in to the laptop using their campus student ID. I did, however, occasionally walk around the tables in the room in case a student was navigating the current website or using a separate site to complete the tasks. Once the students completed the tasks and answered the questions, I collected the handouts and the screen -capture videos by copying them to a flash drive. LIMITATIONS During the usability study session, there were two technical issues that hindered the initial process. On the first day, there were difficulties accessing the campus Wi-Fi in the room as well as difficulties accessing the Snagit video recording application. This limitation affected some of the students' experiences and feedback. These issues were resolved and not present on the second and third day of the study. RESULTS AND OBSERVATIONS The results and observations collected from this study mirror results from the studies conducted by Azadbakht and Swanson.13 I found that students searched the catalog search box for library collections, citations, and other library terms they didn’t understand, even though it was a catalog search box with the keywords “find articles, books, and other materials” labeled in the search bar. Another finding was that the navigation design can detrimentally affect a user's experience with the website. Mixed reviews were received from utilizing the audience-based navigation design. The study also found that students are adept at finding research materials. For example, most students knew how to search, find, print, email, and cite an article. Students in general are also familiar with book requests, ILL accounts, and filling out book request webforms. This indicates that, in terms of utilizing library services, students are well aware of how to find, request, and acquire resources, using the website on their own. What was most difficult for students was interpreting library terminology. This was explicitly shown in their attempts to complete tasks 1 and 6: finding how to cite a legal document in APA style and finding information on Special Collections and the University Archives. The following results and observations are divided into three categories: written responses, video recording observations, and data collected. Data was collected based on observations from the video recording and the written responses. Data was then input into eight separate charts. Written Responses Observations Comments from both non-human-centered website design students and human-centered design students included mixed reviews on the navigation layout, overall positive outlook on the page layout design, suggestions to add a search-all “search bar,” and frustrations with tasks 1 and 6. Video Recording Observations The Ex Libris Primo search box was constantly mistaken as a search-all search box. This occurred during students’ search for tasks 1 and 6: citation help and university archives, respectively. Students also used the research guides search box in LibGuides as a search-all search box. Students found the citation style guides easily because of this feature, however on the proposed new website, it was difficult to find citation help. Students were also using research guides to complete other tasks, such as task 6. A search bar for the entire website was continuously mentioned as a solution from student participants. INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 NAVIGATION DESIGN AND LIBRARY TERMINOLOGY | OCHOA 6 Tasks 2 and 3, regarding library hours and study spaces, were easily completed. Tasks 4 and 5 were also easily accessible. After completing task 4 (book request form) it was easier for participants to complete task 5 (thesis submission form) because both tasks required students to search the top main navigation menu. To complete task 4, several students immediately signed-in to their ILL account or login to Primo for CSU+, which was expected as signing-in to these accounts are alternate modes to request a book. An additional other observation, regarding task 4 is that the confusion revolving the library terms, “call number,” was solved by adding an image reference pointing to the call number in the catalog. The call number image reference was opened several times for assistance. Most students completed task 7 (find a research article) but not all students used the catalog search box on the homepage to complete it. Several students searched the top main navigation and clicked on the “Research Help” link. Others utilized research guides and the research guides search box on the homepage. A particular unique observation was made by some computer science students. Most computer science students were quicker to give up on a task as opposed to non-computer science students. Some computer science students did not scroll down when browsing pages. These students failed to complete several tasks because they didn’t scroll down the page after being on the page for less than ten seconds. DATA COLLECTED Figure 3. Ease of navigation (overall). INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 NAVIGATION DESIGN AND LIBRARY TERMINOLOGY | OCHOA 7 Figure 3 illustrates the ease of navigation rating overall from all student participants. Students were asked to rate the ease of access of the website (see Appendices A and B). Other than the keywords “ease of navigation (1 difficult; 10 easy)” students were given the freedom to define what “easy” and “difficult” meant to them individually. The mean for the ease of access rating for all student participants was 7.7. The lowest rating of ease of access was 3 and the highest rating of ease of access was 10. Figure 4. Ease of navigation (computer science major). Figure 4 illustrates the ease of access rating by the student participants based on whether the student was a computer science major or not a computer science major. The lowest ease of access ratings were from computer science majors. Overall non-computer science majors had higher ease of access ratings than computer science majors. INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 NAVIGATION DESIGN AND LIBRARY TERMINOLOGY | OCHOA 8 Figure 5. Ease of navigation (human-centered design). Figure 5 illustrates the ease of access rating by the student participants based on whether the student was taking the human-centered design course. The human-centered design students’ learning outcomes include website user-interface design and an assignment on how to create a usability study. Similar to patterns found in figure 2, human-centered design students had lower ease of access ratings. Figure 6. Tasks – Status of completion. INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 NAVIGATION DESIGN AND LIBRARY TERMINOLOGY | OCHOA 9 Figure 6 illustrates whether a task was completed or not. Completion of task was determined by analyzing whether or not the student not only found the page(s) that provided the solution to the task. It was determined that a student did not complete the task if the student was unable to find the page(s) that provided the solution to the task. “Not applicable” was determined if th e student did not use the website prototype (e.g., followed a link that led elsewhere or opted to use Google search instead). Most students completed tasks 2, 3, 4, 5, and 7. The task with most “did not complete” was task 1 , which 64 percent of student participants did not complete. Task 6 had neutral completion, 63 percent. 86 percent of students completed tasks 2 and 4, and 90 percent of students completed tasks 3, 5 and 7. It is evident that task 1 was a difficult task to complete, regardless of the stu dent’s area of study. Task 1 required students to find APA legal citation help. The terms “APA legal citation” confused users. Likewise, for task 6 (Special Collections), students did not understand what “collections” referred to or where to search them. Figure 7. Tasks – Number of clicks (complete). Figure 7 illustrates how many clicks it required students to complete the task. The clicks were separated into three categories: 1-2 clicks, 3-5 clicks, and more than 6 clicks. This figure only illustrates data collected from tasks that were completed. The number of clicks began at the website prototype’s homepage or from the main menu navigation found in the website prototype’s header, when it was evident that the student was starting a new task. Tasks 2 and 3 were completed in 1-2 clicks, whereas tasks 1, 4, 5, 6, and 7 required an average of 3-5 clicks. Because of experience helping students find articles at the librarian's research help desk, Task 7 (find research articles) was expected to require 6+ clicks. Task 1 may have a pattern of needing a high number of clicks to complete because it was a generally a difficult task to complete. INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 NAVIGATION DESIGN AND LIBRARY TERMINOLOGY | OCHOA 10 Figure 8. Tasks – Number of clicks (did not complete). Figure 8 illustrates how many clicks a student participant made before they decided to skip the task or if they believed they had completed the task. This figure only illustrates data from tasks that were not completed. The clicks were separated into three categories: 1-2 clicks, 3-5 clicks, and more than 6 clicks. The number of clicks began at the website prototypes homepage or from the main menu navigation found in the website prototype’s header, when it was evident that the student was starting a new task. Task 1 and 6 show the most patterns in this figure. Task 1 (citation help) shows that students generally skipped the task after more than 6 clicks. Task 6 (Special Collections) was generally skipped after 3-6+ clicks. Figure 9 illustrates the duration to complete each task. The duration was separated into three categories: 0-1 minutes, 1-3 minutes, or more than 3 minutes. This figure only illustrates data for tasks that were completed. The duration began when the student started a new task. This was determined when it was observed that the students started to use the main menu navigation, or if the student directed their screen back to the website prototype’s homepage. There are parallels between the number of clicks and duration of tasks. For tasks 2, 3, and 5, the duration to complete the task was less than 1 minute. Task 5 was a task similar to task 4 (both are forms, linked once on the website), but the duration for task 5 may have averaged lower than the duration of task 4, because task 5 was after task 4. Having completed a form before task 5 may have influenced the student’s behavior on searching for forms. Tasks 1, 6, and 7 averaged 1-3 minutes to complete. INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 NAVIGATION DESIGN AND LIBRARY TERMINOLOGY | OCHOA 11 Figure 9. Tasks – Question duration (complete) Figure 10. Tasks – Question duration (did not complete) INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 NAVIGATION DESIGN AND LIBRARY TERMINOLOGY | OCHOA 12 Figure 10 illustrates the duration of each task that wasn’t completed. The duration was separated into three categories: 0-1 minutes, 1-3 minutes, or more than 3 minutes. This figure only illustrates data for tasks that were not completed. The duration began when the student started a new task. This was determined when it was observed that the students started to use the main menu navigation, or if the student directed their screen back to the website prototype’s homepage. Similarly, to observations for figure 7, there are parallels between the number of clicks and duration of tasks. For task 1, the average time before students skipped the task varied, however most students who didn’t complete the task skipped it after more than 3 minutes of trying to complete it. For task 6, the average duration before skipping the task was 1-3 minutes. CONCLUSION AND RECOMMENDATIONS The purpose of this study was primarily designed to test the user-centered study approach and the navigational redesign of the library website. The results, however, provided the library with a variety of outcomes. Based on suggestions and comments on the website prototype navigation design, menus, and page content, there are several elements that will be integrated to help lead the redesign of the library’s website. Students found that the navigation design of the website was clear and simple, but also required a “getting used to.” Because of this, and due to navigation design literature, it is recommended to design a menu navigation that is a topic-based navigation as opposed to an audience-based navigation. Our findings also highlighted the effects of the use of library terms. To make menu links exceptionally user-friendly, it is recommended to utilize clear and common terminology. Student participants also voiced that a search-all search box for the website was necessary. This will enable users to access information efficiently. Library website developers should also map more than one link to a specific page, especially if the only link to the page is on an image or slideshow. The user-centered usability approach for this case study worked well in collaboration with campus faculty and as an informal recruitment. It provided relevant and much needed data and feedback for the University Library. In terms of future usability studies, a heuristic approach may be effective. A heuristic study approach will enable moderators to gather feedback and analysis from library web development experts.14 Moreover, the usability study could be conducted over a semester long time and include focus groups to acquire consistent feedback. 15 Overall, website usability studies are evolving and require constant improvements and research. INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 NAVIGATION DESIGN AND LIBRARY TERMINOLOGY | OCHOA 13 APPENDIX A Major: ___________ Year (freshman, sophomore, etc.): ______________ Link to site: URL - please do NOT use URL Please complete the following situations. For some of these, you don’t need to actually submit/send, but pretend as if you are. 1. Find research help on citing legal documents - a California statute - in APA style citation. 2. Find the library hours during spring break. 3. Find information on the library study spaces hours and location. 4. You’re a student at the Stan State campus and you need to request a book from Turlock to be sent to Stockton. Fill out the request-a-book form. 5.You are a graduate student and you need to submit your thesis online. Fill out the thesis submission form. 6. For your history class, you need to find information on the university’s history in the University Archives and Special Collections. Find information on the University Archives and Special Collections. 7. Find any article on salmon migration in Portland, Oregon. You need to print it, e-mail it yourself, and you also need the article cited. Complete the following questions. 1. Rate the ease of access of the website (1= really difficult to navigate, 10=eas y to navigate) 1 2 3 4 5 6 7 8 9 10 2. Did you ever feel frustrated or confused? If so, during what question? 3. Do you think the website provides enough information to answer the above questions? Why or why not? INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 NAVIGATION DESIGN AND LIBRARY TERMINOLOGY | OCHOA 14 APPENDIX B CS 3500 Major: ___________ Year (freshman, sophomore, etc.): ______________ Link to site: URL - please do NOT use URL Please complete the following situations. For some of these, you don’t need to actually submit/send, but pretend as if you are. 1. Find research help on citing legal documents - a California statute - in APA style citation. 2. Find the library hours during spring break. 3. Find information on the library study spaces hours and location. 4. You’re a student at the Stan State campus and you need to request a book from Turlock to be sent to Stockton. Fill out the request-a-book form. 5.You are a graduate student and you need to submit your thesis online. Fill out the thesis submission form. 6. For your history class, you need to find information on the university’s history in the University Archives and Special Collections. Find information on the University Archives and Special Collections. 7. Find any article on salmon migration in Portland, Oregon. You need to print it, e-mail it yourself, and you also need the article cited. Then, complete the following questions. 1. Rate the ease of access of the website (1= really difficult to navigate, 10=easy to navigate) 1 2 3 4 5 6 7 8 9 10 2. What did you think of the overall web design? 3. What would you change about the design? Please be specific. 4. What did you like about the design? Please be specific. INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 NAVIGATION DESIGN AND LIBRARY TERMINOLOGY | OCHOA 15 ENDNOTES 1 Mark Aaron Polger, “Student Preferences in Library Website Vocabulary,” Library Philosophy and Practice, no. 1 (June 2011): 81, https://digitalcommons.unl.edu/libphilprac/618/. 2 Jakob Nielsen, “Is Navigation Useful?,” NN/g Nielsen Norman Group, https://www.nngroup.com/articles/is-navigation-useful/. 3 Junior Tidal, “Creating a User-Centered Library Homepage: A Case Study,” OCLC Systems & Services: International Digital Library Perspectives 28, no. 2 (May 2012): 95, https://doi.org/10.1108/10650751211236631. 4 Suzanna Conrad and Christy Stevens, “‘Am I on the Library Website?’: A LibGuides Usability Study,” Information Technology and Libraries (Online) 38, no. 3 (September 2019): 73, https://doi.org/10.6017/ital.v38i3.10977. 5 Eric Rogers, “Designing a Web-Based Desktop That's Easy to Navigate,” Computers in Libraries 20, no. 4 (April 2000): 36, ProQuest. 6 Katie Sherwin, “Audience-Based Navigation: 5 Reasons to Avoid It,” NN/g Nielsen Norman Group, https://www.nngroup.com/articles/audience-based-navigation/. 7 Rogers, “Designing a Web-Based Desktop That's Easy to Navigate,” 36. 8 Conrad, “‘Am I on the Library Website?’: A LibGuides Usability Study,” 71. 9 Susan Augustine and Courtney Greene, “Discovering How Students Search a Library Web Site: A Usability Case Study,” College & Research Libraries 63, no. 4 (July 2002): 358, https://doi.org/10.5860/crl.63.4.354. 10 Conrad, “‘Am I on the Library Website?’: A LibGuides Usability Study,” 70. 11 Kate A. Pittsley and Sara Memmott, “Improving Independent Student Navigation of Complex Educational Web Sites: An Analysis of Two Navigation Design Changes in LibGuides,” Information Technology and Libraries 31, no. 3 (September 2012): 54, https://doi.org/10.6017/ital.v31i3.1880. 12 Elena Azadbakht, John Blair, and Lisa Jones, “Everyone's Invited: A Website Usability Study Involving Multiple Library Stakeholders,” Information Technology and Libraries 36, no. 4 (December 2017): 43, https://doi.org/10.6017/ital.v36i4.9959. 13 Azadbakht, “Everyone's Invited,” 43; Troy A. Swanson and Jeremy Green, “Why We Are Not Google: Lessons from a Library Web Site Usability Study,” The Journal of Academic Librarianship 37, no. 3 (February 2011): 226, https://doi.org/10.1016/j.acalib.2011.02.014. 14 Laura Manzari and Jeremiah Trinidad-Christensen, “User-Centered Design of a Web Site for Library and Information Science Students: Heuristic Evaluation and Usability Testing ,” Information Technology and Libraries 25, no. 3 (September 2006): 164, https://doi.org/10.6017/ital.v25i3.3348. 15 Tidal, “Creating a User-Centered Library Homepage: a Case Study,” 97. https://digitalcommons.unl.edu/libphilprac/618/ https://www.nngroup.com/articles/is-navigation-useful/ https://doi.org/10.1108/10650751211236631 https://doi.org/10.6017/ital.v38i3.10977 https://www.nngroup.com/articles/audience-based-navigation/ https://doi.org/10.5860/crl.63.4.354 https://doi.org/10.6017/ital.v31i3.1880 https://doi.org/10.6017/ital.v36i4.9959 https://doi.org/10.1016/j.acalib.2011.02.014 https://doi.org/10.6017/ital.v25i3.3348 ABSTRACT Introduction Literature Review Methodology Limitations Results and Observations Written Responses Observations Video Recording Observations Data Collected Conclusion AND Recommendations Appendix A Appendix B ENDNOTES