ACRL News Issue (B) of College & Research Libraries 590 / C&RL News ■ September 2002 C o l l e g e & R e s e a r c h L i b r a r i e s n e w s How useful is your homepage? A quick and practical approach to evaluating a library’s Web site by M ignon Adams and Richard M. Dougherty A library’s Web site has becom e the tool most used by students and other informa- tion seekers to unearth the information riches o f the Internet. As a result, the Web site has become one o f the library’s most important resources, serving both as a source o f informa­ tion about the library and its holdings and as a gateway to the Web. Because o f its importance, we need to know more about the effectiveness o f library Web sites, how they are used, what features our users like, and what is confusing to them. This article describes how the library at the Univer­ sity o f the Sciences at Philadelphia (USP) tack­ led these and other issues. The information gained is now being used to revamp the library’s Web site. The history of a Web site When the USP library’s campus Web site was set up in 1995, the library’s homepage de­ scribed the library and had links to hours, staff, explanations o f services, and Web-accessible databases— at that time, maybe two or three. From there, the homepage steadily grew. In the ensuing years, databases were added to the site until there were 13 pages o f them, along with collections o f full-text, and explanations o f what was there. The library’s W eb master intended to do something different with the pages, but there always seemed to be more pressing items on her to-do list. When she was promoted to a campuswide position, it quickly became evident that someone else had to main­ tain the library’s site. No one else on our small staff expressed interest (in fact, they cringed at the thought), so the task fell to the library director, Mignon Adams. Since the campus Web site was also undergoing a redesign, redo­ ing the library site seemed appropriate. After attending several sessions on how to make Web sites more usable, and looking at the Web sites o f other libraries, library staff realized that our database page was too long and cumbersome; students often did not know what “full-text” meant, let alone “bibliographic database”; and w e had too many icons. Last year the library’s Web site received 165,105 hits, second only to the “prospective student” sections o f the campus Web site. But w e had no idea who was visiting the site, for what reason, and whether they found what they wanted. Web usability studies indicate that one way to find out is to observe individual users as they looked for specific kinds o f in­ formation, but this sounded like a labor-inten­ sive activity that w ould have to wait until a slow period to implement. Richard Dougherty had given two workshops in the Philadelphia area exploring a technique called “RADAR” (Recognizing Actual Desires And Requirements), a facilitated process designed to gain input quickly from users and staff. Dougherty and Adams had been working together on another project, and she asked him if his methods could be applied to a Web site analysis. A b o ut th e auth ors I M ignon Adams is the director o f libraries a t the University o f the Sciences in Philadelphia, e-mail: m.adams@usip. edu; Richard Dougherty, retired director o f Libraries a t the University o f Michigan, is n o w president o f Dougherty and Associates, e-mail: rmdoughe@umich.edu mailto:rmdoughe@umich.edu C&RL News β S e p te m b e r 2002 / 591 D e scrip tio n of th e p ro cess RADAR is a tool specially designed to help a library’s staff stay in touch with its users. The objectives o f the RADAR process are twofold: to generate planning inform ation about the current and changing needs o f library users and to identify constructive actions librarians can take to respond to identified users’ needs and desires. The underlying premise o f RADAR is that a library’s ow n staff knows user information needs and desires, and, because they work with users on a day-to-day basis, are able to place users’ needs into an overall service context. A unique feature o f RAD AR is its panels o f actual users. The panelists share h o w they obtain information, what sources they use, and, w hy they make the choices they do. Panelists also o ffe r suggestions to the library for im­ provem ent. W ith this inform ation staff can assess what they hear, ask for clarification, and, m ore im portantly, determ ine w hat actions should be taken in light o f what they have heard. The process is structured so that the staff/ user interactions take place in an atmosphere that maximizes the willingness o f staff to speak out without fear o f contradiction. W hy not fo cu s gro u ps or a s u rv e y ? Well-designed surveys can generate useful in­ formation, particularly w h en the library re­ quires quantitative data. D evelopin g a ques­ tionnaire, however, and constructing a reliable sample o f p e o p le w h o are actual users o f a W eb site are not easy tasks. Surveys take time, expertise, and money. C arefu lly se lec te d panels and w e ll-d e - signed questions with skillful facilitation can p ro vide invaluable insights as w ell. But fo ­ cus groups need people w h o are k now ledge­ able about the subject and w h o are opin ion leaders. T h e m ost often -h ea rd com p lain t about focus groups am ong librarians is the d iffic u lty o f fin d in g fa cu lty and students w h o are both k n ow ledgeable and w illin g to participate in the sessions. The use o f panels in RADAR also avoids on e o f the limitations o f library-oriented fo ­ cus groups: cost. In the corporate world, a fo ­ cus group w ill use well-trained, experienced facilitators and one-w ay mirrors with observ­ ers to identify topics that need to be probed. H o w often do w e use professional facilitators w h o haλ'e enough experience in library issues to recognize when probing follow-up questions need to be asked? This is important because a librarian, even as an observer, should not be present. T h e RAD AR approach avoids this problem o f interpretation because panels in­ teract directly with the staff w h o are given an opportunity to assess what they have heard and ask probing questions. How w e created our u ser pan els In the USP test, faculty and students w ho were thought to be actual users o f the library’s W eb site w ere contacted. The director invited fac­ ulty w h om she believed to be regular users o f the library and its resources. She asked student workers w h o had the time available, and o f­ fered to pay them their student w orker sala­ ries. Even w ith this inform al approach, the director had difficulty finding faculty and stu­ dents with time to participate. We assumed that the participants in our test panels were users o f the library’s W eb site, but as w e found out later, about half o f the panel­ ists somewhat apologetically admitted that they w ere not frequent users. This proved to be an unexpected bonus because the staff was able to find out why, and, o f course, these discus­ sions led to other revelations, a sampling o f w h ich are presented in the fo llo w in g para­ graphs. W hat w e heard from our custom ers The informal presentations and the follow-up interactions between the panelists and the staff w ere revealing and produced a wealth o f in­ formation. W e learned: • The library’s homepage was rarely used by faculty and students; some w ere unaware it was there. • Students admitted that they w ere often confused by the W eb site; they thought the database page was too complicated to figure out. • Instead o f using the library’s resources, students are more likely to use Yahoo or Google; they don ’t use the directories o f these search engines but type in their search request in natu­ ral language. • Jargon presents problems for some: what is a database; what do charged out, availabil­ ity, and browse mean? • Faculty are familiar with the databases that they most frequently use. H ow ever, they 592 / C&RL News ■ September 2002 didn’t try out new ones, they didn’t read de­ scriptions, and they were unaware o f some useful sources. • The layout o f the Web site was criticized for having too many words and a type-size that was too small. After the panelists departed, staff were asked: “What is the meaning o f what you have heard?” The staff comments quickly validated what the students had been saying about going di­ rectly to search engines and bypassing the li­ brary and its resources. The staff weren’t a bit surprised, and it took only a few minutes for staff to compose a telling message. Again, what follows is a sampling: • Students don’t know how to use the re­ sources the library is already providing. • There is really a mismatch between what w e are offering and what customers are using. • They aren’t asking us; they are going di­ rectly to the Web. • Ease o f getting information is more im­ portant than the quality o f information. • There are lots of terms that aren’t understood. • They are asking us to make the “stuff” simple. The staff responds The group generated a long list o f sugges­ tions and recommendations. Some o f the ideas dealt w ith organization o f the Web site, the presentation o f information, and additional training workshops. Suggestions that received greatest priority through a vot­ ing process are listed here: • Make the Web site more basic; the main page must be more direct— shorter and to the point. • D on’t overwhelm students with termi­ nology and jargon. • Categorize resources by academic major: pharmacy, medicine, history, etc. • Design the Web site for different audi­ ences: only a first-time user needs a descrip­ tion o f the library, its hours, or location. • Give quick ways to get into the sources: the “top three” resources for psychology or bi­ ology, for example. • Provide basic instructions. Lessons learned What did w e learn from the RADAR experi­ ence that warrants telling others? First, pro­ viding an opportunity for staff to listen, com­ ment, and offer recommendations has greatly enriched everyone’s understanding of what was right and wrong with the design of the library's existing Web site. It also provided the Web master with valuable information about spe­ cific things that needed to be done and about how to achieve them. Second, the staff now sees that everyone has a stake in making the Web site more responsive to the needs of users. We were con­ vinced that the staff would be frank because they didn’t feel that they had a stake in the design o f the current Web site. We were right about that, but didn’t fully appreciate how quickly some staff began talking in terms of “our tool” and what “w e” need to do in the library to make this important tool more help­ ful and valuable to our users. Third, simplicity is a virtue— less is more … less text … less items on the page. We really have to provide basic information for underclass students. The Web site at present is or appears to be complex for them. Finally, our original design didn’t pay enough attention to the desires, needs, and preferences o f our audiences. The fact that the homepage is so often not the entry point underscores the point that what w e think of as our homepage is not relevant to our users’ needs. Next steps A staff task force has been appointed to help with the design of the revised Web page. Staff have already said that w e need to pay more attention to what other academic libraries have done. Let’s take advantage o f what others have done well. Future designs w ill strive to incorporate what w e heard the students and faculty telling us— keep it simple for the undergraduates. There w ill have to be more instructional workshops for students, even though staff have been providing instruction that should have addressed many o f the student and fac­ ulty questions and concerns. H ow w e can do this better is just one o f the challenges facing the library’s staff. A staff member reminded us that the li­ brary needs to do a better job o f marketing existing library services. She asked: “Why should they have to come to us? We need to reach out to them.” Hopefully taking action based on what w e learned represents a first step towards achieving this. ■ C&RL News ■ September 2002 / 593