Evidence Based Library and Information Practice
Evidence Based Library and Information Practice 2009, 4:4
Evidence Based Library and Information Practice
Article
Testing an Academic Library Website for Usability with Faculty and Graduate Students
Judith Z. Emde
E‐Resources Librarian
University of Kansas
Kansas, USA
Email: jemde@ku.edu
Sara E. Morris
American History Librarian
University of Kansas
Kansas, USA
Email: semorris@ku.edu
Monica Claassen‐Wilson
Information Specialist
University of Kansas
Kansas, USA
Email: monicacw@ku.edu
Received: 17 July 2009 Accepted: 10 November 2009
© 2009 Emde, Morris, and Claassen‐Wilson. This is an Open Access article distributed under the terms of the
Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted
use, distribution, and reproduction in any medium, provided the original work is properly cited.
Abstract
Objectives – This usability study was developed to observe faculty and graduate
students’ reactions to a recent redesign of the University of Kansas (KU) Libraries’
website. The redesign included new navigational features, introduction of a federated
search tool, a quick search box on the front page, and research subject pages. The
study also provided the opportunity to observe the practices of faculty and graduate
students in locating and retrieving information on the Libraries’ website.
Methods – Ten participants (five faculty and five graduate students) representing
diverse disciplines were solicited for the study. Participants were required to access
the Libraries’ website to answer a series of questions regarding new and updated
features of the website. Observational analysis using Morae™ software was
24
mailto:jemde@ku.edu
mailto:semorris@ku.edu
mailto:monicacw@ku.edu
http://creativecommons.org/licenses/by/2.0
Evidence Based Library and Information Practice 2009, 4:4
25
conducted and interviews with each participant provided details of their opinions on
how these new features would influence their research and teaching activities.
Results – Most of the participants either did not notice or ignored the major website
changes. Links to and locations of commonly used resources (e.g. catalogue;
databases; e‐journals) had been changed minimally, and the faculty and graduate
student participants gravitated to those familiar features to complete tasks. Prior to
the study, participants had not accessed the new discovery tools; however, once
previewed, responses to the tools’ utility were generally favourable. After using the
federated search tool on a familiar topic, several participants noted that, when
directed to databases they had not previously considered, they were able to locate
citations they had missed in the past. Observers noted pitfalls in navigating the site
such as inconsistent underscoring of links, ambiguous terminology, and unclear icons
meant to expand subject heading lists. Unexpected searching behaviours were
observed, including inconsistent and lack of conceptual understanding in searching
for e‐journal content.
Conclusions – This study provides evidence regarding the usability of a library
website with a population already familiar with library resources. It demonstrated
that faculty and graduate students are not interested in experimenting with new
discovery tools but are amenable to their potential value to undergraduate students.
The recent trend toward minimizing content and links on websites satisfies this
population, one which is already comfortable with the basic attributes of a library’s
website.
Introduction
Academic research libraries bear the challenge
of meeting information needs of patrons
varying from entering freshmen to eminent
scholars. Historically, face‐to‐face transactions
permitted staff to adjust to the needs,
characteristics, and skills of the individuals
before them. Using information gained from
the reference interview, body language, and
other visual clues, librarians easily guided
diverse patrons to the sources that best met
their informational needs. Today, patrons visit
the library not only through the front door,
but also electronically via library websites.
Library websites bear the burden of serving
the same range of patrons, but without the
same capabilities for adaptation. As libraries
work to better serve one group of patrons,
does this hinder services to others?
During the summer of 2008, the University of
Kansas Libraries revised their home page
to incorporate a
variety of resource discovery tools (Figures 1
and 2). The main objectives were to
incorporate tools for easier searching, such as
a federated search, and to improve access to
subject‐focused resources and services. To
achieve these objectives, a quick‐access tabbed
box for federated, catalogue, and e‐journals
searching was added to the home page along
with a “Research by Subject” access point.
Before the new website went live, the Libraries
undertook a usability study with
undergraduate students. Although the
redesign was aimed to improve
undergraduates’ use of library resources, the
design team desired to also understand how
graduate students and faculty might use it as
researchers and as instructors. Due to time
constraints, a usability study with graduate
students and faculty did not occur until two
months after the redesigned website went live.
Literature Review
Leo Robert Klein declared in 2003 that, “The
Expert User is Dead.” Klein urged librarians to
recognize that they could not create websites
http://www.lib.ku.edu/
Evidence Based Library and Information Practice 2009, 4:4
or support databases with complicated
searching interfaces, techniques, and countless
options. Although librarians aspired to change
users into experts, Klein reinforced that
attempting to do so was unrealistic and
impractical. Instead, he suggested librarians
redefine their expectations of patrons and to
work at giving users what they wanted (36).
Commentators and critics like Klein advocated
librarians to adapt technology to meet users’
expectations.
As the Web matured and matriculating college
students came of age with the Internet,
libraries faced a stark reality. Although
librarians suspected that college students
turned to the Web for information before they
visited the bricks‐and‐mortar library, a 2005
OCLC report confirmed this. The authors of
College Students’ Perceptions of Libraries and
Information Resources concluded that 89% of
undergraduates begin their research with a
search engine (De Rosa et al. 7). Even prior to
OCLC’s quantification of undergraduate
information‐seeking behaviours, librarians
looked for solutions to make using library
resources more appealing. A number of
commentators have argued that federated
searching could match Google in ease of use
and return better results than a Web search
(Bell 107‐108; Luther 36; Tennant 28). A simple
search box integrated multiple databases,
assumed the Boolean AND operator, and
required little library knowledge from the user
(Kitalong, Hoeppner, and Scharf 178). This
seemed like the perfect solution to meet the
needs of undergraduates because they “expect
to take their Google searching skills and apply
them to find library resources” (Ponsford and
van Duinkerken 162).
Federated searching has not been without
criticism. Critics direct their frustration toward
the duplicated results from multiple sources,
slowly executed searches, and results
presented in no particular order (Belliston,
Howland, and Edwards 474; Tang, Hsieh‐Yee,
and Zhang 216). Yet librarians have generally
accepted its use as a discovery tool for users
lacking basic searching skills. For these
reasons it is perceived as a tool to initiate
searching for undergraduates and not for
extensive use by graduate students or faculty.
Librarians have adopted usability testing as an
essential step in the Web design process. Jakob
Nielsen, an authority on usability, advocates
that websites undergo frequent examination.
Nielson asserts that testing more than five
participants is a waste of resources because it
is the point of diminishing returns. According
to Nielsen, the only time testing needs to
extend beyond five is when a site has many
distinct user groups (Nielsen).
Library literature contains many usability
studies involving federated searching, with
the majority focusing on undergraduates. This
is not surprising since undergraduates make
up the largest portion of users in academic
libraries. Their lack of awareness of library
resources and tools is reflected in their
perception of library websites (Kitalong,
Hoeppner, and Scharf 178). Ed Tallent found
in an early study that students do not care to
learn about the various avenues available to
obtain information. Instead, an individual user
desires to have everything “combined for
searching and sees little need to separate the
access” by resource (71). But federated
searching is not without its problems. As
Tang, Hsieh‐Yee, and Zhang found, patrons
do not understand what they are searching or
how to interpret results (216). Belliston,
Howland, and Roberts emphasize that even
with these problems, federated searching
remained the choice of undergraduates. In
their examination at three Brigham Young
University campuses, the authors determined
that undergraduates favoured the speed of the
federated search over traditional searching
(479‐480). There are no published studies
focusing solely on graduate students and
faculty, and those studies that do include these
more advanced users among the participants
make few references to the reactions of this
subset of patrons. Belliston and his fellow
authors suggested the need for studies
specifically exploring graduate student and
faculty perceptions of federated searching
(480).
26
Evidence Based Library and Information Practice 2009, 4:4
Usability studies focusing on library websites
have consistently documented the typical
behaviours of users. Overall, patrons have
clearly established expectations of websites
based on their prior Web experiences (George
14; Ponsford and van Duinkerken 162).
Patrons are most likely to read and use links
on the left‐hand side of the screen. (Fichter 8;
Whang and Ring 80). Users do not want
anything elaborate; they expect the basics.
Links, for example, need to be clearly
identifiable as links and library jargon should
be avoided. In a 2005 test, patrons sought help
on the “Library Instruction” page. Unaware of
the meaning, they assumed the page provided
instruction on using the library (Cobus, Dent,
and Ondrusek 237‐238). The simplicity and
success of Google reinforces that sites can be
uncomplicated and still be effective. Jung et al.
determined, “by attempting to make all
resources visible, the library has perhaps
made none very accessible” (384). In creating a
more user friendly website and not one just for
information professionals, the need for
multiple access points leading to the same
place has become apparent (Fichter 7).
The majority of usability studies explored only
the reactions of undergraduates or a
combination of undergraduates along with an
insignificant number of graduate student and
faculty participants. Overall, reported findings
inform the profession only about the habits of
undergraduates and not those of advanced
researchers. A gap in the literature exists
regarding graduate student and faculty
opinions about and usage of these new
discovery tools. This study set out to address
these gaps.
Methods
Ten participants consisting of five faculty and
five graduate students were solicited for the
study. The participants were recommended to
the observers by library faculty subject
liaisons, and were selected to represent the
four main disciplines from across the
university: Humanities, Social Sciences,
Science/Technology, and International
Programs. The graduate students represented
English, Geology, Journalism, Musicology,
and Russian, East European & Eurasian
Studies (REES). The faculty included
individuals from Business, English,
Linguistics, Math, and Spanish/Portuguese.
Prior to each observation session, the panel of
five observers selected one spokesperson to
recite the set of Institutional Review Board‐
approved questions (see Appendix), which
directed each participant to access several
features of the Libraries’ website, including a
new federated search tool. The questions were
intended to address new and updated features
of the website and to ask participants’
opinions of how these features would
influence their research and teaching activities.
A specific goal was to understand how and
whether the federated search tool, structured
as a Google‐like search and located at the top‐
center position of the home page, might be
utilized. Each participant was interviewed
separately during a one‐hour session, and
participant mouse clicks and audio were
captured using Morae™ software, a well‐
known product for recording usability testing
sessions. Morae™ allowed the researchers to
later review participants’ actions as they
navigated the site and to match written
observations with actual screen captures.
The Libraries’ home page (Figure 1) was
redesigned in August 2008 (Figure 2) to serve
as a database and subject portal, facilitating
the linking of researchers to the information
that would meet their needs. The federated
search tool, 360 Search™ from Serials
Solutions, was added in the fall of 2008 to
provide a means for a quick search of selected
databases directly from the Libraries’ home
page. The databases selected for inclusion in
the basic search were Academic Search
Premier, Business Source Premier, Project
MUSE, and Wilson OmniFile. Tabbed
searching also included quick access to the
library catalogue and e‐journal search boxes
(Figure 3).
27
Evidence Based Library and Information Practice 2009, 4:4
Fig. 1. KU Libraries Home Page Prior to the Redesign.
Fig. 2. KU Libraries Home Page as Presented for the Study.
28
Evidence Based Library and Information Practice 2009, 4:4
Results
Overall Response
Ninety percent of the participants did not
know about, ignored, or found ways to work
around the major website changes. For
example, the Spanish/Portuguese professor
indicated he had noticed the changes prior to
participating in the study but had chosen to
avoid them entirely by utilizing a link to the
library catalogue from the University’s
Libraries page. All of the participants were
interested in knowing more about the purpose
and function of the new features. The main
navigation scheme of the site had been
purposely maintained with minimal changes
to facilitate a smoother transition for users
who were already familiar with the site.
Observers noted that the participants
gravitated to familiar features to complete
many of the tasks requested of them. Once the
tools had been previewed, participants’
responses were favourable regarding their
utility for some purposes and users.
Federated Search
Participants were asked to give their
impressions of the quick search and to explain
the purpose of each tab in the search box (see
Appendix). The “Articles & Databases Quick
Search” tab puzzled the REES graduate
student. She stated, “I know there’s a way to
select specific databases to search at once,” but
was not sure how to determine which
databases were included in the search.
Participants were then asked to conduct a
basic search using the federated search tool
Fig. 3. Quick Search Tabs on Libraries’ Home Page.
and to comment on the results. Some of them
found results that related to their topic, but the
majority commented that the results were
unrelated or too broad. The graduate student
in Musicology tried a search for “Peter, Paul
and Mary” and got results as varied as
authors’ names in a medical journal and Peter,
Paul, and Mary Magdalene. When she ran the
search a second time with her search terms in
quotations, she was pleased to find a citation
that she had not previously discovered.
Another participant stated that, while some of
the results might be useful, it would take too
much time to filter through all of the citations.
After using the federated search tool on a
familiar topic, several participants noted that
by being pointed to databases they had not
previously thought of using, they were able to
locate citations they had missed in the past.
When asked if they would recommend the
federated search tool to their peers or their
students, all participants stated that they
would recommend the tool to undergraduate
students as a good place to begin their
research. According to one person, “Students
don’t often know where to start. A more
restricted database set in a federated search
can be helpful and less intimidating.” While
most participants had a favourable view of the
multidisciplinary nature of the results
presented in the federated search, a few
remarked that they would not be very likely to
use the tool themselves, as they tended to rely
on one to three subject‐specific databases for
the bulk of their research activities.
In some subject areas, a subject‐specific
federated search tool was made available to
29
Evidence Based Library and Information Practice 2009, 4:4
allow users to narrow their search to databases
selected by a subject librarian (Figure 4).
Responses to this tool were similar to reactions
towards the quick search on the home page.
Some participants were surprised to be
presented with a search tool tailored to their
discipline. Two commented that there were
databases included in the search tool that they
had not been aware of previously and that
they would be more likely to use the subject‐
specific tool as a result.
Journal Article Search
The investigators asked participants to take a
citation for an article from Social Science
Quarterly and demonstrate how they would
identify whether the article was available in
the Libraries (see Appendix). The results of
this exercise were varied; half of the
participants utilized the E‐journals link in the
left‐hand navigation section of the home page
and searched for the journal title. Other
methods included searching for the article title
using the “Articles & Databases Quick
Search”, using the online catalogue to search
for the journal title, and navigating to a
Fig. 4. Example of Subject Page for English Language and Literature.
specific social science database to search for
the article title. The least successful methods
employed by participants involved searching
for the journal title in a search box that was set
up to search for database titles, and searching
for the article’s author in the online catalogue.
The investigators were surprised that half of
the participants began the exercise by
searching for the article title or author rather
than the journal title. Of the five who used this
approach, two were faculty. This may point to
a changing perception among scholars that
journal articles are unique entities apart from
the journals in which they are published. As a
result, search structures or tools that force
scholars to navigate to an article through a
database‐to‐journal‐to‐article path may
present a challenge.
Navigation
Observers noted pitfalls in navigating the site,
such as inconsistent underscoring of links and
unclear library verbiage. When participants
were asked to identify the purposes of the
three tabs on the search box on the home page,
their difficulty pointed to shortcomings in the
labeling and help links on the tabs.
30
Evidence Based Library and Information Practice 2009, 4:4
Additionally, 70% of the participants,
including four graduate students and three
faculty members, did not know how to click
on the plus symbol (+) to expand the main
subject headings to view the second tier of
database subject headings on the “Research By
Subject” page (Figure 5). This came as a
surprise to the observers, given that the tree
structure of the subject list was assumed to be
a ubiquitous navigation feature.
Research Help
An unexpected result of the study was that,
when asked the question “If you need detailed
information about a subject in your area,
where would you go for information?” 80% of
the participants responded that they would
contact their subject librarian first. This
response was evenly divided between faculty
and graduate students. Other responses
included clicking the “Research by Subject”
link or seeking help at a reference desk. This
contradicts the current literature that suggests
that declining statistics of face‐to‐face
reference interviews are indicators of a
changing perception of librarians as a primary
source for research assistance (Applegate 184).
Discussion
With studies indicating less reliance on
libraries for information, it is crucial for
institutions to learn how resources are being
accessed not just by undergraduates but also
by more advanced researchers (Housewright
and Schonfeld 5). For many constituents, a
library’s website may be their only interaction
with its services and collections. How can
websites be augmented and improved to
enhance the patrons’ experiences and,
Fig. 5. Research By Subject Page with Subject Tree Structure.
hopefully, encourage them to return again?
The presentation and ease of use are crucial.
Clear navigational features such as font size,
unambiguous links, and comprehensible
terminology are minor attributes when
compared to content, but unless these simple
features are obvious, the content will not be
read. Usability studies such as this one inform
and often supplant misinformation and
assumptions of library staff about how patrons
access library websites.
One of the major objectives of the study was to
learn how the participants reacted to the quick
search box introduced on the front page,
which included a federated search across four
pre‐selected databases (Figure 2). Several
years previously, a similar attempt to
introduce a federated searching tool, from a
vendor now out of business, was
technologically unmanageable and perceived
31
Evidence Based Library and Information Practice 2009, 4:4
as not user‐friendly by the library staff. The
decision to implement a federated search tool
a second time supported the commitment to
improve resource discovery and make the
search easier and more familiar to patrons.
Even with an improved federated search
product, some staff stated that the research
process was being “dumbed down” and that
more effort should be placed on instructing
patrons to effectively apply proven database
search techniques. The prominent quick search
box, resembling a Google‐like search, was
intended to grab the attention of those
undergraduate students who are unaware of
where to begin a search for information from
the Libraries’ website. The federated search
would hopefully retrieve a few good articles to
start them on a research project. The observers
wondered if the faculty, in particular, would
have similarly negative reactions to
simplifying the initial search procedure.
Several participants identified the potential
value of the tool. Two of the faculty
recognized that students often do not know
where to start conducting research or which
database to select. The English faculty member
noted that the clustered results identified
common keywords that students might not
think of using as search terms. Even though
the faculty might not use the federated search
tool in their research, they acknowledged the
benefit of the tool for undergraduate students.
This study indicates that federated searching
as a resource discovery tool would not be
extensively used by faculty and graduate
students for their own research projects.
Apart from the quick search box, the visual
and contextual changes to the website were
not obtrusive, as these modifications were
mostly unnoticed by this study’s participants.
Familiar links and labels had been carried over
from the previous version to the updated
version of the Libraries’ home page. “Articles
& Databases”, “Catalog”, and “Request
materials” links were recognizable to the
participants and their presence overshadowed
the new features (Figure 2). These links were
retained on the left‐hand navigation panel on
the redesigned website and were typically
selected as access options.
A new feature intended to inform patrons of
available resources in a subject area was
entitled “Research by Subject” (Figure 2). This
link leads users to organized lists of databases,
websites, contacts, and research guides by
subject. No participants in the study had
previously reviewed the content, and none
seemed overly enthusiastic about the
organization of these sources by subject, nor
expressed an interest in using this feature as a
teaching tool. The organization attempts to
compartmentalize “databases” and scholarly
“websites” which appeared to be a foreign
concept. The English faculty member had
previously attempted to locate a noteworthy
Middle English website was not aware that the
Libraries’ divided scholarly websites and
licensed databases into two distinct silos
(Figure 4). The participants believed they were
knowledgeable of the major resources in their
area and consequently did not express a need
to consult a “Resources by Subject” site. The
reactions to the subject pages indicate that this
grouping of resources would not be of any
particular value for research or teaching for
these participants. Subject guides are most
pertinent when designed for specific courses
where context helps to reinforce resources
introduced during classroom instruction (Reeb
and Gibbons 126). KU Libraries will continue
to address subject organization with the
implementation and development of
LibGuides™ by Springshare and emphasize
the application specifically for courses.
Locating journal articles from known citations
is a difficult concept even for some advanced
researchers. The traditional paradigm of a
journal with issues and articles identified
within those issues is shifting even though
many electronic journals continue with those
traditional enumerations. Articles can be
identified through the searching of databases
and Google, and are therefore not necessarily
tied to the context of an issue and volume as
with the print version. Patrons often
successfully find journal articles by searching
for the article title in a federated search or
database, rather than by the traditional means
of searching for the journal title and locating
the volume (George 13). An awareness of how
32
Evidence Based Library and Information Practice 2009, 4:4
patrons are locating articles will help library
staff to guide patrons to other access options.
Each participant had his/her own unique
approach to searching the Libraries’ website
and locating sources. Several had previously
identified their favourite databases and did
not deviate from using them. MathSciNet was
the exclusive database for the Mathematics
professor. The Musicology graduate student
relied on ProQuest Dissertations and Theses
for recent data not available elsewhere. The
Journalism graduate student only considered
resources with full‐text articles available in the
database or through the link resolver. The
Linguistics faculty member found library
hours by searching Google. To locate
databases or websites, the English faculty
member used the “find” application on the
browser particularly helpful if the exact title
was not known. The participants’ attention
was channeled to those links and sites that had
worked well for them in the past, and they
tended to avoid taking the time to explore the
potential value of uninvestigated links and
resources.
Conclusion
Faculty and graduate students participated in
a usability study of the University of Kansas
Libraries’ redesigned website two months
after its launch. The redesign included new
navigational features, introduction of a
federated search tool, a quick search box on
the front page, and subject‐specific research
pages. Most of the participants reported that
they had not paid much attention to the
changes. The left‐hand navigation links on the
page had been minimally changed during the
revision process, and most participants were
content with using the familiar links to
databases, e‐journals, library catalogue, and
interlibrary loan. Very little interest was
exhibited in experimenting with the new
features implemented during the redesign.
Once prompted to try the federated search
tool, participants noted the potential value for
novice researchers and interdisciplinary
research, along with discovery of relevant
citations in databases not previously searched.
This evidence proved that the website design
could satisfy the information needs of the
advanced researchers while introducing
resource discovery features intended for
undergraduates. This study revealed how
faculty and graduate students struggled to
locate the full text of a specific article from a
citation. To improve navigation of the website,
recommendations for highlighting links,
improving labels, and particularly changing
the tree structure to expand database subject
lists were submitted to an internal Web
services council. No suggestions for changes to
the federated search tool were included in the
final list of recommendations. Studying
faculty and graduate student researchers as a
separate population from undergraduate
students generated distinct data, and
reinforced the need for adequate
representation from the varied populations
that the University of Kansas Libraries serves
when conducting usability testing.
33
Evidence Based Library and Information Practice 2009, 4:4
Works Cited
Applegate, R. “Whose Decline? Which
Academic Libraries are ʺDesertedʺ in
Terms of Reference Transactions?”
Reference & User Services Quarterly
48.2 (2008): 176‐89.
Bell, Colleen. “‘Don’t Make Me Think’: A Plea
for Simplicity and Transparency.”
Public Services Quarterly 2.4 (2007):
103‐9.
Belliston, C. Jeffery, Jared L. Howland, and
Brian C. Edwards. “Undergraduate
Use of Federated Searching: A Survey
of Preference and Perceptions of
Value‐Added Functionality.” College
& Research Libraries 68.6 (2007): 472‐
86.
Cobus, Laura, Valeda Frances Dent, and Anita
Ondrusek. “How Twenty‐Eight Users
Helped Redesign an Academic
Library Web Site.” Reference & Users
Services Quarterly 44.3 (2005): 232‐46.
De Rosa, Cathy, Joanne Cantrell, Janet Hawk,
and Alane Wilson. College Students’
Perceptions of Libraries and
Information Resources. Dublin, OH:
OCLC Online Computer Library
Center, Inc.: 2006. 13 March 2009
.
Fichter, Darlene. “Designing a Better Subject
Page to Make Users’ Searches More
Successful.” Computers in Libraries
25.9 (October 2005): 6‐8, 54, 56.
George, Carole A. “Lessons Learned: Usability
Testing a Federated Search Product.”
The Electronic Library 26.1 (2008): 5‐
20.
Housewright, Ross, and Roger Schonfeld.
“Ithaka’s 2006 Studies of Key
Stakeholders in the Digital
Transformation in Higher Education.”
18. August 2007. Web. 28 April 2009.
.
Jung, Seikyung, Jonathan L. Herlocker, Janet
Webster, Margaret Mellinger, and
Jeremy Frumkin. “LibraryFind:
System Design and Usability Testing
of Academic Metaserach System.”
Journal of the American Society for
Information Science and Technology
59.3 (2008): 375‐89.
Kitalong, Karla Saari, Athena Hoeppner, and
Meg Scharf. “Making Sense of an
Academic Library Web Site: Toward a
More Usable Interface for University
Researchers.” Journal of Web
Librarianship 2.2/3 (2008): 177‐204.
Klein, Leo Robert. “The Expert User is Dead.”
Library Journal 128.17 (Fall 2003 Net
Connect): 36.
Luther, Judy. “Trumping Google?
Metasearching’s Promise.” Library
Journal 128.16 (2003):
36‐9.
Nielsen, Jakob. “Why You Only Need to Test
with 5 Users.” Jakob Nielsenʹs Alertbox
19 March 2000. Web. 16 February 2009
.
Ponsford, Bennet Claire and Wyoma van
Duinkerken. “User Expectations in the
Time of Google: Usability Testing of
Federated Searching.” Internet
Reference Services Quarterly 12.1/2
(2007): 159‐78.
Reeb, Brenda and Susan Gibbons. “Students,
Librarians, and Subject Guides:
Improving a Poor Rate of Return.”
portal: Libraries and the Academy 4.1
(2004): 123‐30.
34
http://www.oclc.org/reports/pdfs/studentperceptions.pdf
http://www.oclc.org/reports/pdfs/studentperceptions.pdf
http://www.ithaka.org/ithaka-s-r/research/Ithakas%202006%20Studies%20of%20Key%20Stakeholders%20in%20the%20Digital%20Transformation%20in%20Higher%20Education.pdf
http://www.ithaka.org/ithaka-s-r/research/Ithakas%202006%20Studies%20of%20Key%20Stakeholders%20in%20the%20Digital%20Transformation%20in%20Higher%20Education.pdf
http://www.ithaka.org/ithaka-s-r/research/Ithakas%202006%20Studies%20of%20Key%20Stakeholders%20in%20the%20Digital%20Transformation%20in%20Higher%20Education.pdf
http://www.ithaka.org/ithaka-s-r/research/Ithakas%202006%20Studies%20of%20Key%20Stakeholders%20in%20the%20Digital%20Transformation%20in%20Higher%20Education.pdf
http://www.ithaka.org/ithaka-s-r/research/Ithakas%202006%20Studies%20of%20Key%20Stakeholders%20in%20the%20Digital%20Transformation%20in%20Higher%20Education.pdf
http://www.useit.com/alertbox/20000319.html
http://www.useit.com/alertbox/20000319.html
Evidence Based Library and Information Practice 2009, 4:4
Tallent, Ed. “Metasearching in Boston College
Libraries—A Case Study of User
Reactions.” New Library World 105.1
(2004): 69‐75.
Tang, Rong, Ingride Hsieh‐Yee, and Shanyun
Zhang. “User Perceptions of MetaLib
Combined Search: An Investigation of
How Users Make Sense of Federated
Searching.” Internet Reference
Services Quarterly 12.1/2 (2007): 211‐
36.
Tennant, Roy. “The Right Solution: Federated
Search Tools.” Library Journal 128.11
(2003): 28‐9.
Whang, Michael and Donna M. Ring.
“Student‐Focused Usability Study of
the Western Michigan University
Libraries Home Page.” Journal of Web
Librarianship 1.3 (2007): 67‐88.
35
Evidence Based Library and Information Practice 2009, 4:4
36
Appendix
Questions:
A. Please explain for what purpose(s) you use the Libraries’ Web site.
B. Think of a previous project where you had to consult the Libraries’ Web site, preferably
before the redesign. Please show us how you would go about completing the project on this
site.
C. What is one of your favorite databases? How would you access the database?
D. With the redesign, we’ve inserted a search box on the front page. Please explain to us what
you think each tab in the search box is for. Would you use any of these 3 tabs? (Ask the
participant to run a search on a favorite topic under Quick Search.) Would you recommend to
your students to use the Quick Search box?
E. If you need detailed information about a subject, e.g. the specifics on searching a research
database, where would you go for information?
F. Research by Subject page questions.
a. How does this page help you to identify databases in a subject area?
b. Would any of the other links on this page be useful to you? (Web sites,
contacts/guides)
c. (Pull up a subject page with a 360 search option and run a search on a favorite topic
or ask the participant to select databases to conduct a federated search.) Review the
results. What do you think of capability of searching across pre‐selected databases?
Useful to their research? Useful to their students?
G. What link do you access to ask for materials not available in the Libraries?
H. How would you identify if the following journal article is available in the Libraries:
a. Rocca, M. Personal attributes and Latino voting behavior in Congress. Social Science
Quarterly, 89(2):392+. 2008.
I. What is your overall reaction to the Libraries’ Web site? Suggestions?