ACRL News Issue (B) of College & Research Libraries 12 / C &RL N e w s ■ J a n u a ry 200 1 INTERNET RESOURCES Resources for library assessment Tools for a new era by Kathleen Bauer L ibrarians are increasingly turning to the Internet and electronic resources to pro­ vide patrons w ith n ew services. As electronic resources and services becom e an increas­ ingly large segment o f all library resources and services, librarians must find w ays to ad­ equately report their usage. Finding adequate and accurate measures becom es more important as libraries experi­ e nce a decline in traditional services such as questions at the reference desk. For many libraries, electronic resources represent an area o f growth, and so reflecting their use in library statistics is k ey to justifying budget requests and demonstrating the importance o f the library. There are many w ays o f measuring library usage: typically librarians count b ooks circu­ lated, volum es shelved, m oney spent on ac­ quisitions, gate counts, and questions an­ sw ered at the reference desk. W hile these measures are w idely accepted, they do not reflect electronic services. To accurately re­ flect this usage, libraries must at a minimum be able to count and report Web hits, data­ base sessions and searches, and e-mail con­ tacts. Counting these things w ill force librar­ ians to look at some n ew tools for assess­ ment. In this climate, w here electronic and print resources and services co-exist, libraries seek­ ing to perform careful analysis o f usage and funding w ill n eed to use both traditional and n ew tools. Indeed, this n ew climate m ay rep­ resent an opportunity for librarians to review all the data that they gather and all o f the tools they use to analyze that data. Some o f the tools librarians may find helpful are avail­ able via the Internet, including software pack­ ages for studying Web usage and for perform­ ing statistical analyses, sites that provide data for use in benchmarking with comparable libraries, and sites that can help w ith study­ ing dem ographics in their community. W e b a n a ly sis so ftw a re M easurem ent o f W eb u sa ge d e p e n d s on stu d yin g server lo g files, w h ic h are text file s that re co rd e a c h tim e a file o n its server is req u ested . T his line at a mini­ mum includes the date and time, IP address o f the requester, size in bytes o f the file re­ quested, and a result status o f every hit on a W eb site. Log files w ere actually designed to help network administrators gauge traffic on a site, not to judge h o w p eo p le use a site, and so they are not a particularly elegant tool for studying h o w som eone uses a site. However, they are the only tools available, w ithout us­ ing cookies or registration and logins, w hich can measure usage o f files on a Web site. Log files are usually incredibly long text files that would b e impossible to interpret b y hand. A b o u t th e a u t h o r Kathleen Bauer is librarian at the Yale School o f Medicine, e-mail: kathleen.bauer@yale.edu mailto:kathleen.bauer@yale.edu C& RL News ■ January 2001 /1 3 Luckily software is available to help parse these files. • Analog. Analog is a popular free software tool designed by Stephen Turner from the Uni­ versity of Cambridge. This is very basic soft­ ware that will do simple analysis o f log files. Nothing fancy, but it will tell you the number of hits on your Web site and the most requested files. Access: http://www.analog.cx/. • WebTrends. This software is offered in different packages at different prices depend­ ing on the complexity o f your Web site and the desired level of sophistication o f analysis. WebTrends lets the user customize detailed reports through a m en u -d riven system . WebTrends offers more graphics than the free software alternatives. WebTrends reports search strings and search engines used to find your site. Access: http://www.webtrends.com/. Demographics To accurately plan for what new ser­ vices are necessary and desired by pa­ trons, librarians need to understand de­ m og raphics in th e ir c o m m u n itie s . Demographic information can help libraries to better understand the population they serve and to look for trends that may affect them in the future. • U.S. Census B u reau A ccess Tools. The U.S. Census Bureau gathers detailed data on the population through the diennial census and other surveys. Tools available at this site include State and County Quickfacts, where commonly asked for county-level informa­ tion can be accessed through a series o f pull­ d o w n b o x e s . Data available in c lu d e p e r ­ cen tage o f the p o p u latio n by r a c e , g e n d e r, education level, home ownership, and median household in­ come listed with the corresponding state-level information. More detailed census tract level information is available from Censtats.1 Access. http://www. census.gov/main/www/access. html. Eco n o m ic b en ch m arks Benchmarking data can help a library design reports that show where they stand in rela­ tion to other libraries or in relation to more general indicators o f the economy. • C o n su m er P rice In d ex. Economic data indicate w hether a library’s funding is keep­ ing pace with inflation. An important infla­ tion indicator is the Consumer Price Index, produced by the Bureau o f Labor Statistics in the Commerce Department o f the U.S. gov­ ernment. This index measures the cost o f an imaginary basket o f market goods, and re­ ports the increase or decrease in that price. Data is available on a monthly and an an­ n u al b a sis. A c c e s s : http://stats.bls.gov/ cpihome.htm. Lib ra ry b en ch m arks • A s s o c ia tio n o f R e s e a r c h L ib ra rie s (ARL) S tatistics a n d M e a s u re m e n t P r o ­ g ra m . The Association o f Research Libraries has b een measuring various aspects o f library services since the 1970s and some o f this data is available on their Web site. A drawback to ARL ‚ Statistics and Measurement Program this data is that ARL represents an elite group o f 120 large academic and research libraries, so the data is not a good reference point for all libraries. It does however represent a sig­ nificant repository o f information about trends in some libraries. Data available include the “ARL Annual Salary Survey” with selected data available from 1995-99. Data collected from 1973 to 1994 are available only in print. Data include average and median salaries by race, gender, position, experience, and geographic region. Also available from ARL is the “Library Ma­ terials Budget Survey” com piled by Collec­ tion Developm ent Officers o f the Large Re­ search Libraries Discussion Group. This data com es from an even smaller group o f 40 li­ braries in ARL, although, starting in 1998, the budget survey was sent to all ARL libraries. Detailed spreadsheets are available dating back to 1987. The survey began to ask about expenditures for electronic resources in the 1 9 93-94 fiscal year, but this information is still sketchy. For exam ple, no members re­ ported electronic expenditures for 1998-99. http://www.analog.cx/ http://www.webtrends.com/ http://www http://stats.bls.gov/ 14 / C&RL News ■ Ja n u ary 2001 A third source o f data from ARL is the “E&G Report,” or library exp en ses as a per­ centage o f educational and general exp en d i­ tures. Data available starting in 1982 show steep declines in the percentage o f university dol­ lars spent on libraries. A ccess: http://www. arl.org/stats/. • N atio n a l C e n te r f o r E d u c a tio n S tatis­ t ic s (N C ES) L ib r a r y S ta tis tic s P r o g r a m . NCES b eg an collectin g library statistics in 1989. Inform ation is available from a range o f library types, including academ ic, school and public libraries. T h e NCES’s Academic Library Survey is a com plem ent to the Inte­ grated Postsecondary Education Data System (IPEDS). Data available for academ ic librar­ ies include total salaries, gate count, exp en ­ ditures, expenditures per full-time equivalent student, expenditures per category (e.g., com ­ puter hardware and software, serials subscrip­ tions, etc.), and reference transactions. Access: http ://nces. ed . gov/ surveys. • State L ib ra ry R e s o u rc e s A rra n g e d b y State. This is a compilation o f data available for libraries in the 50 individual states by Jo e Ryan. Each state com piles data independently and this is not a coordinated effort to collect data for com parison betw een states. A ccess: http://web.syr.edu/~jryan/infopro/statsl.html. • U.S. N atio n al C o m m is s io n o n L ib ra r­ ies a n d I n f o r m a tio n S c ie n c e (N C O S ) Sta­ tis tic s a n d S u rv ey s. This site contains the results o f a series o f surveys beginning in 1995 discussing public libraries and Internet connectivity. A ccess: http://www.nclis.gov/ statsurv/ statsurv. htm l. M e a su rin g u sa g e o f e le c tro n ic re s o u rc e s Unfortunately there are no accepted standards for gathering and reporting data about elec­ tronic resources, and there are n o nationally av a ila b le d ata fo r b e n c h m a r k in g . S o m e groups that follow are attempting to create standards. H ow ever n o n e o f th ese groups offer an easy blueprint for others to adopt. They serve as a starting point in thinking about measuring usage o f electronic resources in your library. • D e v e lo p in g N a tio n a l L ib r a r y N et­ w o r k S tatistics & P e r f o r m a n c e M easu res. Not m uch at this site seem s to b e working, but “Proposed Data Elements/Statistics” can help give you a b rief idea o f som e things you could use to measure usage o f electronic resources in your library. T h ese proposed measures cam e from a project run by Jo h n Carlo Bertot, an associate professor at the Sch ool o f Inform ation Scien ce and Policy o f the State University o f New York at Albany. A c c e s s : h ttp ://www.albany.edu/~im lsstat/ propstats.html. • E q u in o x L ib r a r y P e r f o r m a n c e M ea­ s u r e m e n t a n d Q u ality M a n a g e m e n t Sys­ te m . Equinox is a European project funded by T elem atics fo r Libraries. T h e E qu inox project proposes a list o f possible library per­ form ance indicators and seeks to encourage libraries to adopt these indicators. T he pur­ p ose is to provide standard data that would allow com parison b etw een libraries. How­ ever, these have not b een m ade into a widely agreed to standard. A ccess: http://equinox. dcu.ie/. • G uidelines f o r S tatistical M easures o f U sag e o f W eb -B ased I n d e x e d , A b s tra c te d , a n d F u ll T e x t R e s o u rc e s . Adopted in 1998 by the International Coalition o f Library Con­ sortia (ICOLC), these guidelines represent the data the ICOLC wish database vendors to in­ clude in statistical reports. This represents another method in attacking the problem o f the lack o f standards. Instead o f asking li­ braries to adopt the standards, these guide­ lines are aimed at database vendors and elec­ tronic full-text publishers. The guidelines re­ quest that vendors supply to libraries data on a monthly basis and give counts o f the num­ bers o f searches and sessions in each data­ b ase. A c c e s s : http://www.library.yale.edu/ consortia/webstats.html. S ta tis tic a l s o f t w a r e a n d o th e r s t a tis tic s re so u rc e s Carrying out assessm ent activities will at some point require the u se o f statistical manipula­ tion o f data. Most spreadsheet software prod­ u cts in clu d e som e statistical cap ab ilities. M icrosoft Excel, for exam ple, can perform many statistical calculations, such as measure­ ments o f central tendency (e.g., average), stan­ dard deviation, t-tests, and others. More so ­ p h is tic a te d a n a ly s is m ay r e q u ir e m ore powerful com puting abilities. T h e two com ­ panies listed b elow offer statistical software packages o f varying pow er and price. ( c o n t i n u e d o n p a g e 2 8 ) http://www http://web.syr.edu/~jryan/infopro/statsl.html http://www.nclis.gov/ http://www http://equinox http://www.library.yale.edu/ 2 8 / C&RLNews ■ January 2001 arly communications crisis are different in each discipline, but they are intertwined. The dia­ log is enriched by the opportunity for faculty from different disciplines to assess the con ­ sequences o f proposed actions and solutions from their own perspectives. The Tem pe Principles recognize that high costs and restrictive licenses are symptoms o f a d eeper crisis in the scholarly com muni­ cations system. Any on e library or any one university working in isolation cannot resolve this crisis. Even so, discussions leading to collective determ ination to alter scholarly practice must begin at local levels, particu­ larly am ong colleges and universities that em ploy most working scholars and set the standards for prom otion and tenure. Librar­ ies cannot b e the prim ary, arena in which those changes are enacted; it is vital that teach­ ing faculty and researchers assume responsi­ bility for resolving this crisis. However, li­ braries and librarians can act as an important institutional catalyst by initiating and spon­ soring campus discussion. Notes 1. The Tem pe Principles are available on the ARL W eb site at http://www.arl.org/ scomm/tem p e. html 2. Sem in ar m aterials are p o s te d at http://www2.lib.ukans.edu/scholcomm/tempe/ tempe.htm. ■ ( “R es o u rce s . . . ” c o n t in u e d f r o m p a g e 1 4 ) • SAS. Software from the SAS Institute. A ccess: http://www.sas.com/. • R ice V irtu al Lab in S tatistics. This site offers som e nice reviews o f statistical con­ cepts. It includes HyperStat, an online text­ book, and simulations that demonstrate how s o m e sta tistic s e q u a tio n s w o rk . A c c e s s : http://www.ruf.rice.edu/~lane/rvls.html. Note 1. At the time this review was written, data from the 2000 census w ere not avail­ able. ■ http://www.arl.org/ http://www2.lib.ukans.edu/scholcomrn/tempe/ http://www.sas.com/ http://www.ruf.rice.edu/~lane/rvls.html mailto:custserv@archival.com http://wwvv.archival.com