Evidence Based Library and Information Practice Evidence Based Library and Information Practice 2010, 5.3 62 Evidence Based Library and Information Practice Commentary Musings on Collection Analysis and Its Utility in Modern Collection Development Jane Schmidt Head, Collection Services Ryerson University Library Toronto, Ontario, Canada Email: jschmidt@ryerson.ca Received: 03 Aug. 2010 Accepted: 12 Aug. 2010 2010 Schmidt. This is an Open Access article distributed under the terms of the Creative Commons- Attribution-Noncommercial-Share Alike License 2.5 Canada (http://creativecommons.org/licenses/by-nc- sa/2.5/ca/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly attributed, not used for commercial purposes, and, if transformed, the resulting work is redistributed under the same or similar license to this one. Introduction Academic librarians are frequently called upon to provide library reports in support of new program proposals and for periodic program reviews. Accreditation and quality assurance bodies want to know that students have access to library services and collections that are sufficient to aid them in the successful completion of their degrees. The individuals responsible for the completion of these reports vary from institution to institution, but the initial request almost always goes to the liaison librarian first. Inevitably, a request for collection analysis makes its way to the Collections Department. As a collection manager in a university that serves a large undergraduate population and an ever growing graduate cohort, I have often been dismayed at the lack of specificity from various accreditation bodies that request library reports, and further bewildered by the level of detail about physical collections desired by librarians charged with completing these reports. In the five years that I have been engaged in collection analysis, I have formed definite opinions about the traditional methods and the lack of quality and utility of the data gathered from these processes. But as we know, old habits die hard. This commentary is an attempt to articulate what I feel academic libraries should prioritize when demonstrating their ability to support a new or existing program with their services and collections. What Defines Library Support? The scholar’s view of the library has been changing steadily over time. According to an Ithaka Report, that change has not been in a favourable direction. Last year, only 24% of faculty surveyed used the library’s online catalogue as a starting point for their research (Kolowich, 2010). Why is it, then, that the librarian’s first inclination is to study the holdings of library catalogues in extraordinary detail in order to demonstrate that the library can support “X” program? mailto:jschmidt@ryerson.ca Evidence Based Library and Information Practice 2010, 5.3 63 My institution, Ryerson University, will soon be undergoing accreditation review for the Association to Advance Collegiate Schools of Business (AACSB). Wanting to be proactive, I did some investigation about what would be required from the library. In the AACSB 2010 standards and I found two references to the library: Services: Does the organization provide services (e.g., library, academic services, residence life, parking, maintenance of grounds and facilities, human resource management policies and services, information technology infrastructure, etc.) necessary to sustain the activities of the applicant academic unit? Are the costs of those services charged back to the applicant academic unit? (p. 7) A school with only campus-based undergraduate programs would normally be expected to provide sufficient classroom and computing facilities for students, student advising, faculty development including instructional enhancement, library and other information access, technology assistance, and support for faculty intellectual contributions. With the addition of master’s level programs would come expectations for applied research support. (p. 27) My next step was to see if I could find what other institutions had done in this regard, and if their reports were available online. One of the first references I found was a Pennsylvania State University Libraries study, published in the peer-reviewed journal Collection Building. The study sought to determine “if libraries which sustain accredited MBA programs possess a significantly greater number of selected titles from the Harvard Business School Core Collection of titles than those that do not sustain such accredited programs.” (McGuigan, Crawford, & Kubiske, 2004, p. 78). The methodology was meticulous in its detail and execution. Not surprisingly, the authors’ hypothesis was proved accurate, and the conclusion of the study upheld, that there is a “link between the strength of library collections and AACSB accreditation of MBA programs” (McGuigan et al., 2004, p. 81). I was disappointed in the extent of the study. To summarize the results of the study, the libraries of AACSB accredited schools have more books from the HBS Core List than do libraries from schools not accredited by the AACSB. What does this tell us about these libraries? In my opinion, not much. What about research support? What about electronic resources and federated searches? What about outreach and just-in-time support from qualified professionals? What about data curation and the availability of historical time series financial data? Is there an information commons available to undergrad students seeking support in numeracy and writing skills? Are there qualified IT support providers available to offer technical support? One would hope that the AACSB would be far more interested in these aspects of the library than in the number of business books they have. To be fair, this was a stand-alone study and was not intended to be a report used for AACSB accreditation; however, for those seeking guidance in the process, it supports the predilection to chase down information that reveals little substance. Thankfully, the other reports that I found during my search, such as those listed on the AACSB Resource Guide, presented by the Business Reference in Academic Libraries Committee (2001), were more fulsome and talked about the library as a whole. That being said, each one of the studies included a micro-collection analysis that included subject heading book counts. As I hope to demonstrate, this is no longer an effective measurement of the quality of a library’s collection. Progressive Collection Analysis is not Bean Counting Collection analysis means many different things. From usage statistics to impact factors to cost-per-use analysis, there are a myriad of tools available to help librarians understand Evidence Based Library and Information Practice 2010, 5.3 64 the strengths and weaknesses of their collections more fully. The only problem is, we seem to have the tools, but we aren’t exactly sure what to do with what they tell us. In a way, tools (such as those just mentioned) that are focused more on journal literature and databases are easier to use. They can help us determine what our users need in order to advance their research, and they can tell us what is being used frequently, and what is not being used at all. In tough budget times, these figures can help us determine priorities and assist us in defending our collections budgets through demonstration of community need. Ulrich’s Serials Analysis (http://www.ulrichsweb.com/ulrichsweb/anal ysis/default.asp) reveals which peer-reviewed publications are missing from your collection in a few easy steps. Unfortunately, when it comes to monograph collection analysis, the answers are not so straightforward. A literature review of methods used for monograph collections analysis yielded many articles about different ways to determine the strength of one’s collection; everything from the classic Brief Tests of Collection Strength (White, 1995) to the new and improved version called Better than Brief Tests (White, 2008). Hyödynmaa, Ahlholm-Kannisto, and Nurminen used a collection mapping technique in their study of Finnish libraries (2010), and OCLC’s WorldCat Collection Analysis (http://www.oclc.org/collectionanalysis/) offers another means of analysis. What seemed to be missing from this intensive study were definitive recommendations for action following a collection analysis. One exception is the article by Henry, Longstaff and Kampen (2008), which focused on outcomes. However, the reported outcome was a new system of collection management integrating faculty involvement with a more focused approach in selection. Rather than creating innovative solutions such as interlibrary loan/acquisitions partnerships for just-in-time acquisitions (i.e. purchasing on demand), this approach seems almost regressive rather than progressive. At Ryerson, we have found that while faculty involvement in collection development is always encouraged, it represents only a small percentage of acquisitions. Increased faculty involvement in selection would put our collection management practice back decades, not to mention the lost productivity gained through the use of sophisticated approval plans and streamlined acquisition workflows. In this era of just-in-time acquisitions, digitizing on demand, and new discovery technology such as Serials Solutions’ Summon (http://www.serialssolutions.com/summon/) and OCLC’s WorldCat Local, even if an analysis reveals that a physical collection is lacking in a particular subject area, does it make sense to spend scarce resources filling the shelves with titles that may possibly be needed by someone, sometime in the future? A recent study by Courant and Nielsen (2010) found that it costs $4.26 per year to maintain one book in open stacks in an academic library. Wholesale purchasing of physical volumes to improve a library collection doesn’t seem to be the smartest way to expend resources. That money could be invested in people and technology to improve digitization efforts, and to hire additional staff to more effectively deliver information on demand. Space would be saved to better equip libraries to build user-focused common spaces outfitted with services to better help them understand what information is available to them and how best to use it. The Association of College and Research Libraries (ACRL) encourages peer comparison for assessing library operations and the provision of library services. A close look at ACRL’s Standards for Libraries in Higher Education (2004) reveals that micro- comparisons, such as using subject headings to determine the number of books in one’s library catalogue compared to another, are not mentioned. Yet, this is often the first request that I get from liaison librarians. ACRL places higher value on comparing the distribution of resources by student and faculty FTE (full time equivalent) and departments within the library. The problem is that this information is much more difficult to obtain than volume http://www.oclc.org/collectionanalysis/ http://www.serialssolutions.com/summon/ Evidence Based Library and Information Practice 2010, 5.3 65 counts in various OPACs. There is still a great deal of fixation on subject-heading based volume counts as collection analysis, despite the fact that it is statistically flawed in many ways (e.g., variant cataloguing practices and different integrated library systems’ (ILS) display conventions). Furthermore, we rarely identify the overlap or omissions in our collections upon completion of these peer comparisons, thus negating any potential utility of the information gathered. WorldCat Collection Analysis (WCA) was meant to be the ideal alternative to this method. Although it has been available to libraries for several years, it is still in the fledgling stages of adoption and implementation, much to the chagrin of those who hoped to standardize its use. It would reveal what titles were missing from your collection, and would also tell you how current your collection was compared to your peers. It would accomplish this through sophisticated algorithms that would hold up to statistical scrutiny. My institution invested in WCA enthusiastically, only to discover that it was only as good as the cataloging standards and currency of the participating libraries available for comparison. WCA allows the generation of title lists for analysis on a one-to-one comparison, but individual permissions must be obtained from each library. We recently obtained permission for a one-to-one comparison from a peer library, only to discover that the institution in question was three years behind in its holdings updates, with no commitment from them as to when that update would occur. The program review due date had long passed by the time they had contacted us. Old fashioned subject heading volume counts were submitted as the collection analysis portion of the report instead. Not only are librarians lacking the proper tools to collect collection analysis data, we are unable to decide what to do with it, or how to act on the information once we get it, accurate or not. Rather than focusing our energies on describing what we can provide in services, electronic resources, and access, we spend an inordinate amount of time counting beans. A more literal interpretation of peer comparison, as per the ACRL standards, would produce a more useful and readable report. We don’t actually need to make more work for ourselves, when chances are the omission of this micro-analysis would not even be missed. How Should We be Analyzing Our Collections? Ideally, collection analysis should be an ongoing process to inform decision making at renewal time for electronic resources and serials, and during regular approval plan review. Approval plan revision in and of itself is action-oriented collection analysis, and should be treated as a routine task for subject librarians. This helps to keep librarians aware of not only what is in the collection, but also how (or if) it is being used. Many methods have been discussed in this commentary including White’s Brief Tests, WorldCat Collection Analysis, Ulrich’s Serials Analysis and various electronic resource usage statistics. The appropriateness of each of these methods depends on several factors. Good collection analysis should be action-oriented. If space is at a premium at your institution, and a digital-preferred strategy is in place, perhaps using WorldCat Collection Analysis to generate lists of titles not held in your library isn’t the best approach. You could still use WCA to demonstrate the extent of your holdings and how they compare to peer institutions, and then explain the processes in place to provide access to titles not held (i.e., interlibrary loan). You could further showcase the strength of your electronic resources and extol the virtue of 24/7 access. That being said, in the print-heavy world of the social sciences and humanities, perhaps you require more funding to expand your monographs budget. You could work with your vendor to generate a retrospective “wish list” of titles compiled, for example, from the output of a peer institution’s approval plan. This is far more useful than simply stating that another library Evidence Based Library and Information Practice 2010, 5.3 66 has more volumes in a particular subject heading, as is the case in so many library reports I’ve seen. In areas such as science, technology, and medicine, where the focus is on current journal literature and electronic access, you would be wise to highlight subscriptions to top tier journals and demonstrate value for investment in the library through usage statistics and access to the number of publications in which faculty have published. Furthermore, if you have an institutional repository (IR), you could campaign for an open access (OA) policy and encourage hosting OA journals locally, highlighting the benefit of increased exposure to publications loaded on the IR. The library has the power to heighten the reputation of the university with these services – an attention-grabbing (and budget friendly) feature, most definitely. The bottom line is that the number of books in a library does not indicate a “good” library. “What about good old-fashioned serendipity?” is a common refrain from traditionalists. Libraries are not retail outfits. We do not simply toss titles that are “not moving” from the shelves. There is something to be said for having information at your fingertips that you didn’t even know you needed until you did a keyword search in the catalogue. This portal to knowledge discovery has long been one of the library’s key functions. However, as it has been demonstrated, it is not very efficient or realistic to strive for comprehensive research collections in each and every library, nor is that the highest valued function expected of us from faculty and students. As Martin, Robles- Smith, Garrison, and Way (2009) astutely observed, the necessity of direct support for curriculum and the desire to maintain core collections to also support research requires “an approach to collection management that is highly elastic and sensitive even to minor program changes in course offerings and program structure” (p. 214). This means that libraries in comprehensive universities must be prepared to be nimble in service provision. Services include access to collections, but access does not have to equal ownership. Part of the problem lies in lack of consistency in the directives from various accreditation bodies and their reviewers. It is our responsibility to create guidelines and best practices for demonstrating library support. Continuing to invest time in poring over micro-collection analysis in library reports instead of focusing on what the library offers as a whole is a disservice to us as busy professionals and to the expectations of accreditation bodies. Libraries need to do better in sharing information as recommended in the ACRL standards (2004) for peer comparison. Transparency in resource allocation and a commitment to improving resource sharing needs to be a priority. In almost every other way, librarians have progressed far beyond being merely the “keepers of books.” If we, as collection managers, continue to perpetuate the notion that having more books is the best way to support programs, we will not advance alternative models of access. We should be focusing instead on more advanced discovery tools and on lobbying for more skilled staff to assist in facilitating the delivery of information. References Association to Advance Collegiate Schools of Business. (31 Jan. 2010). Eligibility procedures and accreditation standards for business accreditation. Retrieved 10 Aug. 2010, from http://www.aacsb.edu/accreditation/AA ACSB-STANDARDS-2010.pdf BRASS Business Reference in Academic Libraries Committee. (11 May 2001). AACSB Resource Guide. Retrieved from http://www.ala.org/ala/mgrps/divs/rusa/s ections/brass/brassprotools/aacsbresourc/ aacsbresource.cfm Association of College and Research Libraries. (2004 June). Standards for Libraries in Higher Education. Retrieved from http://www.ala.org/ala/mgrps/divs/acrl/st andards/standardslibraries.cfm Evidence Based Library and Information Practice 2010, 5.3 67 Courant, Paul N., & Nielsen, M. B. (June 2010). On the cost of keeping a book. In The idea of order: Transforming research collections for 21st century scholarship (pp. 81-105). Washington, DC: Council on Library and Information Resources. Retrieved 10 Aug. 2010 from http://www.clir.org Henry, E., Longstaff, R., & Van Kampen, D. (2008). Collection analysis outcomes in an academic library. Collection Building, 27(3), 113-117. Hyödynmaa, M., Ahlholm-Kannisto, A., & Nurminen, H. (2010). How to evaluate library collections: A case study of collection mapping. Collection Building, 29(2), 43-49. Kolowich, S. (7 Apr. 2010). Eroding library role? Insider Higher Ed. Retrieved 8 Aug. 2010 from http://www.insidehighered.com/news/20 10/04/07/survey Martin, H., Robles-Smith, K., Garrison, J., & Way, D. (2009). Methods and strategies for creating a culture of collections assessment at comprehensive universities. Journal of Electronic Resources Librarianship, 21(3/4), 213-236. McGuigan, G. S., Crawford, G. A., & Kubiske, J. L. (2004). Accreditation and library collections: The monographic holdings of academic libraries that support AACSB accredited and non-accredited MBA programs in the state of Pennsylvania, USA. Collection Building, 23(2), 78-86. White, H. D. (1995). Brief tests of collection strength: A methodology for all types of libraries. Westport, CN: Greenwood Press. White, H. D. (2008). Better than brief tests: Coverage power tests of collection strength. College & Research Libraries, 69(2), 155-174.