ACRL News Issue (B) of College & Research Libraries 336 / C&R L News RESEARCH FORUM C ollection assessm en t a n d v erifica tio n studies: Two reports from the R esearch Libraries Group S e v e r a l years ago, RLG institutions embarked on a program of collection evaluation studies: these have come to be known as “verification” studies. A collection “universe” (such as English literature) is defined and identified; a sample is draw n from this universe using a variety of techniques. The result­ ing titles then form the corpus of the study, provid­ ing a statistically sound instrum ent for measuring retrospective collection strength. The studies that RLG institutions have designed and undertaken have added markedly to RLG’s understanding of its members’ collections and have enabled member libraries to compare themselves against a common standard. They have given RLG a concrete picture of its range of collection strengths and of the p a t­ terns of collection overlaps in several, widely vary­ ing disciplines. The general library com m unity’s interest in col­ lection assessment techniques and verification studies suggests th a t the following brief reports may be welcome. The first, “The N ature and Uses of RLG Verification Studies,” looks at a few of the studies RLG has completed, and offers some con­ clusions about their uses and value. The final sec­ tion discusses the question of evaluating current collecting, and describes RLG’s solution, supple­ mental guidelines for large groups or sub-groups of th e C o n sp ectu s. T h e second, “ V e rific a tio n Studies—Design and Im plem entation,” was w rit­ ten to guide those in RLG developing collection as­ sessment tools. It argues th at clarity about assess­ m ent intention and standards is p aram o u n t in creating such tools, and offers a methodological framework for developers. W hile directed specifi­ cally at studies designed to verify Conspectus data values, it should nevertheless be easily adaptable for those not using the RLG C onspectus.—Jim Coleman, R LG . The nature and uses of the R LG verification studies By P a u l M osher Director o f Research Services Stanford University The RLG “verification studies” are comparative collection analyses, designed by small teams of ex­ pert bibliographers in member libraries, which al­ low comparative study of the strength of holdings of each member library in specific fields, or certain lines of a segment of the Conspectus. They perm it comparison of collection strengths and distribution of titles among reporting libraries. Let me amplify a bit on these two purposes: Verification of the comparability of collection value reporting to the Conspectus July /August 1985 / 337 Completion of the RLG Conspectus requires use of standardized codes to describe the collections of different libraries in specified Library of Congress classes. To do this it is necessary for the sake of com­ parability to assign a common, empirical content to the terms being used. Is a level 4 collection at one library comparable to a level 4 collection at an­ other library? In an attem pt to deal w ith this issue, the Collection M anagem ent and D evelopm ent Committee of RLG undertook certain collection assessment or verification projects intended to con­ firm the collection levels reported to the Conspec­ tus. They have been completed to date in a num ber of fields including English literature, French liter­ ature, Swiss history, food and agricultural eco­ nomics, art and architecture, m athem atical jour­ nals, and music. Other studies are in progress. RLG central staff has now produced consistent reports on each of these verification or overlap studies w hich will allow the Conspectus Task Force of CMDC to assess the comparability of re­ ports and make recommendations to ensure more standard assignment of collecting intensity codes for the RLG Conspectus, as well as clarifying un­ derstanding of the appropriateness of assigning pri­ m ary collection responsibilities within the consor­ tium. The studies also describe the distribution of the literature of a subject, and unique titles, among member libraries. This helps to demonstrate p a t­ terns of little-held materials among the mem ber­ ship, and the relative and comparative strengths of collections at member institutions. The RLG series of verification and overlap studies differs from other studies of this type in representing groups of major national research libraries scattered across the country rather than smaller groups of libraries in state or regional consortia. Examples of findings of verification studies A study carried out in English literature from the Cambridge Bibliography o f English Literature produced some interesting results. Four test li­ braries carried out the initial verification study. The results of this study suggested that values re­ ported to the Conspectus were on the whole cor­ rect. Interest in the findings and their significance, both consortial and local, prom pted eleven other RLG institutions to replicate the study. This first verification study proved, not surprisingly, that English literature is an area of considerable con­ centration for all RLG libraries. There is a fairly high rate of absolute holdings and collection over­ lap. Of the four libraries with the most extensive holdings, the strongest (Cornell) held 88% , while the top four together held 97 % . Library holdings of the many smaller RLG libraries, however, dem ­ onstrate not only lower overlap rates than among these very large collections, but also a smaller pro­ portion of titles. Nevertheless, the pooled total holdings of these smaller libraries still result in a surprisingly large proportion of the total sample. Among these smaller libraries, while one library held 51 % of the sample, four hold 85% , a gain of over thirty percentage points. This large increase in coverage among the combined holdings of smaller RLG libraries underscores the advantages of re­ source pooling to an even greater degree th an among large institutions. These studies underscore the benefits of resource sharing to scholars. The area of Renaissance and Baroque art history is obviously a more specialized subject. W hat did the overlap study reveal in this instance? Of twenty reporting libraries, Yale and Rerkeley each hold slightly over 72%. The top four holding libraries, including the Library of Congress, which is not a member of RLG, hold 91 % of the total. The bot­ tom four reporting libraries hold only 49.4 % of the total (the smallest of these holds but 18%, so the gain is substantial). Thus it is clear that a few libraries in the consor­ tium hold by far the greater num ber of titles, and th at the combined collections represent for most members very much stronger holdings than any one, or even a combination, of all but the few li­ braries with the greatest concentration of holdings. The composite holdings of these major research libraries are greater than the holdings of local, re­ gional, or multitype consortia, and richer in m ate­ rials im portant to research. Indeed, the combined holdings of the major national research libraries represent collectively the strongest research re­ source collection the world has ever known. The results of the studies to date underscore and demon­ strate the benefits of resource sharing to scholar­ ship. They also suggest th at a num ber of options are available to developers interested in collaborating on ways to enhance the use of resources for both en­ riching the base of seldom used research materials, and for reducing, as local policies may dictate, ar­ eas of unnecessary redundancy among participat­ ing libraries. Supplemental guidelines to the segments of the R LG Conspectus Experiences of RLG and ARL test libraries in completing the Conspectus had demonstrated the need for prospective measures to guide librarians in reporting values to the Conspectus. This has led to the preparation of w hat are called “supplemental 338 / C&RL News guidelines” for each appropriate segment or sub- segment of the Conspectus. These supplem ental guidelines are prepared by small committees of ex­ pert bibliographers, and consist of appropriate p er­ centages of holdings for the subject from certain s ta n d a rd reference, b ib lio g ra p h ic , or cita tio n guides to the literature of a specific field. For example, the supplem ental guidelines to the N atural History and Biology Conspectus contain a brief description of the n atu re of library literature supporting biology, and the suggestion of certain standard guides and periodical indexes to the liter­ ature. The definitions of levels 2, 3, 4, and 5 are then expanded to include certain proportions of holdings from these indexes or guides th a t should be held at each subsequent level. For the basic inform ation level 2, for example, a few indexes, such as the Biological and Agricul­ tural Index, and a selection of general periodicals, including 30 % or more of the biological titles in the “Periodicals Indexed” section of Biological and A g ­ ricultural Index, are called for. At instructional support level 3, the collection should include a w id e ra n g e of m o n o g ra p h s , access to non- bibliographic databases, and general texts. This level should also include the m ajor indexing and abstracting services in the field, and a wide range of basic serials, including 90% or more of the titles pertinent to the subject described in the “Periodi­ cals Indexed” section of the Biological and Agricul­ tural Index. Each successive level of intensity is similarly outlined in term s of holdings. These sup­ plem ental guidelines allow a reporting library to test reported values against some generally ac­ cepted standards, and m ay also provide the basis for useful com parative overlap studies to examine the distribution of the literature w ithin RLG li­ braries. Verification studies: Design and implementation By Jim C olem an Program Assistant Research Libraries Group, Inc. Study design V erification study proposals should begin by clearly defining the purpose and goals of the instru­ m ent to be developed. At its most basic, such a statem ent will include: 1) the scope of the topic to be investigated (such as “French L iterature since 1789”); 2) the Conspectus subject lines associated w ith the topic (if appropriate); 3) the purpose of the study (e.g., to test coverage of the field among institutions, to examine relative strength of collec­ tions at certain levels, to set a dividing line between collection levels); and 4) the m ethod to be em ­ ployed to attain these goals. Previous studies at RLG have focused their ef­ forts on testing 1) absolute strengths of existing col­ lections (as m easured against the percentage of to­ tal holdings of the sample universe); 2) relative strengths of existing collections (study participants m easured against each o ther); 3) absolute and rela­ tive strengths of collections w ithin such subgroup­ ings as form at and collection type (e.g ., “basic” or “research” collections); and 4) m easuring collec­ tion sizes (shelf list counts). Verification studies testing collection coverage, absolute and relative strengths, and overlap are interested in describing and comparing retrospective populations, while popu latio n size m easurem ents are interested in quantifying these populations. In designing verifi­ cation studies, it is im portant to rem em ber th a t dif­ ferent goals and purposes will call for different in­ strum ents and sam pling techniques. Studies testing holdings and the associated C on­ spectus lines descriptively require particular a tten ­ tion to strategies th a t will support classification of sections (or titles) by type, e.g. “basic,” “inform a­ tional,” “research.” T hat is, a verification study will be most helpful if its structure helps uncover the different collection characteristics associated w ith collection levels as defined w ithin the C on­ spectus. This m ay, for example, be accomplished by devising sections verifying “core” m aterials. Studies th a t are interested in fine differentiations w ithin levels, e.g. relative strengths of research level collections, can be of use in specialized situa­ tions, but are not of greatest need at present. W here the potential range or size of a population is unknown or seriously in doubt, procedures to quantify will be useful as a measure of the size of the total universe, and the range w ithin w hich in­ stitutional holdings m ay fall. However, shelf list measurements generally present greater m ethod­ ological difficulties w hen used for Conspectus veri­ fications. Simple shelf list m easurem ents based strictly on call num ber ranges are generally in ap ­ propriate for testing Conspectus values, although they may be of use in assessing special form ats, such as journals or m ultiple editions, or in subject fields overwhelmingly dom inated by a single form at. July /August 1985 / 339 Descriptive studies: Retrospective materials General methodology for descriptive verifica­ tion studies involves defining the universe of titles to be tested w ithin the specified scope of the study and then deriving a sample from this universe. For retrospective m aterials, this generally means iden­ tifying the bibliographic sources whose range is closest to th at of the desired universe. Studies strati­ fied according to collection type will almost always require employing m ultiple sources, using one or more sources for each stratified section. W hile it may not always be possible to be certain th at these sources both define and exhaust the universe actu­ ally, the closer one approaches such exhaustiveness the more accurately the study will reflect the con­ stitution of institutional holdings. This is the point in the study design to resolve particularly difficult questions (see, for example, the comments under the “Editions” p art of section I I ) . If the subject area or sources are too narrowly defined, it is likely th at no discernible differences am ong institutions, other than fluctuations in percentage of total, will emerge. If d raw n too widely, collection w eak­ nesses or differences may be indicated where none, in fact, exist. In both instances, the results may heighten the incredulity w ith which “scientific” measurements of collections are often met, and vi­ tiate the results of the exercise. W hen selecting bibliographic sources, consider­ ation must be given to their strengths, weaknesses, and age. Supplem ental sources may be necessary to overcome any difficulties with the m ain sources th at would adversely affect the study. If appropri­ ate, the study can include a discussion of the lim ita­ tions or potential drawbacks of the design. Descriptive studies: C urrent collecting V e rific a tio n of c u rre n t co llectin g activ ities should rely, wherever possible, on existing RLG supplem ental guidelines. W hile these sources need not in every case comprise the complete sample universe, they should be used as the basis. Addi­ tional sources may be used as appropriate. Additional considerations W hen using several bibliographic sources, cita­ tion overlaps are to be expected. A description of the methods used, if any, to eliminate these over­ laps should be included. The proposal should give the estimated or exact total of the universe from which the sample is being draw n; the num ber of ti­ tles to be draw n for the sample; and a description of the sampling techniques employed. For most p u r­ poses, sample sizes ranging from 300 to 600 titles will be adequate. Only those studies covering very broad subject areas or stratifying sections over sev­ eral subject areas should consider samples in excess of 600 titles. Study implementation The accuracy and confidence we can invest in any verification study is a direct result of our cer­ tainty th at like things are being tested. The im ple­ m entation of a verification study goes hand in hand with its conceptual design: it is therefore im portant to consider at the outset w hat circumstances will lead to a participating library counting a title as a “h it.” Simple shelfl ist measurements are inappropriate. Four areas have given rise to the most frequent questions for past studies: journals, editions, alter­ nate formats, and alternate languages. These com­ ments are presented as suggestions for handling the most usual conditions. Procedures th at vary sub­ stantially from the guidelines presented below should be used only for compelling reasons. No m atter w hat principles or procedures are used to determ ine sample size and hit determ ination, resist all tem ptations to deviate from them for the sake of expediency or convenience. Journals. Depending on the universe designation methods, journals will generally be considered ei­ ther as individual items or as runs. As individual ti­ tles, they should be counted as hits only if individu­ ally held. As runs, levels of acceptable holding should be indicated, e.g., 65% of total titles. In such instances, also providing the total num ber of titles expected will aid p a rtic ip a n ts. T he most stringent demands would require a current sub­ scription and retrospective holdings of at least 75%. Editions. In general, acceptable editions should be identified before sampling takes place. If only certain editions lie w ithin the universe under con­ sideration, all others should be elim inated. If the range of editions is considered of im portance, then no editions should be excluded. In either case, a hit is attained only when an edition is m atched ex­ actly, or it can be determ ined th at another edition contains m aterial identical to th at of the edition re­ quested. A less stringent procedure would offer a choice of editions. The least stringent (and least preferable) would claim any edition as a hit. A lternate form ats. R eprints and m icroforms count as hits if they are identical to the requested m aterial. Alternate languages. As in the case of editions, language considerations should be resolved before 340 / C&RL News sampling. Alternate languages (translations) are generally not counted as hits. Wherever the title to be checked, however, is itself a translation, the original material should be counted unless there are reasons that dictate otherwise (philological ap­ paratus, etc.). Citations should follow Library of Congress in most instances, and partial names or initials should be avoided. Citations should be verified before the study is distributed generally. Lastly, study designers should include a prelim i­ nary reporting form in their proposal, and indicate the length of time they estimate necessary to com­ plete the study. ■ ■ C o m p u terized a ccess to a c h a p b o o k c o lle c tio n By Judith E. E ndelm an Assistant Librarian, Lilly Library Indiana University and D ia n e K. B au erle Technical Services Assistant, Lilly Library Indiana University How one library uses a micro to catalog ephemera. I n 1983 Lilly Library, the rare book library of In- diana University, received the Elisabeth Ball col­ lection of children’s literature. This collection, considered by many to be the finest collection of children’s literature still in private hands, was be­ gun by Elisabeth Ball’s father, George Ball, one of the five Ball brothers of Muncie, Indiana. Over many years, father and then daughter devoted a great deal of care and attention to building up the collection. Following Elisabeth Ball’s death in 1982, the books and manuscripts were left to the George and Frances Ball Foundation. After weigh­ ing many factors, the foundation selected the Lilly Library to be the recipient of the Ball library. Of the 14,000 books and approximately 1,000 m anu­ scripts which the library received, approximately tw o-thirds were books for children. One of the Balls’ collecting interests was chapbook literature, and their collection of two thousand chapbooks is among the largest collections of its kind. While many of the chapbooks they collected were w ritten for children, a large num ber were clearly intended for an adult audience. Most of the Ball collection chapbooks are English with a good representation of American imprints, and a few French titles. The word chapbook is a 19th-century term used to describe the popular literature formerly circu­ lated by peddlers, hawkers, or chapmen (hence the source of the term) consisting chiefly of small pam ­ phlets of p o p u lar tales, ballads, tracts, and so forth. Chapbooks were small and easily transport­ able, usually consisting of eight, sixteen, or thirty- two pages. They were generally cheaply printed w ith bad type and w orn, crude woodcuts. The badly draw n picture of the knight on horseback that adorned the cover of the tale of Guy, Earl of W arw ick (see cover) m ight also appear on the cover of Sleeping Beauty or Cinderella. Their price was low—usually between one and five pennies. Typical subjects included legends, ballads, fairy tales, dream interpretation, or sensational tales of crimes and criminals. By the mid-19th century,