ACRL News Issue (B) of College & Research Libraries September 1991 / 485 Maintaining a high-quality, cost- effective journal collection B y Dawn B ick Director, Collection Development Houston Academy o f Medicine- Texas Medical Center Library and R e e ta Sinha Collection Development Assistant Houston Academy o f Medicine- Texas M edical C enter Library Use this practical technique to make your serial retention decisions. R ising journal subscription costs and the decreasing purchasing power of serials budgets make it imperative for collection to spend their dollars as wisely as possible. A cost- benefit ratio for each journal title can be deter­ mined on the basis of use and other critical reten­ tion factors. At the Houston Academy of Medicine- Texas Medical Center Library we have determined that this ratio effectively supports retention and cancellation decisions. A cost-benefit ratio f o r each journal title can be determined on the basis o f use and other critical retention factors. Those of us who have had the experience of cancelling journal titles realize the importance of having use data at hand in order to make sound cancellation or retention decisions. Ironically, use data are commonly not available for technical or logistical reasons, or both. Even if use data are available, they alone cannot be the basis of a cancel­ lation or retention decision. Suppose five titles show relatively little use, and each supports a pro­ gram or area of research in your institution. I f you could afford to retain only two of them, which ones would they be? How would you justify your deci­ sions? Raw use data do not expedite decisions in this m case. But, if the use data were to be weighted by factoring in other journal worth criteria such as ainnadgeexrisn g, local availability, local contributors, im­ pact factor etc., then one could compute a cost- benefit ratio for each title. This ratio would give the support needed for cancellation or retention deci­ sions. SERIALS D atab ase In order to locate in one central area the data essential for providing support for journal deci­ sions, a database of all active serial titles was devel­ oped. The SERIALS database lists all currently received journal and monographic serial titles and is updated on an ongoing basis as titles are added to the collection and/or cancelled. The initial list of titles and ISSNs was transferred in ASCII format from the library’s serials control system to a data­ base management system. The DBMS in current use is Paradox, and the SERIALS database now contains some 3,000 journal titles. The data fields that have been utilized to store information about each title include identifi­ ers and cost, as well as the criteria that are used to evaluate the journals’ worth. The data were gath­ ered from various sources, compiled on workforms, and then entered into the database. Unfortunately, most of the data were not available electronically such that ASCII files could be loaded into the database; these data had to be keyed in. The initial keying was labor-intensive, but updating has been smooth and efficient, as it has been integrated with 486 / C&RL News the workflows of the Serials and Collection Devel­ opment Departments. O n goin g jo u rn a l use study Since August o f 1989, the library has been study­ ing in-house journal use. The in-house use data in combination with checkout data are central to the Library’s journal evaluation decision support sys­ tem. The objective of the study is to monitor in-house uses of journal titles over consecutive one-year periods. Since no previous data were available, it was acknowledged that the ideal study design would be one in which in-house use statistics would be collected for every hour that the library was opera­ tional. The reality is that the resulting workload is unmanageable and impractical. Thus, the current survey is a descriptive study designed to collect in- house use data for bound and unbound journal issues from a representative sample of the Library’s operational hours over consecutive 12-month peri­ ods. These periods correspond to the library’s fiscal year, Septmber 1 to August 31. The study population consists of a stratified ran­ dom sample of one-hour study segments drawn from the total numbers of hours the library’s in operation during a given month. Each month, the stratified random sample is obtained by arbitrarily dividing the population (i.e. ‚ total number of opera­ tional hours) into strata designated as “morning hours,” “afternoon hours,” and “evening hours” and numbering all the hours in each. For example, there may be 122 “morning hours” in the month. Then a random sample o f the numbered one-hour study segments is drawn from each stratum. This methodology reduces the sample variances so the population estimates obtained are more precise. The required sample sizes are calculated each month in order to obtain accurate estimates of population statistics, such as average number of journal uses. In general, choosing a sample size requires (1) knowledge o f population variances based on previously collected data and (2) defining limits for the tolerated level of error.1 For this study, estimates should be accurate within one journal use and, since no previous data were available, popula­ tion variances are estimated using the range of uses possible for each stratum. For the duration of the study, monthly samples have represented 8%—10% of total operational hours per month. The data collection procedure involves a pick-up of all unshelved journals prior to start o f each one- hour sampling period. Then, journals used by the library’s clients are allowed to accumulate during the period and are collected at the end of the hour. The bar codes of these collected joumals are wanded into the library’s automated system prior to reshelving. Thus, in-house uses by journal title are tabulated by the library’s automated system. The fact that participation in the journal use study continues to be a positive experience for the shelving staff helps to ensure the reliability of the data collected. The shelvers have been able to incorporate the data collection procedure into their daily routine and look forward to the study seg­ ments. It has been a morale booster for them to be taking part in an important library project, and they have welcomed the opportunity to receive hands- on experience using the library’s automated system. From the investigator’s point o f view, it has also been encouraging to learn that the shelvers have been meticulous about collecting data, to a point beyond the general instructions given. Fig u re 1 MATRIX FO R JOURNAL EVALUATION CR ITER IA September 1991 / 487 T able 1: JOURNAL WORTH CRITERIA FO R TH E HOUSTON ACADEMY O F MEDICINE-TEXAS MEDICAL CEN TER LIBRARY Weight Factor Weight Normalized/100 Indexing I (LSI‚HLI‚CINAIIL) 37.0 11.5 Core Subject 37.0 11.5 Indexing II (BIO,SCI,CC,PSYCH) 32.0 10.0 Medical Center Holdings – no 24.0 7.5 Significant Impact Factor 23.5 7.3 Libraiy - only copy 22.6 7.0 Library Holdings - long 20.8 6.5 Indexing I II (CHEM ,EM ,OTHER) 19.3 6.0 Regional Holdings - no 17.3 5.4 Non-Core Subject 16.8 5.2 Houston-Area Holdings - no 16.0 5.0 Full-Text Access - no 12.6 3.9 Library Holdings - short 8.6 2.7 Indexing IV (not indexed) 7.0 2.2 Medical Center Holdings - yes 6.0 1.9 Houston-Area Holdings - yes 5.8 1.8 Insignificant Impact Factor 5.5 1.7 Regional Holdings - yes 5.0 1.6 Full Text Access - yes 4.0 1.2 Library - duplicate copy .5 .2 The library’s automated system compiles monthly statistics generating a list of journal titles with associated in-house and checkout use data. The use statistics are then stored in a data field in the SERIALS database so that usage can easily be factored in with the other evaluation criteria for individual journal titles. D eveloping jo u rn a l evaluation c rite ria The basic criteria used to determine the worth of any journal title were weighted and ranked in order of importance using an evaluation matrix described by Mudge.2 The advantage of using the matrix approach is that the weighting of journal evaluation criteria is an objective and relative process rather than an arbitrary one. These weights apply specifi­ cally to the institution in which they were deter­ mined and they necessarily reflect collection devel - opment policy priorities. It is anticipated that these weights would change only as the collection devel­ opment policy changes. The evaluation matrix is designed so that only two criteria are compared at a time. For each pair of criteria, first determine which is the more impor­ tant for retention of a journal title and then indicate whether the difference in importance is major, medium, or minor— assigning the number 3 for a major difference in importance, 2 for a medium difference, and 1 for a minor difference. For each comparison, two questions were asked: “Which is the factor that is more likely to make you want to retain a title?” and “Is the difference in importance major, medium, or minor?” A major difference in importance is obvious and the decision is almost instantaneous. A medium difference in importance requires a relatively short period of time for a decision. A minor difference in importance takes a considerable amount of time for a decision; but you are forced to decide. For example, Figure 1 shows part o f the matrix we used. The letters listed across the top of the matrix correspond to the underlined criteria listed down its side. To start off, factor A, “Core Subject,” was compared to B, “Non-Core Subject,” at the top of the matrix. F actor A was considered to be more important in retention of a title than B, so A was entered into the first cell. The number 3 was entered beside it indicating a major difference in importance. Working across the first row of cells, A was then compared to C, “Significant Impact Fac­ tor,” and then to D, etc., until it had been compared to all of the criteria listed across the top of the matrix. Likewise, B in the next row down was compared to C, D, E, etc., in turn. The process continued until each factor was compared to all of the other criteria. After the matrix is complete, a score for each row o f criteria is obtained by sum- ming the numbers indicating importance (1,2, or3). For example, in Figure 1 ‚ all of the As were totalled 488 / C&RL News to get a score of 43, all o f the Bs gave a total of 25, etc. For our purposes, the matrix was completed individually by six members of a serials committee. Their numerical scores for each factor were aver­ aged to obtain the final weight used. The average weight for factor A was 37, the average weight for B was 16.8, etc. These weights were normalized to a base of 100 (see Table 1). The normalization was accomplished by totalling all of the criteria weights (total weights = 321.3) and then using this total in the formula: Table 2: PARADOX W O RKSH EET FO R SERIA L T IT L E S N um erical w eight fa c to rs a re listed to the left o f ea ch facto r. The numerical f a c t o r weights pertaining to a particular title a re a d d ed together to obtain a Jo u rn al Score. The scores f o r ea ch title a re com pu ted and stored in the SERIALS database. September 1991 / 489 T able 3: CHEMISTRY JOURNALS RANKED IN ORDER OF DESIRABILITY FOR THE HOUSTON ACADEMY O F MEDICINE-TEXAS MEDICAL CEN TER LIBRARY Cost-Benefit Title Ratio 1. Chemical & Engineering News .82 2. Journal of the American Chemical Society 2.18 3. Journal of Chemical Information & Computer Sciences 5.65 4. Magnesium 6.21 5. Journal of Heterocyclic Chemistiy 6.73 6. Journal of Physical Chemistry 7.09 7. Accounts of Chemical Research 7.37 8. Journal of Colloid & Interface Science 7.86 9. Biopolymers 8.79 10. Angewandte Chemie 9.01 11. Journal of Chemical Physics 10.67 12. Macromolecules 14.71 13. Journal of the Chemical Society. Faraday Transactions 16.85 14. Inorganic Chemistry 19.39 15. Journal of the Chemical Society. Perkins Transactions II 20.86 16. Biological Trace Element Research 22.09 17. Chemical Senses 26.46 18. Journal of the Chemical Society Dalton Transactions 28.09 19. The Analyst 29.99 20. Acta Chemica Scandinavica. Series A. Physical & Inorganic Chemistry 41.36 21. Journal of the Chemical Society. Chemical Communications 173.73 D eterm inin g th e jo u rn al sco re Once the evaluation criteria weights have been established, they can be applied to individual jour­ nal titles to determine a Journal Score for each. In the example shown in Table 2, the applicable nu­ merical weights are listed to the left of each evalu­ ation factor. For example, since the Library has relatively long holdings for the Jo u r n a l o f C o llo id & I n te r fa c e S cien ce, the weight of 6.5 has been assigned using the evaluation factor "Library Hold­ ings—long” from Table 1. Similarly, since the title is indexed by the National Library of Medicine’s List o f Serials Indexed f o r Online Users, Biological Abstracts, Chemical Abstracts, and Excerpta Medica, the established weights for the evaluation criteria “Indexing I ” (11.5), “Indexing II” (10.0), and “In­ dexing I I I ” (6.0) from Table 1 have been assigned. Not all of the weights listed in T able lwillbeused when evaluating individual journal titles. By nature, some pairs of evaluation criteria are mutually exclu­ sive. For example, in the use of the criteria “Core Subject” and “Non-Core Subject” from Table 1, only one of these factors will apply for any given title since the subject of a title cannot be both core and non-core. In our example in Table 2, the subject “Chemistry” is considered to be non-core for our Library; therefore, the weight of 5.2 has been assigned. Similarily, had the title only been indexed by Excerpta Medica, “Indexing III” from Table 1 would have been the only applicable evaluation factor for indexing. Therefore, only the weight of 6.0 would have been assigned. Once all of the evaluation criteria have been assigned their respective weights, a Journal Score for each title can be determined by simply adding the weights. For our library, the scores for each title are computed and stored in the SERIALS database. D ecision support To obtain the perceived benefit of a journal the Journal Score is multiplied by Use: B EN E FIT = Use x Journal Score. According to this formula, since the score of a journal stays the same, Benefit increases as Use increases. In the example given in Table 2, the Inhouse use for Year 2 (17) was multiplied by the Journal Score (64.4) to obtain a Benefit of 1094.8. Next, the formula for determining the cost-ben- efit ratio of a journal was developed. The annual subscription Cost is divided by Benefit and then multiplied by 10 just to make the numbers more manageable. 490 / C &RL News C O ST -B E N E F IT ratio = (COST/BENEFIT) x 10 Again, taking the example in Table 2, the Cost for Year 2 ($861.00) was divided by the journal’s Ben­ efit (1094.8) and multiplied by 10 to obtain the rounded off Cost-Benefìt Ratio of 7.86. This resulting ratio reflects the desirability of a journal. A journal with a high Benefit value is more desirable. Therefore, as Benefit increases, accord­ ing to the formula, the cost-benefit ratio decreases. In other words, journal desirability increases as the cost-benefit ratio decreases. Table 3 shows a list of selected journals ranked in order o f present desir­ ability in our library. C o n clu sion Having the ability to produce, on demand, a list o f the library’s current journal titles ranked in order o f desirability is a major step toward maintaining a high-quality journal collection which meets the needs of the clientele. Should the library have to cancel titles, faculty and other interested parties can be supplied with lists of journal titles, sorted by subject and ranked by desirability. Decisions can be supported in an objective way. Other collection management concerns, including purchasing du­ plicate subscriptions, remote storage, and preser­ vation decisions, can also be supported ■ ■ 1George W. Snedecor, and William G. Cochran, Statistical M ethods, 7th ed. (Ames, Iowa: Iowa State University Press, 1980), 53. 2Arthur E. Mudge, Value Engineering: A Sys­ tem atic A pproach (NewYork: McGraw-Hill, 1971), 6 8-74. Mudge, one of the pioneers in value engineering, describes a systematic approach for comparative analysis o f a product. This approach includes using an evaluation matrix which compares the relative importance of the basic functions of each product being studied. We have used the evaluation matrix technique to compare the relative importance of basic worth criteria for the journals being studied.