lib-s-mocs-kmc364-20140601051521 - 39 SELECTIVE DISSEMINATION OF MARC: A USER EVALUATION Lorne R. BUHR: Murray Memorial Library, University of Saskatchewan, Saskatoon, Saskatchewan After outlining the terms of reference of an investigation of user reaction to the selective dissemination of MARC records, a summary of the types of users is given. User response is analyzed and interpreted in the light of recent developments at the Library of Congress. Implications for the future of SDI of MARC in a university setting conclude the paper. INTRODUCTION F. W. Lancaster ( 1968) in his detailed study of MEDLARS makes the following statement, which has application to all SDI work: "In order to survive, a system must monitor itself, evaluate its performance, and upgrade it wherever possible." ( 1) Since SELDOM operates in a fairly new field, SDI for current monographs, an evaluation is most important. To a great extent it must be made without reference to other systems since most of the operational SDI services deal with tape services in various fields of scientific journals, and although there are some parallels, there are numerous dif- ferences. Whereas services such as Can/ SDI cater primarily to the natural and applied sciences, SELDOM opens up the possibilities for SDI in the humanities and social sciences. The background to the SELDOM Project at the University of Saskat- chewan has been outlined earlier by Smith and Mauer hoff ( 1971) and will not be repeated here. (2) After five months of operation a major question- naire was sent out to each of 121 participants in the experimental SELDOM service. This questionnaire was based almost entirely on the one used by Studer ( 1968 ) in his dissertation at Indiana State University. (3) The general purpose of the study was to elicit user reaction to SELDOM, their eval- Material appearing in this paper was originally presented at the third annual meeting of the American Society for Information Science (Western Canada Chapter), Banff, Alberta, October 4, 1971. 40 Journal of Library Automation Vol. 5/1 March, 1972 uation of its usefulness, time necessary to scan the weekly output, sugges- tions regarding continuance of the service, etc. Besides this general purpose, the gathering and analyzing of data on SELDOM will be useful to the library administration in determining the future of an SDI service of this nature. A separate cost study is being prepared in this connection. Several factors prompt a cautionary stance in assessing the value of an SDI system on the basis of one questionnaire: ( 1) There is no control situation to which we can compare SELDOM, i.e., there was no systematic service for current awareness in the field prior to the advent of SELDOM. Faculty and researchers were dependent on their ingenuity to ferret out information on new books which were pertinent to their field of research and instruction. SELDOM is therefore being compared to a conglomera- tion of ad hoc methods which may be as numerous as the individuals using them. Therefore, we must be cautious or we will tend to say, "Something in the field of current awareness is better than nothing," when we really do not know what that "nothing" is. (2) Although SELDOM had been operational for some twenty weeks when evaluation began, this is a rela- tively short period on which to base an assessment. On the other hand Studer's evaluation was based on the experiences of thirty-nine users and covered only eight weekly runs against the MARC tapes scheduled on an every other week basis. (3) SELDOM was implemented without any study to determine the adequacy of the ad hoc approaches, to which I have already referred, nor to assess the patterns of recommendation for purchase. It was assumed that there was a need for SELDOM and some of the response would indicate that this is a fairly valid assumption, since almost 90 percent of the respondents wanted the service continued. A random investigation in mid-August of 748 current orders in the acquisi- tions department for books with imprint of 1969 or later revealed that ninty-five or 12~ percent referred to SELDOM as the source of information for a particular recommendation to purchase. This may or may not be significant since there is no way of assessing whether these items would have been recommended anyway, only later perhaps. One by-product of orders based on SELDOM information is that correct LC and ISBN numbers are given and with the capabilities of the TESA-1 cataloging/ acquisitions system such orders can be expedited more quickly and can also be cataloged sooner than non-MARC materials, thus ostensibly getting the desired item to the requestor in less time than previously. SELDOM is valuable in our university setting, therefore, not only as a means of awareness of new items, but also in the actual retrieval of the item for the user, in this case through acquisition. Our analysis, however, must be directed to the effectiveness of SELDOM as an awareness service, vis-a-vis the ad hoc approach. USER GROUP Of 121 questionnaires sent out, seventy-seven or 63.5 percent were re- turned. Six of these had to be rejected for the purposes of this study since Selective Dissemination of MARC /BUHR 41 Table 1 I. Library and Information Science A. On-campus 12 B. Off-campus 17 29 II. Social Sciences and Humanities A. On-campus 15 B. Off-campus 2 17 III. Natural and Applied Sciences A. On-campus 23 B. Off-campus 2 25 either only a few questions had been answered or a general letter had been sent instead of answering the questionnaire. Thus, the data presented in this study will be based on seventy-one completed questionnaires or 58.6 percent return. Three additional verbal comments were made to the writer and thus we in fact heard from eighty or 66 percent of the users. The term "users" will designate the seventy-one who completed their question- naires, although comments from the other nine individuals will also be referred to. The users have been grouped into three categories according to Table 1. Categorization was along fairly traditional lines, with category I being necessary because of the large number of people falling into this area. The seventeen off-campus users coming under designation (I) represent the library schools in Canada as well as librarians/ information scientists in Canada and the United States. The on-campus users are library depart- ment heads and heads of branch libraries. - Included in the social sciences and humanities are the fields of psy- chology, sociology, history, economics, English, commerce, classics, etc. The natural and applied sciences include all the health sciences plus physical education since the two profiles in that area are tending toward the health sciences. Engineering, poultry science, physics, chemistry, bi- ology, etc. , are represented here. OBSERVATIONS A sample of the questionnaire used appears on p.47-50 and includes a tally of the number of responses for each possible alternative answer to each question. In some cases the total number of replies for a question is less than seventy-one. This is explained by the fact that some questions on some questionnaires were not answered or were answered ambiguously so they could not be tallied. Generally speaking, users found SELDOM to be good to very good in providing SDI for new English monographs. 25.8 percent of the users found the lists very useful while 48.5 percent said they were useful. Six users said the listings were inconsequential for their purposes; in several 42 Journal of Library Automation Vol. 5/1 March, 1972 instances this may be due to poor profiling or profiling for a subject area in which little would appear on the MARC data base. 23.6 percent of the users indicated that in most cases items of interest found on the SELDOM lists were previously not known to them. 45.8 percent said that "of interest" items were frequently new. 76 percent of the group believed that the proportion of "of interest" items which also were new was satisfactory, a percentage which speaks well for the currency and effectiveness of an SDI capability. One of the chief drawbacks for which SDI services are often cited is the absence of evaluative commentary or abstract material to accompany the citations. Some tape services do provide either an abstract or a good number of descriptors, and this has proved to be an asset in helping the subscriber. SELDOM is based on the MARC tapes which provide complete cataloging data but do not give either evaluations or a multiplicity of descriptors. (Some indications are that the information now available in Publishers' Weekly might at some time in the future be added to the MARC tapes. ) Interestingly enough, 83.5 percent of the users said the information included in the entries was adequate to determine whether an item was of interest or not. Predictably, title, author/ editor, and subject headings were the three indicators, in that order, which were found most useful in making evaluations. This is significant since titles in the humani- ties and some of the social sciences, particularly, are often not as specific in describing the contents of a work as are titles in the physical sciences. 63.5 percent of the users indicate that SELDOM information is used for recommending titles for acquisition by the library. As a result it is quite possible that purchasing in the areas covered by SELDOM profiles may increase and the tendency to broaden the collection should increase. Unfortunately, no pattern of pre-SELDOM recommending for purchase is known. Some instructors use the weekly printouts to keep current biblio- graphies on hand both for teaching purposes and for research purposes. Since over half the users ( 55.8 percent) needed no more than ten minutes per week to scan the printouts, there is no indication that excessive time is taken up in the use of such an SDI service. In reply to the question, "Would you be willing to increase the number of irrelevant notices received in order to maximize the number of relevant ones?" opinions were nearly balanced with 58 percent replying in the affirmative and 42 percent answering negatively. On the other hand, increases in the MARC data base expected some time in 1972 when other Roman alphabet language imprints and records for motion pictures and filmstrips are added, did not seem problematic with only 25 percent of users asking that an upper limit be placed on the quantity of material retrieved by their profiles. Numerous individuals (thirty ) responded fav- orably to the prospect of wider language coverage by MARC. On the other hand, several individuals commented that non-English output on SELDOM would not enhance the service for them, and this likely reflects languagt Selective Dissemination of MARC /BUHR 43 capabilities more than a lack of non-English material in their subject area. The question regarding format brought interesting comments, especially from library personnel and off-campus librarians: "Computer type format is often confusing." "A book designer should be consulted to improve the format." "Spacing could be improved to separate title and imprint infor- mation from subject headings and notes at foot of entry. Would make scanning easier." Questions fourteen, nineteen, and twenty-one provide an overall sum- mary of user reaction. 88.6 percent of users want the service to continue. Overall value of SELDOM was rated "very high" by 11.3 percent, "high" by 33.8 percent, "medium" by 42.2 percent, and "low" by 12.7 percent. SELDOM served to demonstrate the possibility of SDI for monographs "amply" according to 36.6 percent of users, "adequately" to 50.6 percent of users, and "poorly" to 12.65 percent of users. There was less certainty on how such a program should be administered or coated, particularly since a long-range cost study was not yet available. Clearly those who were impressed with SELDOM's effectiveness and future possibilities wanted other faculty to have the same opportunities, yet they cautioned against a blanket service. One comment sums this up best, "It should be available to anyone who has a perceived need for it-but require them to at least make the effort of setting up the profiles, etc." Many of the less than enthusiastic comments about SELDOM could be correlated with little or no user feedback to the search editor in order to improve relevancy and recall. User education in this regard is crucial in order that all users fully understand the possibilities and limitations of the SDI service. The success of any existing SDI service in the periodical literature has hinged on a good data base and up-to-date, specific profiling according to Smith and Lynch ( 1971 ). ( 4) The effectiveness of the profiling is a direct function of the ingenuity and persistence of the user and the profile editor. DISCUSSION This study has attempted to weigh the usefulness of an SDI service primarily with regard to its utility as a current awareness service. SELDOM, in order to be worthwhile, must either be faster or broader in its coverage than existing services. Two comparisons readily arise out of the com- mentary of the users. Some library science professors felt that the LC proofslip service was just as fast as SELDOM and thus there was no advantage in having the latter when the former was available. A study done at the University of Chicago by Payne and McGee ( 1970) repudiates this argument fairly effectively. ( 5) Findings at Chicago show that MARC is faster than the corresponding proofslips. A number of users rely heavily on publishers' blurbs and prepublication notices and find that often books for which records appear on SELDOM are already on the library shelves. This observation is not altogether an indictment of SELDOM since another user observed that he appreciated being able to have the hard copy im- 44 Journal of Library Automation Vol. 5/1 March, 1972 mediately; and in some cases he might not even have known about the item except for SELDOM. Some users mentioned that waiting for evalua- tive reviews could put one at least a year behind just in placing the order for the book, let alone receiving it. SELDOM has the virtue of informing individuals of the existence of new books, but the delay in having the actual item might be problematic, so one question was directed to this consideration. Some people felt that it was at least worth something to know that a book existed even if one could not consult it immediately. Numerous complaints were aired re- garding the slowness of obtaining items ordered through a library's ac- quisitions department. In fact one user said this slowness meant he had to purchase personal copies of items he wanted/ needed. As indicated earlier in the introduction, the TESA-1 acquisitions-cataloging routine at the University of Saskatchewan library does have the capability to speed up actual receipt of books by the patron. A recent development at the Library of Congress has definite implica- tions for the future of SELDOM and any other MARC-based SDI pro- grams. The CIP (Cataloging in Publication) 0 program initiated this summer means that LC will now be able to make available cataloging in- formation, except for collation, for books about to be published, at a time factor of up to six weeks before publication. Such MARC records will have a special tag designating them as CIP material. Furthermore, CIP records will appear only on MARC, the number predicted is 10,000 for the first year and 30,000 by the third year, a figure which would include all American imprints. (6) MARC-OKLAHOMA o o has already surveyed the subscribers to its SDI Project to determine whether they would prefer to receive both CIP MARC records and regular MARC records or only one of the two categories. Users preferred to receive both types of infor- mation and appropriate changes have been made to the Oklahoma SDI programs. (7) Beginning with September MARC CIP records will appear and present information on books thirty to forty-five days before they are published. Several library personnel appreciated the usefulness of SELDOM as an outreach service of the university library into the academic community. They see SELDOM as a public relations tool. Numerous efforts are at the present time being made by librarians to alert individuals to materials in their several fields of interest, and SELDOM can play an important role in providing an active dissemination of information on a systematic basis. This is the direction in which we need to move so that our role becomes both that of a collector of information and a disseminator of information. Special librarians have been doing this kind of thing for years and SEL- DOM allows for speCialized service to a larger user group. IMPLICATIONS AND CONCLUSIONS 1. An SDI service based on MARC can be helpful in building a bal- anced library collection depending on the efforts of faculty and/ or bib- Selective Dissemination of MARC /BUHR 45 liographers in setting up their proRles and maintaining them. The article by Ayres ( 1971 ) is particularly good on this aspect. ( 8) The parameters of the MARC data base must constantly be kept in mind, just as the constraints of the ad hoc methods must be considered in any comparisons. Pub- lishers' blurbs in journals have the limitation of not systematically covering all the publications in a given subject area; book reviews tend to appear too late to allow users to receive current information on new books; SEL- DOM corrects the first shortcoming at the expense of not having the eval- uations appearing in book reviews. On the other hand MARC tapes do represent the cataloging of books in the English language by one of the largest national libraries in the world, and thus provide a coverage which is hard to duplicate by any one other alerting service. 2. Comments, especially from users in the social sciences and humani- ties, indicate that an SDI system for new monographs has greater pertinence in their area than perhaps in the natural and applied sciences simply be- cause of the nature of research done in the two areas. A recent study by J. L. Stewart ( 1970) substantiates this factor for the field of political science. ( 9) His detailed analysis of the fatterns of citing in the writings ap- pearing in a collective work in politica science indicated that 75 percent of such citations were from monographs leading him to the obvious con- clusion that "monographs provide three times as much material as do journals" in the field of political science. By contrast, journals are likely more crucial for the fields of natural and applied science, and provide the key access point for vital information. 3. SDI of MARC, most users felt, should demand a fair amount of effort on the part of users to assure that the service would obtain optimum return for money invested. A blanket service to all faculty would be waste- ful since many faculty would not have a perceived need for it and others would not use it enough if it was simply offered free to everyone. Comments tended to favor making contact through the departmental library repre- sentative and channel weekly printouts through this individual. A cost study will help determine whether it is economically feasible to operate SELDOM in an academic setting with at least 100 users. If current sub- scription costs for SDI services such as those offered by Can/SDI of the National Science Library, Ottawa can be maintained, and early indications are that they can, a cost of $100 per profile per year may be feasible bring- ing the annual expenditure for 100 users to $10,000. A chief variable which makes effective costing difficult is the variation in the number of records appearing on each weekly tape and this is a variable which can only be dealt with by prediction on the basis of the number of records on past tapes. 4. SELDOM has the virtue of adding a major role of dissemination of information to libraries which up until now have primarily operated as starers of information. - Selective Dissemination of MARC/BUHR 47 822 33~ SHAKESPEARE WILLIAM FLEAY. FREDERICK GARO. 1831-1909. SHAKESPEARE MANUAL. NEW YORK.AMS PRESS<1970> XXIII. 312 P. 19 CM. lC 76-130621 PR2895 P1002 EN 01 TW 000 WT 000 S R0252 FC LENG 822.33 ISBN 0~0~02~08~ SELDOM Evaluation Questionnaire l. What is your feelin g about the SDI lists as a source for finding out about the existence of newly published works in your fields of interest? Would you say that the lists provided a source which was: (a) very useful (b) useful ( c ) moderately useful (d) inconsequential 18 34 12 6 2. Do you feel that the SDI lists brought to your attention works of in- terest which are not generally cited b y other sources that you use to learn of new publications? (a) many works (b ) some works (c) a few works (d) none 10 39 19 2 3. How would you characterize your feeling about the relative propor- tions of the items "of interest" ( relevant items) and "those not of interest" (irrelevant items) included in the SDI lists? (a) the proportion of relevant items in the lists was satisfactory. 57 (b) the proportion of irrelevant items in the lists was too high. 13 48 Journal of Library Automation Vol. 5/1 March, 1972 4. It is inevitable that some "not-of-interest" items are included in the SDI lists. Was the inclusion of irrelevant notices bothersome to you? (a) yes ( b ) no 6 65 REASONS: 5. On the other hand, it is possible that for any given search run, some relevant items in the file are missed. The chance of relevant items being missed can generally be minimized by certain search adjustments, but with a resulting increase in irrelevant notices. Would you be will- ing to increase the number of irrelevant notices received in order to maximize the number of relevant ones? ( a) yes ( b ) no 40 29 REASONS: 6. The SDI lists notified you of an average of--items per list which you judged to be "of interest." On a purely quantitative basis, would you say that this number was satisfactory, or for some reason too small or too large? (a) satisfactory 48 ( b ) too small 16 ( c ) too large 1 7. When the input to the MARC file is increased, your SDI output would also likely increase. Do you feel that you would like to be able to set some arbitrary upper limit on the quantity of items included in each SDI list even at the risk of missing a number of relevant items? (a) yes (b) no If yes , Maximum number__ _____________ _ 17 51 REASONS: 8. The SDI lists alerted you to a number of items which you judged to be "of interest." Would you say that "of interest" items were new to you? (a) in most cases (b) frequently (c) occasionally (d) seldom 17 33 17 5 9. Do you feel that the proportion of items "of interest" which were also "new" to you was: (a) satisfactory (b) too low 54 17 10. Would you say that, in general, information given for the entries in the SDI lists is adequate to judge whether an item is or is not of interest to you? ( a) yes ( b ) no 58 10 11. What elements of the entry did you most often find useful in making evaluations? (a) author/ editor (b) title (c) publisher (d) series note (e) sub- 38 55 9 4 35 ject headings (f) classification numbers (g) other (please specify) 8 1 Selective Dissemiootion of MARC /BUHR 49 12. What is the primary use to which you put the SDI information? (a) recommendation for library acquisition (b) personal purchase of 51 12 item (c) other (please specify) 15 13. If your recommendation originates the library order for a publication, it will be some time before the work is available; and even if already on order, most of the publications included in your lists were probably too new to be available from the library at the same time you received the list. Do you feel that this diminishes the value of the SDI service? (a) significantly (b) somewhat (c) negligibly 2 w ~ For what reasons? 14. A potential value of SDI service, based on the large volume of newly published works cataloged by and for the Library of Congress, is to bring together in one list timely notices for those works in the file which correspond to your several fields of interest. Do you feel that the experimental SDI service demonstrated this capacity? (a) amply (b) adequately (c) poorly 26 36 9 15. Is the format of the SDI notices satisfactory? (a) yes (b) no 61 9 If not, what format would you suggest? 16. Is the distribution schedule of once a week satisfactory? ( a ) yes ( b ) no 71 0 17. On the average, how much time would you estimate it took to examine an SDI list? Roughly: Minutes: (a) 5 (b) 5-10 (c) 10 (d) 10-15 (e) 15 (f) 15-20 (g) 20 23 16 9 11 5 1 5 18. A possible by-product of this SDI service is the building up of a cum- ulative MARC tape file which can be searched in various ways by computer. Would you make use of such a file? (a) yes (b) no 40 18 If no, for what purposes? 19. Judging from your total experience with the SDI service, would you characterize its overall value to you as: ( a) very high (b) high (c) medium (d) low R 24 30 9 50 Journal of Library Automation Vol. 5/1 March, 1972 20. The MARC file at present represents English monographs cataloged by the Library of Congress on a week-by-week basis. Sometime in 1972, the Library of Congress will begin to add some non-English monographs to the MARC file. Keeping in mind the forthcoming expanded MARC file on which future SDI service would be based, do you feel that its value to you would then be: (a) increased (b) the same (c) less 30 33 7 21. Do you personally want this SDI service to be continued? (a) yes (b) no (c) it doesn't matter ~ 3 5 22. Do you faculty? (a) yes feel that this SDI service should be offered to the entire 42 REASONS: (b) no 14 23. Do you feel that this SDI service should appropriately be made avail- able by the university, i.e., that the university should organize and administer the service? (a) yes ( b ) no (c) don't know 36 5 23 24. Do you feel that the university alone should pay for this faculty SDI service? (a) yes (b) no (c) don't know 30 6 25 25. Optional: General comments, pros and cons, elucidation of above re- plies, attitudes, suggestions, etc., concerning the SDI service.