Evidence Based Library and Information Practice 2011, 6.3 1 Evidence Based Library and Information Practice Evidence Summary Master’s Students in History Could Benefit from a Greater Library Sensitivity and Commitment to Interdisciplinarity, and from More Efficient Document Delivery A Review of: Sherriff, G. (2010). Information use in history research: A citation analysis of master’s level theses. Portal: Libraries and the Academy, 10 (2) 165-183. doi: 10.1353/pla.0.0092. Reviewed by: R. Laval Hunsucker Information and Collection Specialist Emeritus University Libraries, Universiteit van Amsterdam Amsterdam, Breukelen, The Netherlands Email: amoinsde@yahoo.com Received: 01 Mar. 2011 Accepted: 15 July 2011 © 2011 Hunsucker. This is an Open Access article distributed under the terms of the Creative Commons-Attribution-Noncommercial-Share Alike License 2.5 Canada (http://creativecommons.org/licenses/by-nc-sa/2.5/ca/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly attributed, not used for commercial purposes, and, if transformed, the resulting work is redistributed under the same or similar license to this one. Abstract Objective – This study sought to determine the characteristics of research materials used by history students in preparing their master’s theses. Of which information resources formats did such students make use, and in what proportions? What was the age distribution of resources used? What was the dispersal over journal titles and over subject classification, i.e., the degree of interdisciplinarity? To what extent did the master’s students make use of non- English-language materials? To what extent did their institution’s library hold the resources in question? The investigator was especially interested in finding quantitative support for what he terms two “hypotheses.” The first of these is that historical research depends to a high degree on monographs, journal articles being far less important to it than they are to research in, especially, the natural sciences and technology. The second is that the age distribution of resources important to historical research is much flatter and longer than that of resources upon which researchers in the natural sciences and technology rely. Design – Citation analysis, supplemented with comprehensive catalogue searches. mailto:amoinsde@yahoo.com� http://creativecommons.org/licenses/by‐nc‐sa/2.5/ca/� Evidence Based Library and Information Practice 2011, 6.3 65 Setting – Southern Connecticut State University (SCSU), a mid-sized public university located in New Haven, Connecticut, U.S.A. Subjects – MA and MS theses (N=47) successfully submitted to the Department of History over the period from academic year 1998/1999 through academic year 2007/2008, inclusive. Methods – The investigator initially identified the theses through a search of the online catalogue (“Consuls”) of the Connecticut State University system, and retrieved all of them in either electronic or hard-copy form. He then subjected all citations (N=3,498) listed in the references sections of these theses to an examination in order to identify for each cited resource the format, the age, the language, and, in the case of scholarly journal articles, the journal of publication. He carried out bibliographic searches in order to rectify any citations which he had noted to be faulty or incomplete. The study took no account of possible additional citations in footnotes or endnotes or in the text, and did not measure citation intensity (whether, for instance, a thesis referred only once, or perhaps many times, to a given resource). Duplicates “were ignored.” He furthermore performed systematic searches in Consuls and in the Library of Congress (LC) online catalogue in order to establish, insofar as possible, into which assigned LC Classification class each resource fell, and whether it belonged to the holdings of the SCSU library. “Holdings,” as used here, includes physical resources owned, as well as those resources to which the library has licensed access. Not marked as either “held” or “not held” were: resources available online without restriction or charge, items not identified in either Consuls or the LC catalogue, and all government documents. Ages of cited resources were calculated based on the edition or version date actually given in a student’s citation, without any consideration of a possible earlier date of the original version of the publication or document concerned. Main Results – Format, age distribution, and journal frequency. The local citation analysis found that 53.2% of all cited resources were monographs, 7.8% were scholarly articles, 5.3% were contributed chapters in books, and 0.6% were dissertations or theses. Non-scholarly periodicals accounted for 15.7%, government documents for 6.7%, and freely available web documents for 4.1%. The remainder, approximately 6.5%, comprised archival papers, judicial documents, directories, interviews, posters, audiovisual materials, and 13 other formats. Cited resources, measured back from the date of acceptance of the citing thesis, ranged from 0 to 479 years old; the mode was 3 years, but the median was “25” (p. 170) or “26” (p. 177) years. Just over 70% (i.e., 2,500 cited resources) were more than ten years old. Almost one thousand of the cited resources were fifty or more years old. The 274 scholarly journal articles included in the references sections were spread over 153 distinct journal titles, of which 105 titles made only one appearance, and 136 titles three or fewer appearances. The mean was 1.8 appearances. Subject dispersal and language. Of the 2,084 cited resources for which LC classification was locatable, 51.5% had a classification other than history, i.e., other than class C, D, E, or F. Nearly two thirds (66.0%) of the cited scholarly journal articles had appeared in journals with a focus other than history. (Note: table 4 is incorrect, precisely reversing the actual ratio.) Of all cited items, 98.5% were in the English language. Half (27) of the non-English-language resources cited were in Korean, all cited in the same thesis. Books (i.e., monographs plus compilations from which contributed chapters were cited) accounted for 87.0% of foreign-language citations. More than four fifths of the examined theses (83.0%) cited not a single non-English- language resource. Local holdings. Of all 3,498 cited items, 3,022 could be coded as either “held” or “not held” by the SCSU library. Of the items so coded (not, as Evidence Based Library and Information Practice 2011, 6.3 66 indicated on p. 180, of all cited items), scarcely two fifths (41.0%) belonged to the library’s holdings. The holdings percentage was highest (72.6%) for the 274 scholarly journal articles cited, followed by the 186 contributed chapters (50.0%), the 550 non-scholarly periodical items (49.5%), and the 1,861 monographs (46.8%). For other cited formats, the percentage was much lower, and in some cases, e.g., for the 55 archival and the 44 judicial documents, it was 0.0%. Of the 54 foreign-language resources cited, the institution’s library held only two. Conclusion – The investigator concludes that his study’s findings do indeed lend quantitative support to his two “hypotheses.” This outcome will surprise few, if any, librarians; it is in accord with what Koenig (1978) long ago saw as a matter of “intuition” and “all conventional wisdom,” something that many subsequent studies have confirmed. Sherriff accordingly recommends, firstly, that collections which strive to support historical research should, in matters of acquisition policy and budget allocation, take serious account of that field’s relatively strong dependence on monographs. Secondly, the data on age distribution carry obvious implications for librarians’ decision- making on matters such as de-accessioning and weeding, relegation to remote storage, and retrospective acquisitions. This finding should also be considered, for instance, in connection with preservation policy and the maintaining of special collections. He even suggests that librarians “need to teach students the value of reviewing literature historically and showing them how to do so effectively” (p. 177). Sherriff considers a number of further (tentative) conclusions to be warranted or suggested by the results of this study. First of all, that historical research is now characteristically an interdisciplinary matter, in the sense that it requires extensive access to information resources, including journals, which libraries have traditionally not classified as belonging to the discipline of history itself. For a library supporting such research, this phenomenon “has implications for matters including collection budgets, reference work, bibliographic instruction, and the location of collections and departmental libraries” (p. 168). It also means “that librarians working with history students and history collections need to be aware of the relevant resources in other disciplines. This can improve reference work, research assistance, and bibliographic instruction; it may also help the coordination of acquisitions across departmental lines” (p. 179). Secondly, one may conclude that “there is no ‘core’ collection of journals for history” (p. 178) which will be able to satisfy a large proportion of master’s students’ research needs. Thirdly, the fact that a library such as SCSU’s holds significantly less than half of what master’s students require for preparing their theses “may exercise a narrowing effect on students’ awareness of the existing literature on their topics” (p. 180), “increases the importance of departmental faculty, reference librarians, and subject specialist librarians drawing students’ attention to resources beyond the library’s catalogues and collections” (p. 180), and requires that the library give serious attention to effective document delivery arrangements. Finally, this study’s finding that only a small percentage of master’s students in history made use of non- English-language materials, but then in certain cases used them rather extensively (27 Korean items cited in one thesis, ten Italian in another, nine Spanish in yet another), suggests that acquisition, or at least proactive acquisition, of such materials needn’t be a priority, as long as, once again, the students concerned have easy access to efficient and affordable document delivery services. Sherriff does concede, however, that his finding could indicate “that students are unaware of relevant resources in other languages or are aware of them but lack the language skills necessary to use them” (p. 179). Evidence Based Library and Information Practice 2011, 6.3 67 Commentary Sherriff did not set out to answer a limited, well constructed, explicitly stated, and practice- oriented research question. His results are not of the kind that practitioners can put directly or easily to use. The significance for us of his study is that it does now provide us with some welcome hard data on certain features of thesis citation practice in a specific context (a small history program, without doctoral component, at a young non-research-extensive state university), and in particular on: the high degree of interdisciplinary and journal-reference dispersal; the low degree of local availability of cited resources; and the extremely low frequency of foreign-language resource citation. This is a good beginning. What we now need, to give such data genuine evidentiary value for others, is further and comparable data from similar programs elsewhere, as well as findings, preferably longitudinal, on the same variables from studies of history research at other types of institutions and on other levels. This reviewer was therefore indeed surprised to note Sherriff’s strong implication that his conclusions from this single study should be taken as generalizable to graduate research in history at all types of institutions, or indeed even to scholarly research by historians in general. To buttress such a view, he cites no relevant earlier research beyond the tentative findings of a study of citations to journal articles in the fields of geology and biology (Zipp, 1996). It is in fact possible, and, if so, quite significant, that his quantitative results regarding interdisciplinarity are representative for historical research more broadly (Buchanan & Hérubel, 2011; Dalton & Charnigo, 2004; Delgadillo & Lynch, 1999; Hellqvist, 2010). So too, perhaps, those regarding age-distribution of cited resources (Lowe, 2003; Smyth, 2011). But that they are in further respects representative is open to significant doubt, in view of findings reported by other researchers (Broadus, 1985; Dalton & Charnigo; Delgadillo & Lynch; Haycock, 2004; Jones, Chapman, & Woods, 1972; Knievel & Kellsey, 2005; Lowe 2003; Pancheshnikov, 2007). Even his assumption that master’s research in its citation practice and patterns does not as such significantly differ from doctoral research, conflicts not merely with conventional wisdom but also with what numerous investigations have shown, with regard not only to monograph/journal article ratio but also to the usage of foreign language resources, of non-scholarly periodicals, and of primary source documents (Barrett, 2005; Eckel, 2009; Feyereisen & Spoiden, 2009; Kushkowski, Parsons, and Wiese, 2003; Pancheshnikov, 2007; Smyth, 2011). In at least all of these other respects one can, unfortunately, not view Sherriff’s study as contributing meaningfully to the evidence base for practitioners concerned with library support for history researchers more generally. The potential usefulness of Sherriff’s findings suffers as well from certain inherent, research design, and methodological shortcomings. The population studied is unexemplary for the academic discipline at large, since many categories of historical scholarship are poorly represented, or not represented at all. More than two thirds of the theses dealt with North American history. There is no apparent sensitivity to differing sub-disciplines, or diverse historiographic or theoretical/critical approaches, though these may vary considerably in resource requirements and citation practice (Hellqvist, 2010). His commitment to an entirely quantitative and unobtrusive approach meant that he – notably unlike, for example, Junni (2007) in her study of master’s students; or Fuchs, Thomsen, Bias, and Davis (2006) in their application of “behavioral citation analysis” to doctoral students – had no contact with any students or faculty members. Interviewing or surveying them could have yielded crucial relevant information on prevailing citation culture, guidelines, or requirements; on specific advisor or departmental expectations; and so on. Having opted for strictly quantitative methods, he can, of course, also tell us nothing about how the Evidence Based Library and Information Practice 2011, 6.3 68 students discovered or accessed – or why they may have failed to discover or to access – relevant materials; nor, for example, can he report on the extent to which they made use of resources which they eventually did not cite, or perhaps mentioned only in the text or in notes. The report, furthermore, provides no demographic information. Measurement bias is also present: The theses were submitted in the years 1998-2008, but Sherriff apparently coded all of their cited resources for local availability on the basis of catalogue searches which he performed at a later point, presumably in 2009. This of course compromises the validity of the holdings data reported. To what extent, we can only guess – but it seems likely that, during the period from 1998 to 2009, much will have changed as far as local resource availability (and citability?) was concerned. Such developments will moreover have exercised a diachronically confounding influence on all of the variables investigated by Sherriff, in some cases perhaps a fairly significant influence. The above-mentioned study by Fuchs et al. (2006) incorporated this same measurement bias, and they frankly stated that this was “problematic,” and “a clear limitation.” Sherriff, on the other hand, leaves entirely unmentioned the same deficiency in his own research design. Indeed, he is in general insufficiently forthcoming about his investigation’s limitations. It takes hardly any account of the role of electronic formats (or the concomitant blurring of boundaries between traditional formats), of emerging new scenarios for scholarly communication, or of the uncertain outlook for monographic publishing. A more fundamental defect is that his article displays only perfunctory sensitivity to the inherent weaknesses of citation analysis – or, as in his case more specifically, references list analysis – as a research method. Much has been written concerning its weaknesses and validity problems, but Sherriff makes no reference even to publications as relevant as those of Line (1977, 1978), Smith (1981), MacRoberts and MacRoberts (1989, 2010), Kelland and Young (1994), nor even to Haycock’s (2004) specific warnings regarding potential graduate student citing tendencies. We should by now all be aware that inclusion in a references list is not necessarily the same thing as use, and that use – of whatever variety – is something quite different from need. Yet Sherriff, having eschewed any qualitative research component, nonetheless clearly implies that his study is a contribution toward understanding not only history researchers’ resource and library use, but even their information resources needs (or simply “information needs,” as he at one point puts it). This makes all the more striking his apparent obliviousness, in his literature review and references, to many relevant previous studies, as well as his paucity of explicit suggestions concerning where further research is necessary or would be useful. Finally, we may note that, although the article presents (in six tables and one figure) only selected statistics from the research study, the author gives no indication whether full data are also available to interested readers. Line (1977, 1978) found that citation analyses were of no value, and were indeed irrelevant, to practical librarianship. Kelland and Young (1994) concluded that the relationship between citation activity and library materials use was for numerous reasons “inherently problematical,” and saw the two measures as no more than “moderately correlated.” Still, the question remains: even if we do in principle accept such a correlation, what should this imply for our actual library and information services practice? Sherriff’s study contributes only modestly toward answering that question, even with regard to library support for research in history. It does, though, at least suggest a few specific considerations which librarians can bring to bear, preferably in combination with other, in particular qualitative, indicators, in attempting to make their services more Evidence Based Library and Information Practice 2011, 6.3 69 appropriate for a particular category of student researchers in a particular kind of institution. References Barrett, A. (2005). The information-seeking habits of graduate student researchers in the humanities. The Journal of Academic Librarianship, 31(4), 324-331. Broadus, R. N. (1985). A proposed method for eliminating titles from periodical subscription lists. College & Research Libraries, 46(1), 30-35. Buchanan, A. L., & Hérubel, J.-P. V. M. (2011). Taking Clio's pulse . . .: Or, examining characteristics of monographic publications reviewed by American historical review. Journal of Scholarly Publishing, 42(2), 160-181. Dalton, M. S., & Charnigo, L. (2004). Historians and their information sources. College & Research Libraries, 65(5), 400-425. Delgadillo, R., & Lynch, B. P. (1999). Future historians: Their quest for information. College & Research Libraries, 60(3), 245-259. Eckel, E. J. (2009). The emerging engineering scholar: A citation analysis of theses and dissertations at Western Michigan University. Issues in Science & Technology Librarianship, 56. Retrieved 7 May 2011 from http://www.istl.org/09- winter/refereed.html Feyereisen, P., & Spoiden, A. (2009). Can local citation analysis of master's and doctoral theses help decision-making about the management of the collection of periodicals? A case study in psychology and education sciences. The Journal of Academic Librarianship, 35(6), 514-522. Fuchs, B. E., Thomsen, C. M., Bias, R. G., & Davis, D. G., Jr. (2006). Behavioral citation analysis: Toward collection enhancement for users. College & Research Libraries, 67(4), 304-324. Haycock, L. A. (2004). Citation analysis of education dissertations for collection development. Library Resources & Technical Services, 48(2), 102-106. Hellqvist, B. (2010). Referencing in the humanities and its implications for citation analysis. Journal of the American Society for Information Science and Technology, 61(2), 310–318. doi: 10.1002/asi.21256 Jones, C., Chapman, M., & Woods, P. C. (1972). The characteristics of the literature used by historians. Journal of Librarianship and Information Science, 4(3), 137-156. doi: 10.1177/096100067200400301 Junni, P. (2007). Students seeking information for their masters' theses: The effect of the Internet. Information Research, 12(2), paper 305. Retrieved 7 May 2011 from http://informationr.net/ir/12- 2/paper305.html Kelland, J. L., & Young, A. P. (1994). Citation as a form of library use. Collection Management, 19(1&2), 81-100. doi: 10.1300/J105v19n01_04 Knievel, J. E., & Kellsey, C. (2005). Citation analysis for collection development: A comparative study of eight humanities fields. The Library Quarterly, 75(2), 142- 168. doi: 10.1086/431331 Koenig, M. E. D. (1978). Citation analysis for the arts and humanities as a collection Evidence Based Library and Information Practice 2011, 6.3 70 management tool. Collection Management, 2(3), 247-264. doi: 10.1300/J105v02n03_06 Kushkowski, J. D., Parsons, K. A., & Wiese, W. H. (2003). Master's and doctoral thesis citations: Analysis and trends of a longitudinal study. Portal: Libraries and the Academy, 3(3), 459-479. doi: 10.1353/pla.2003.0062 Line, M. B. (1977). On the irrelevance of citation analyses to practical librarianship. In W.E. Batten (Ed.), EURIM II: a European conference on the application of research in information services and libraries: 23-25 March 1976, RAI International Congrescentrum, Amsterdam, Netherlands: Proceedings (pp. 51-56). London: Aslib. Line, M. B. (1978). Rank lists based on citations and library uses as indicators of journal usage in individual libraries. Collection Management, 2(4), 313-316. doi: 10.1300/J105v02n04_04 Lowe, M. S. (2003). Reference analysis of the American Historical Review. Collection Building, 22(1), 13-20. doi: 10.1108/01604950310457168 MacRoberts, M. H., & MacRoberts, B. R. (1989). Problems of citation analysis: A critical review. Journal of the American Society for Information Science, 40(5), 342-349. doi: 10.1002/(SICI)1097- 4571(198909)40:5<342::AID- ASI7>3.0.CO;2-U MacRoberts, M. H., & MacRoberts, B. R. (2010). Problems of citation analysis: A study of uncited and seldom-cited influences. Journal of the American Society for Information Science and Technology, 61(1), 1–12. doi: 10.1002/asi.21228 Pancheshnikov, Y. (2007). A comparison of literature citations in faculty publications and student theses as indicators of collection use and a background for collection management at a university library. The Journal of Academic Librarianship, 33(6), 674-683. Smith, L. C. (1981). Citation analysis. Library Trends, 30(1), 83-106. Smyth, J. B. (2011). Tracking trends: Students’ information use in the social sciences and humanities, 1995–2008. Portal: Libraries and the Academy, 11(1), 551- 573. doi: 10.1353/pla.2011.0009 Zipp, L. S. (1996). Thesis and dissertation citations as indicators of faculty research use of university library journal collections. Library Resources & Technical Services, 40(4), 335-342. / Evidence Based Library and Information Practice