Microsoft Word - CLASS4127 Evidence Based Library and Information Practice 2008, 3:4 82 Evidence Based Library and Information Practice Classic Predicting Future Information Resource Utilization Under Conditions of Scarcity: The First Cohort Study in Health Sciences Librarianship A review of: Postell, William Dosité. “Further Comments on the Mathematical Analysis of Evaluating Scientific Journals.” Bulletin of the Medical Library Association 34.2 (1946): 107-9. Figure 1: William Dosité Postell, 1908-1982. Pioneer of the Cohort Design in Librarianship Permission Granted for EBLIP to Publish by William Postell, Jr. at Tulane University, New Orleans, LA . Reviewed by: Jonathan D. Eldredge Associate Professor, School of Medicine and Health Sciences Library and Informatics Center University of New Mexico Albuquerque, New Mexico, United States of America Email: jeldredge@salud.unm.edu Evidence Based Library and Information Practice 2008, 3:4 83 Received: 19 September 2008 Accepted: 14 November 2008 © 2008 Eldredge. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Abstract Objective – To predict future use of journal titles for making subscription decisions. Design – Retrospective cohort study. Setting – Louisiana State University School of Medicine Library in New Orleans. Subjects – All library users, estimated to consist of primarily faculty members or their designees such as research assistants. Methods – Estelle Brodman’s previous citation analysis and reputational analysis (1944) that produced a list of eleven top- ranked physiology journal titles served as the catalyst for Postell’s retrospective cohort study. Postell compiled data on all checkouts for these specific eleven journal titles in his library for the years 1939 through approximately 1945. Main Results – Postell performed a Spearman rank-difference test on the rankings produced from his own circulation use data in order to compare it against journal title rankings produced from three other sources: (1) citation analysis from the references found in the Annual Review of Physiology based upon a system pioneered in 1927 by Gross and Gross; (2) three leading national physiology journals; and, (3) a reputational analysis list of top-ranked journals provided by the faculty members at the Columbia University College of Physicians and Surgeons Department of Physiology. Postell found a relatively high correlation (.755, with 1.000 equaling a perfect correlation) between his retrospective cohort usage data and the reputational analysis list of top-ranked journals generated by the Columbia faculty members. The two citation analyses performed by Brodman did not correlate as highly with Postell’s results. Conclusion – Brodman previously had questioned the use of citation analysis for journal subscription purchase decisions. Postell’s retrospective cohort study produced further evidence against basing subscription purchases on citation analysis. Postell noted that the citation analysis method “cannot always be relied upon as a valid criterion” for selecting journals in a discipline. Commentary Libraries currently devote major portions of their annual budgets toward purchasing journal subscriptions. Similarly large budgetary outlays motivated Gross and Gross to pioneer their use of a rudimentary form of citation analysis based upon all of the references found in the single year 1926 in the Journal of the American Chemical Society to identify top-ranked journals in chemistry (385-9). Estelle Brodman documented 22 derivative applications of the citation analysis method published since 1927, noting that some researchers had enlarged their coverage to include multiple leading journals as sources for to obtaining the pool of references for their citation analyses (479). Brodman conducted a reputational analysis list of top-ranked journals provided by the faculty members at the Columbia University Evidence Based Library and Information Practice 2008, 3:4 84 College of Physicians and Surgeons Department of Physiology as an alternative method to the citation analysis. Brodman concluded, on the basis of her Spearman rank-difference correlations, that the citation analysis method generally led to “unscientific” and “untrustworthy results” when making journal subscription decisions. Brodman concluded further that “In spite of these extremely grave drawbacks, the (citation analysis) method will probably continue to be employed by librarians until the library profession is presented with a better one.” William D. Postell conducted the first cohort study in health sciences librarianship by measuring what journal titles his library users actually checked out. He correlated his ranked frequency usage data with Brodman’s reputation analysis and citation analyses, finding the latter lacking in reliability. In the process, he provided our profession with one of its most durable designs for applied research. By contemporary EBLIP critical appraisal standards, Postell’s 1946 retrospective cohort study does not meet most standards of methodological rigor. Yet, Postell shifted librarians’ emphasis away from citation analysis with this article by introducing what would become one of our profession’s most enduring applied research methods: the cohort study design. Postell also indirectly transitioned librarianship away from the citation analysis method, with its limited validity for journal selection (Garfield), while offering a less authority- oriented alternative to Brodman’s reputational analysis. How does Postell’s 1946 article stand up to two contemporary critical appraisal checklists? Booth and Brice’s critical appraisal checklist prompts reviewers to evaluate any reported research results in accordance with the three major areas: (1) closeness to the truth, (2) credible and replicable results, and (3) applicability to professional practice (108-9 and 263). Postell’s study fulfills most of the Booth and Brice checklist’s first criteria area by offering a close representation of the truth by providing a clear question, couching it in the context of other studies, comparing his results to relevant studies, and representing all library users. Yet, Postell’s study does not indicate who collected the data so it appears to not meet this specific criterion for the first area for critical appraisal. Postell’s classic study clearly does not meet Booth and Brice’s second broad criteria area of credibility and replicability. Postell does not even address these issues, including the area of documenting any study limitations. A biographical account of Postell’s study (Eldredge, “SCC Milestone”) suggests that Postell conducted his study during the unusual historical circumstances of an intensively rapid medical school training program at LSU during World War Two, thereby raising the possibility that his study was subject to “history” as a threat to validity. This kind of threat to validity occurs when researchers study a population during atypical circumstances that might cause behavior to deviate from the norm and therefore be non-representative of most situations (Miller and Salkind; Shadish, Cook and Campbell; Neuman). Characterizing U.S. medical school libraries’ practices during World War II, Walker described “The accelerated (academic) program, reduced staff, the increasing number of ‘our’ doctors who are away and of other doctors who are here, the changed emphasis in teaching and research…” (326), suggesting monumental deviations from normal activities. She also notes, of special relevance to this classic review, the extra emphasis paid during this era to the subject areas of “physiological and chemical research.”(329). For reasons of this history threat to validity, it would be difficult for Evidence Based Library and Information Practice 2008, 3:4 85 anyone to generalize the usage behavior that Postell measured to users’ behavior during a different historical period or set of circumstances. The third area of Booth and Brice’s checklist, applicability to one’s own practice, suggests again that the unique historical circumstances of Postell’s study, coupled to its occurrence at such distant date, cannot validate this first cohort study’s results for modern application to practice. Postell’s study consequently does not meet most of Booth and Brice’s modern standards of critical appraisal. Glynn’s 2006 critical appraisal checklist prompts reviewers to examine the four broad areas of (1) population, (2) data collection, (3) study design, and (4) results in a research study. Postell’s study appears to fulfill all of the relevant or known criteria in the first population area, except for the possible aforementioned problems, again, with a history threat to validity due to the unusual war time circumstances affecting the population of journal users. Postell’s study remains silent on most issues raised by the second data collection category of Glynn’s checklist whereas Postell’s choice of a cohort study for the third category in the Glynn checklist seems appropriate for answering the specific research question. This study does well on the fourth results section of the Glynn checklist, except that Postell apparently never delved into the issues of confounding variables or external validity. Overall, then, Postell’s 1946 study does not fare well when appraised by Glynn’s 2006 checklist. Given its poor performance when evaluated by these two contemporary critical appraisal checklists, how should we interpret then the significance of Postell’s 1946 classic retrospective cohort study? Principally, we should recognize that Postell ingeniously introduced the previously unused cohort research methodology in health sciences librarianship. The cohort design subsequently became a powerful tool for both collection development and library education practices. Historically, the cohort study design can be traced back to James Lind’s first study in 1747 of British sailors contracting scurvy, Louis’ 1835 bloodletting outcomes study in Paris, and Semmelweiss’ 1848 childbed fever study in Vienna. Yet, by even the 1930s, the cohort study design still was not a widely used research method in any field (Eldredge, “Cohort Studies”). Postell published on diverse topics such as medical history (“Principles”), library instruction for medical students (“Formal Training”; “Further Notes”), and library public relations (“Stimulating Interest”). Postell’s historical work on 19th Century medicine in Louisiana (“Principles”) offers the most likely inspiration for his adapting the cohort study design to health sciences librarianship. Postell noted in this 1942 article that 19th Century physicians in Louisiana were influenced by both American and French medical approaches. Physicians’ practice of bloodletting declined in Louisiana, however, when Pierre Charles Alexandre Louis’ research in Paris declared it ineffective. Although Postell never refers to it by its formal modern name, Louis had used the cohort study design to dispute the practice of bloodletting. Postell later most likely adapted the population, exposure, outcome(s) sequential structure of Louis’ cohort study design to answer his own prediction form of research question. Postell’s contribution to applied research seems all the more remarkable when one surveys the health sciences literature in the Bulletin of the Medical Library Association during the years 1911-1946. Ballard’s call for uniformity and comparability of library statistics focused upon holdings data, while barely mentioning usage data, except apparently as gross data only in service Evidence Based Library and Information Practice 2008, 3:4 86 demonstrating that library collections were used at all. Osler’s “The Science of Librarianship” solely revolved around the bibliography, classification, and the general management aspects of libraries. While Osler did praise American libraries for their accessibility, a nod to the efforts of John Cotton Dana (Kingdon), he did not mention usage data in any connection to the “science” of our field. Jenkin’s classic on cost benefit analysis of highly cited journals against price data never contemplated use data in her analysis. We can be certain that Postell’s cohort study led to numerous replications and adaptations within health sciences librarianship during the next two decades, principally through information resource usage studies (Kilgour, “Annual Report”; Morse, Beatty & Hodge; Keys, Kilgour, “Use: Part I”; Kilgour, “Use: Part II”; Kurth; Fleming and Kilgour; Staudt, Halbrook, and Brodman). Today, the cohort study design occupies the pinnacle of single study for evidence in the EBLIP Levels of Evidence for Prediction questions (Eldredge, “Evidence- Based Librarianship”). Therefore, we need to credit William Postell for his remarkable introduction to health sciences librarianship of one of its perennial methods of modern applied research now used frequently in service to major information resource purchasing decisions. Works Cited Ballard, James F. “Uniformity in Library Statistics.” Bulletin of the Medical Library Association 5.2 (1915): 21-6. 24 Nov. 2008. . Booth, Andrew, and Anne Brice. “Appraising the Evidence.” Evidence-Based Practice for Information Professionals. Eds. Andrew Booth and Anne Brice. London: Facet Publishing, 2004: 104- 118. [also in North America reprinted in Introduction to Health Sciences Librarianship. Ed. Sandra Wood. Binghamton, NY: Haworth Press, 2007: 241-70.] Brodman, Estelle. “Choosing Physiology Journals.” Bulletin of the Medical Library Association 32.4 (1944): 479- 83. 24 Nov. 2008. . Eldredge, Jonathan. “Cohort Studies in Health Sciences Librarianship.” Journal of the Medical Library Association 90.4 (2002): 380-92. 24 Nov. 2008. . Eldredge, Jon. “Evidence-Based Librarianship: Levels of Evidence.” Hypothesis 16.3 (Fall 2002): 10-13. 24 Nov. 2008. . Eldredge, Jonathan D. “SCC Milestone in EBL History. South Central Connection. 2003 February; 13 (2): 10, 14. 24 Nov. 2008. . Fleming, Thomas P. and Frederick G. Kilgour. “Moderately and Heavily Used Biomedical Journals.” Bulletin of the Medical Library Association 52.1 (1964): 234-41. 24 Nov. 2008. . Garfield, Eugene. “How ISI Selects Journals for Coverage: Quantitative and Qualitative Considerations.” Current Contents 22 (28 May 1990): 185-93. 24 Nov. 2008. Evidence Based Library and Information Practice 2008, 3:4 87 . Glynn, Lindsay. “A Critical Appraisal Tool for Library and Information Research.” Library Hi Tech 24.3 (2006): 387-99. Gross, P.L.K., and E.M. Gross. “College Libraries and Chemical Education.” Science 66.1713 (October 28, 1927): 385-9. Jenkins, R.L. “Cost Analysis of Medical Periodicals.” Bulletin of the Medical Library Association 22.3 (1934): 115- 23. 24 Nov. 2008. Keys, Thomas E. Applied Medical Library Practice. Springfield, IL: Charles C. Thomas Publisher, 1958. Kilgour, Frederick G. Annual Report of the Yale University Medical Library 1952/1953. New Haven, CT: Yale University, 1953. Kilgour, Frederick G. “Use of Medical and Biological Journals in the Yale Medical Library. Part I. Frequently Used Titles.” Bulletin of the Medical Library Association 50.3 (1962): 429- 43. 24 Nov. 2008. . Kilgour, Frederick G. “Use of Medical and Biological Journals in the Yale Medical Library. Part II. Moderately Used Titles.” Bulletin of the Medical Library Association 50.3 (1962): 444- 9. 24 Nov. 2008. . Kingdon, Frank. John Cotton Dana. Newark, NJ: The Public Library and Museum, 1940. Kurth William H. Survey of the Interlibrary Loan Operation of the National Library of Medicine. Bethesda, MD: National Library of Medicine, 1962. Miller, Delbert C, and Neil J. Salkind. Handbook of Research Design and Social Measurement. 6th ed.. Thousand Oaks, CA: Sage Publications, 2002, 49-51. Morse, E.H, Beatty, W.K., & Hodge H.M. “Annual Report on the Library.” Transactions and Studies of the College of Physicians of Philadelphia 21.4 (1954): 138-50. Neuman, William Lawrence. Social Research Methods: Qualitative and Quantitative Approaches. Boston: Allyn and Bacon, 2006. Osler, Sir William. “The Science of Librarianship.” Bulletin of the Medical Library Association 7.4 (1918): 70-4. 24 Nov. 2008. . Postell, William Dosité. “The Formal Training of Medical Students in the Use of the Library.” Journal of the Association of American Medical Colleges 15.4 (1940): 241-4. Postell, William Dosité. “Stimulating Interest in a Medical Library.” Bulletin of the Medical Library Association 29.3 (1941): 141-6. 24 Nov. 2008. . Postell, William Dosité. “The Principles of Medical Practice in Louisiana During the First Half of the Nineteenth Century.” Bulletin of the Medical Library Association 30.3 (1942): 191-7. 24 Nov. 2008. Evidence Based Library and Information Practice 2008, 3:4 88 . Postell, William Dosité. “Further Notes on the Instruction of Medical School Students in Medical Bibliography.” Bulletin of the Medical Library Association 32.2 (1944): 217-20. 24 Nov. 2008. . Shadish, William R., Thomas D.Cook, and Donald T. Campbell. Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston, MA: Houghton Mifflin Company, 2002. Staudt, Cecilia, Barbara Halbrook, and Estelle Brodman. “A Clinical Librarians’ Program – An Attempt at Evaluation.” Bulletin of the Medical Library Association 64.2 (1976): 236-8. 24 Nov. 2008. . Walker, Louise. “The Medical School Library in Wartime.” Bulletin of the Medical Library Association 31.4 (1943): 326-31. 24 Nov. 2008. .