Hypothesis, vol. 27, no. 1, Winter 2015 3 ARTICLES A LOOK AT THE SCHOLARLY OUTPUT OF THE MEDICAL LIBRARY ASSOCIATION RESEARCH SECTION Terrie R. Wheeler, AMLS1; Noga Yaniv, MSc2; and Ruth E. Fenske, PhD3 The Research Section (RS) has guided the Medical Library Association (MLA) specifically, and the field of health sciences librarianship more generally, in the practice of seeking evidence to inform decision making and practice. The best evidence comes from well- designed research projects. The Section has mentored many health sciences librarians in designing research projects, in conducting research according to the scientific method, and in publishing their results. MLA has adopted a Research Agenda and it relies on the Section to carry out that agenda. Further, the Research Section has played a role over the years in advancing the scholarship of the organization. For example, the MLA has embraced scholarly publishing by setting high standards in its own publication, the Journal of the Medical Library Association. Major articles in the journal are research based, require a structured or academic abstract, and generally consist of objectives, methods, findings, results and conclusions. Objective While the Research Section has been critical in advancing the scholarly communication within the field of health sciences librarianship, rarely has the Section highlighted its own role in scholarly output. That is the purpose of this abbreviated study. The authors were curious to discover what scholarly publications trends might exist among RS members. Some questions to answer were: 1) Which journals were most often selected for publication by RS members? 2) Were any of the works highly cited? 3) Did RS members engage in self-citation? 4) Have any RS members built a body of work on a certain topic or topics that have affected the profession at large? 5) Which RS members are most prolific? 6) Does citation mapping indicate that some RS members have had a national or international impact? 7) Are any RS member papers cited often enough to place them in the top ten percentile [1] of all papers published by discipline and year of publication? 8) Does the Research Section encourage collaboration with authors not in the Section, or, do RS members co-author primarily with other RS members? The authors wanted to answer these questions by looking at the scholarly output of the RS members, and ran the search strategy that was created in Web of Science on April 6, 2014. The search parameters included 1970 to date (April 6, 2014) and all document types. The strategy yielded 1,100 publications by RS members. Methods One author downloaded the 2013 - 2014 roster of members of the Research Section from the MLA website. The authors used surnames, first and middle initials when known, and organizational affiliations to identify the scholarly output of the Section. Thomson Reuters’ Core, formerly Thomson Reuters’ Web of Knowledge, was searched from 1970 to date. Of the 258 section members, 160 published at least one publication that was indexed by the Thomson-Reuters Core during this period. As the names that were downloaded from the section membership file did not always include middle initials, the authors took extra care to review articles by subject matter or affiliation to ensure that as often as possible they had selected the correct records for inclusion. The authors checked webpages and curriculum vitae as necessary to ensure accuracy as well as used the name disambiguation tool in Thomson-Reuters’ Core, along with checking certain records for full names in PubMed. After executing the search strategies on each current member in the section, the search strategies were combined into a single set and the final search strategy was run. Findings Authors ran a citation report on the resulting citations (Figure 1). There were 142 papers written in 1995 but in 1996 there was a drop to 18 papers. Beginning in 1996 there was an increasing trajectory of papers over time. Closer analysis of the 1995 papers reveals that Patricia Reavis, Barbara Epstein and Lynn Piotrowicz authored 115 book reviews that appeared in the December, 1995 issue of Psychiatric Services, which accounted for this unusually large productivity in one year. Peer-Reviewed Hypothesis, vol. 27, no. 1, Winter 2015 4 The citations to RS member articles in each year also reveal a steady upward trend which is generally found in sets of papers. There was a marked increase in 2012 (875) and 2013 (1078). This was a combination of more papers published in 2012 and of certain papers, such as papers 1, 2 and 3 in the top ten highly cited papers (Figure 2) having a high citation rate during that period. The times cited for the entire set in Figure 1 is 7,580, or approximately 194.36 citations per year. Papers published by RS members that are indexed in Web of Science have an h-index of 36. An overview of the h- index and the reliability of such indices can be found in an article by Alonso et al [2]. To find the ten journals in which members of the Research Section most frequently published, we analyzed the set by source title. The top journals were:  Journal of the Medical Library Association (222 papers),  The Bulletin of the Medical Library Association [precedes the JMLA] (148 papers)  Library Journal (139 papers). Several members have authored book reviews for Library Journal, which accounted for this higher number.  Psychiatric Services (115 book reviews)  Neurologist (33 papers)  Library Trends (22 papers)  The Journal of Family Practice (17 papers)  Online CDROM Review (15 reviews)  Reference and User Services Quarterly, formerly RQ (15 papers)  College and Research Libraries (14 papers) The Web of Science subject categories most frequently used to identify papers in this set were as follows: Information Science and Library Science (718 papers), Public Environmental and Occupational Health (134 papers), Health Policy Services (126 papers), Psychiatry (123 papers), and Computer Science Information Systems (79 papers). The nine most prolific members, according to the Web of Science data, during that time frame were Barbara Epstein (129 papers), Margaret Henderson (46 papers), Linda Smith (43 papers), Kay Wellik (43 papers), Joanne Gard Marshall (38 papers), Pru Dalrymple (35 papers), Lucretia McClure (34 papers), Ellen Detlefsen (32 papers), and Scott Plutchak (31 papers). The Research Section’s most influential authors, according to number of citations, were Carol Lefebvre with two papers in the top ten, totaling 1,239 citations; Shandra Protzko with a single paper in the top ten (454 citations); and Michele Tennant, who was co-author on the sixth, seventh and tenth of the top ten most highly cited papers, (394 citations). By far the most highly cited paper is the 1994 BMJ article on systematic reviews co- authored by Carol Lefebvre. Lefebvre’s two papers in the top ten established the librarian’s role in systematic reviews, as Michele Tennant’s three papers influenced thinking on the librarian’s role in the field of bioinformatics. Not only was Joanne Marshall’s Rochester Study in the top ten, but her body of work throughout her career established best practices for identifying the value of information provided by librarians. According to percentile ranking [3] in the social sciences, papers one, two and three of the top ten significantly outperformed their peer publications in citation frequency during 2014, falling into the top 1% of cited papers in their field in their first ten years after publication. These findings were verified with Thomson-Reuters’ Essential Science Indicators™. Essential Science Indicators tracks papers whose citation counts place them in the top 1% of their discipline over a ten-year period after publication. Figure 1. Citation Report of Citation Activity on RS members’ papers By default, a Web of Science Citation Report shows the latest 20 years of citations in a view. One can click on another view to see a graph with all years of citations. A Look at the Scholarly Output, continued Wheeler, Yaniv, and Fenske Hypothesis, vol. 27, no. 1, Winter 2015 5 Section members were conservative with regard to self- citation. The corpus of papers was cited 7,927 times, with a total of 446 self-citations. This comprised 5.75% of total citations, far below the 30% rate of expected self- citation suggested in earlier literature [4], or the between 10% and 20% found by a study in 2012 [5]. Each of the top ten most highly cited papers influenced the field of library and information science. Lefebvre’s paper on systematic reviews, the most highly cited article, demonstrated international influence in the citation histogram shown in Figure 3. Research Section members developed research protocols that enhanced the field of medical information science through publication. Two very interesting findings related to the top ten most highly cited articles included the amount of co-authorship done by RS members with other healthcare professionals and the amount of publications in more widely read medical journals. Five of the article titles in Figure 2 initially looked like false drops, but deeper investigation into the author strings indicated that in each case there was an RS member listed as a co-author. This finding continued throughout the set. Often section members were co-authoring with their clients on medical topics as part of the research team. An example of this collaboration was Kay Wellik’s work with Mayo Clinic Arizona neurology residents on their critical appraisal topics for grand rounds presentations [6]. She also co- authored a series in Neurologist of over 30 critical appraisal articles written by the residents. Dolores Judkins co-authored 13 of the 17 papers in Journal of Family Practice with colleagues from other health disciplines. As collaborators and co-authors, these RS Figure 2. Ten most highly cited publications authored by Research Section members, according to the Web of Science Citation Report A Citation Report shows citations for the last five years, followed by a total of all citations to a work, followed by the aver- age citations per year. The second column from the right is the citation data relevant to our discussion. A Look at the Scholarly Output, continued Wheeler, Yaniv, and Fenske Hypothesis, vol. 27, no. 1, Winter 2015 6 Figure 3. Citation histogram from Thomson-Reuters Web of Science™ Shows Carol Lefebvre’s paper on systematic reviews cited 1071 times by authors in many different Asian, Eastern and Western European, and North American countries. A Look at the Scholarly Output, continued Wheeler, Yaniv, and Fenske Hypothesis, vol. 27, no. 1, Winter 2015 7 members demonstrated their collegial relationship to other health professionals as they increased their scholarly production and expanded the corpus of our profession’s knowledge. As for collaboration with other MLA members, the study authors found that some Research Section members did co-author with fellow Section members, but there was no indication that the Section played in role in those collaborations. Limitations This study was limited to the papers authored by Research Section members that were indexed in Thomson-Reuters’ Web of Science. Since the author disambiguation tools currently available also limited this study, much of the disambiguation was done by hand or by checking websites. Timeframe was limited to the period from 1970 to April 6, 2014. To accurately reflect RS membership, the roster of members for the year 2013 - 2014 was used. This limited membership inclusion criteria to those who had paid memberships in 2014. Conclusions This brief study of the publishing habits of the Research Section revealed that Section members added to the corpus of knowledge supporting evidence-based practice. It demonstrated that RS members appear cognizant of not over utilizing self-citation. Research Section members had respectable citation rates with a few in the very highest percentile ranking and they had papers that influenced thought internationally. The most frequent publication outlets were MLA publications. Since some RS members published in medical or more general information science journals, this indicated further opportunities to take RS member research beyond JMLA to increase awareness of Section research and scholarship to the larger medical community. The authors hope that this brief study revealed some of the value to be found, as well as some of the inherent limitations, in using bibliometric analysis. References 1. Bornmann L, Leydesdorff L, Mutz R. The use of percentiles and percentile rank classes in the analysis of bibliometric data: Opportunities and limits. Journal of Informetrics. 2013;7(1):158-165. doi: 10.1016/j.joi.2012.10.001. 2. Alonso S, Cabrerizo FJ, Herrera-Viedma E, Herrera F. h-index: A review focused in its variants, computation and standardization for different scientific fields. Journal of Informetrics. 2009;3(4):273 -289. doi: 10.1016/j.joi.2009.04.001. 3. Bornmann L, Marx W. How good is research really? Measuring the citation impact of publications with percentiles increases correct assessments and fair comparisons. EMBO Reports. 2013;14(3):226-230. doi: 10.1038/embor.2013.9. 4. Seglen PO. Citation rates and journal impact factors are not suitable for evaluation of research. Acta Orthopaedica. 1998;69(3):224-229. doi: 10.3109/17453679809000920. 5. Wallace ML, Larivière V, Gingras Y. A small world of citations? The influence of collaboration networks on citation practices. PloS ONE. 2012; 2011;7 (3):e33339. doi: 10.1371/journal.pone.0033339. 6. Demaerschalk, Bart, Wingerchuk, Dean, Wellik, Kay, Budavari, Adriane, Carter, Jonathan, Dodick, David. There Is MERIT to Evidence Based Neurology Training in Residency: P01.006. Neurology. 2006;66 (5):A16. - - - Author Affiliations 1 Director, Samuel J. Wood Library & C.V. Starr Biomedical Information Center, Weill Cornell Medicine, New York, NY; tew2004@med.cornell.edu 2 Performance Data Analyst, Hospital Quality Institute, Sacramento, CA; ynoga@umich.edu 3 Associate Librarian Emeritus, John Carroll University, University Heights, OH; rfenske@jcu.edu A Look at the Scholarly Output, continued Wheeler, Yaniv, and Fenske mailto:tew2004@med.cornell.edu mailto:ynoga@umich.edu mailto:rfenske@jcu.edu