










































 

Hypothesis, vol. 27, no. 1, Winter 2015    3 

ARTICLES 
 
 

A LOOK AT THE SCHOLARLY OUTPUT OF THE MEDICAL LIBRARY 
ASSOCIATION RESEARCH SECTION 

 

Terrie R. Wheeler, AMLS1; Noga Yaniv, MSc2; and Ruth E. Fenske, PhD3 

 

The Research Section (RS) has guided the Medical 
Library Association (MLA) specifically, and the field of 
health sciences librarianship more generally, in the 
practice of seeking evidence to inform decision making 
and practice.  The best evidence comes from well-
designed research projects.  The Section has mentored 
many health sciences librarians in designing research 
projects, in conducting research according to the 
scientific method, and in publishing their results.  MLA 
has adopted a Research Agenda and it relies on the 
Section to carry out that agenda.  Further, the Research 
Section has played a role over the years in advancing 
the scholarship of the organization.  For example, the 
MLA has embraced scholarly publishing by setting high 
standards in its own publication, the Journal of the 
Medical Library Association.  Major articles in the 
journal are research based, require a structured or 
academic abstract, and generally consist of objectives, 
methods, findings, results and conclusions.  
 

Objective 
 
While the Research Section has been critical in 
advancing the scholarly communication within the field 
of health sciences librarianship, rarely has the Section 
highlighted its own role in scholarly output.  That is the 
purpose of this abbreviated study.   
 
The authors were curious to discover what scholarly 
publications trends might exist among RS members.  
Some questions to answer were: 
1) Which journals were most often selected for 

publication by RS members?  
2) Were any of the works highly cited?  
3) Did RS members engage in self-citation? 
4) Have any RS members built a body of work on a 

certain topic or topics that have affected the 
profession at large? 

5) Which RS members are most prolific? 
6) Does citation mapping indicate that some RS 

members have had a national or international 
impact? 

7) Are any RS member papers cited often enough to 
place them in the top ten percentile [1] of all 
papers published by discipline and year of 
publication? 

8) Does the Research Section encourage 

collaboration with authors not in the Section, or, do 
RS members co-author primarily with other RS 
members? 

The authors wanted to answer these questions by looking 
at the scholarly output of the RS members, and ran the 
search strategy that was created in Web of Science on 
April 6, 2014.  The search parameters included 1970 to 
date (April 6, 2014) and all document types.  The strategy 
yielded 1,100 publications by RS members. 

Methods 
 
One author downloaded the 2013 - 2014 roster of 
members of the Research Section from the MLA  
website.  The authors used surnames, first and middle 
initials when known, and organizational affiliations to 
identify the scholarly output of the Section. Thomson 
Reuters’ Core, formerly Thomson Reuters’ Web of 
Knowledge, was searched from 1970 to date.  Of the 
258 section members, 160 published at least one 
publication that was indexed by the Thomson-Reuters 
Core during this period.  As the names that were 
downloaded from the section membership file did not 
always include middle initials, the authors took extra 
care to review articles by subject matter or affiliation to 
ensure that as often as possible they had selected the 
correct records for inclusion.  The authors checked 
webpages and curriculum vitae as necessary to ensure 
accuracy as well as used the name disambiguation tool 
in Thomson-Reuters’ Core, along with checking certain 
records for full names in PubMed.  After executing the 
search strategies on each current member in the 
section, the search strategies were combined into a 
single set and the final search strategy was run.  
 

Findings 
 
Authors ran a citation report on the resulting citations 
(Figure 1).  There were 142 papers written in 1995 but 
in 1996 there was a drop to 18 papers.  Beginning in 
1996 there was an increasing trajectory of papers over 
time.  Closer analysis of the 1995 papers reveals that 
Patricia Reavis, Barbara Epstein and Lynn Piotrowicz 
authored 115 book reviews that appeared in the 
December, 1995 issue of Psychiatric Services, which 
accounted for this unusually large productivity in one 
year.  

Peer-Reviewed 



 

Hypothesis, vol. 27, no. 1, Winter 2015    4 

The citations to RS member articles in each year also 
reveal a steady upward trend which is generally found in 
sets of papers.  There was a marked increase in 2012 
(875) and 2013 (1078).  This was a combination of more 
papers published in 2012 and of certain papers, such as 
papers 1, 2 and 3 in the top ten highly cited papers 
(Figure 2) having a high citation rate during that period.  
The times cited for the entire set in Figure 1 is 7,580, or 
approximately 194.36 citations per year.  Papers 
published by RS members that are indexed in Web of 
Science have an h-index of 36.  An overview of the h-
index and the reliability of such indices can be found in an 
article by Alonso et al [2]. 

To find the ten journals in which members of the 
Research Section most frequently published, we 
analyzed the set by source title.  The top journals were: 

 Journal of the Medical Library Association (222 
papers),  

 The Bulletin of the Medical Library Association 
[precedes the JMLA] (148 papers)  

 Library Journal (139 papers).  Several members have 
authored book reviews for Library Journal, which 
accounted for this higher number.  

 Psychiatric Services (115 book reviews) 

 Neurologist (33 papers) 

 Library Trends (22 papers) 

 The Journal of Family Practice (17 papers)  

 Online CDROM Review (15 reviews) 

 Reference and User Services Quarterly, formerly RQ 
(15 papers) 

 College and Research Libraries (14 papers) 

The Web of Science subject categories most frequently 
used to identify papers in this set were as follows:  
Information Science and Library Science (718 papers), 
Public Environmental and Occupational Health (134 
papers), Health Policy Services (126 papers), Psychiatry 

(123 papers), and Computer Science Information 
Systems (79 papers). 
 
The nine most prolific members, according to the Web of 
Science data, during that time frame were Barbara 
Epstein (129 papers), Margaret Henderson (46 papers), 
Linda Smith (43 papers), Kay Wellik (43 papers), Joanne 
Gard Marshall (38 papers), Pru Dalrymple (35 papers), 
Lucretia McClure (34 papers), Ellen Detlefsen (32 
papers), and Scott Plutchak (31 papers).   

The Research Section’s most influential authors, 
according to number of citations, were Carol Lefebvre 
with two papers in the top ten, totaling 1,239 citations; 
Shandra Protzko with a single paper in the top ten (454 
citations); and Michele Tennant, who was co-author on 
the sixth, seventh and tenth of the top ten most highly 
cited papers, (394 citations).  By far the most highly cited 
paper is the 1994 BMJ article on systematic reviews co-
authored by Carol Lefebvre.  Lefebvre’s two papers in the 
top ten established the librarian’s role in systematic 
reviews, as Michele Tennant’s three papers influenced 
thinking on the librarian’s role in the field of 
bioinformatics.  Not only was Joanne Marshall’s 
Rochester Study in the top ten, but her body of work 
throughout her career established best practices for 
identifying the value of information provided by librarians.  
  
According to percentile ranking [3] in the social sciences, 
papers one, two and three of the top ten significantly 
outperformed their peer publications in citation frequency 
during 2014, falling into the top 1% of cited papers in their 
field in their first ten years after publication.  These 
findings were verified with Thomson-Reuters’ Essential 
Science Indicators™.   Essential Science Indicators 
tracks papers whose citation counts place them in the top 
1% of their discipline over a ten-year period after 
publication. 

Figure 1. Citation Report of Citation Activity on RS members’ papers 
By default, a Web of Science Citation Report shows the latest 20 years of citations in a view.  One can click on another 
view to see a graph with all years of citations.  

A Look at the Scholarly Output, continued Wheeler, Yaniv, and Fenske 



 

Hypothesis, vol. 27, no. 1, Winter 2015    5 

Section members were conservative with regard to self-
citation.  The corpus of papers was cited 7,927 times, 
with a total of 446 self-citations.  This comprised 5.75% of 
total citations, far below the 30% rate of expected self-
citation suggested in earlier literature [4], or the between 
10% and 20% found by a study in 2012 [5].  

Each of the top ten most highly cited papers influenced 
the field of library and information science.  Lefebvre’s 
paper on systematic reviews, the most highly cited article, 
demonstrated international influence in the citation 
histogram shown in Figure 3. Research Section members 
developed research protocols that enhanced the field of 
medical information science through publication.   
 
Two very interesting findings related to the top ten most 
highly cited articles included the amount of co-authorship 

done by RS members with other healthcare professionals 
and the amount of publications in more widely read 
medical journals.  Five of the article titles in Figure 2 
initially looked like false drops, but deeper investigation 
into the author strings indicated that in each case there 
was an RS member listed as a co-author.  This finding 
continued throughout the set.  Often section members 
were co-authoring with their clients on medical topics as 
part of the research team.  An example of this 
collaboration was Kay Wellik’s work with Mayo Clinic 
Arizona neurology residents on their critical appraisal 
topics for grand rounds presentations [6].  She also co-
authored a series in Neurologist of over 30 critical 
appraisal articles written by the residents.  Dolores 
Judkins co-authored 13 of the 17 papers in Journal of 
Family Practice with colleagues from other health 
disciplines.  As collaborators and co-authors, these RS 

Figure 2. Ten most highly cited publications authored by Research Section members, according to 
the Web of Science Citation Report 
A Citation Report shows citations for the last five years, followed by a total of all citations to a work, followed by the aver-
age citations per year.  The second column from the right is the citation data relevant to our discussion.  

A Look at the Scholarly Output, continued Wheeler, Yaniv, and Fenske 



 

Hypothesis, vol. 27, no. 1, Winter 2015    6 

Figure 3. Citation histogram from Thomson-Reuters Web of Science™ 
Shows Carol Lefebvre’s paper on systematic reviews cited 1071 times by authors in many different Asian, Eastern and 
Western European, and North American countries.  

A Look at the Scholarly Output, continued Wheeler, Yaniv, and Fenske 



 

Hypothesis, vol. 27, no. 1, Winter 2015    7 

members demonstrated their collegial relationship to 
other health professionals as they increased their 
scholarly production and expanded the corpus of our 
profession’s knowledge.  As for collaboration with other 
MLA members, the study authors found that some 
Research Section members did co-author with fellow 
Section members, but there was no indication that the 
Section played in role in those collaborations.   

Limitations 
 
This study was limited to the papers authored by 
Research Section members that were indexed in 
Thomson-Reuters’ Web of Science.  Since the author 
disambiguation tools currently available also limited this 
study, much of the disambiguation was done by hand or 
by checking websites.   Timeframe was limited to the 
period from 1970 to April 6, 2014.  To accurately reflect 
RS membership, the roster of members for the year 2013 
- 2014 was used.  This limited membership inclusion 
criteria to those who had paid memberships in 2014.  
 

Conclusions 

This brief study of the publishing habits of the Research 
Section revealed that Section members added to the 
corpus of knowledge supporting evidence-based practice.  
It demonstrated that RS members appear cognizant of 
not over utilizing self-citation.  Research Section 
members had respectable citation rates with a few in the 
very highest percentile ranking and they had papers that 
influenced thought internationally.  The most frequent 
publication outlets were MLA publications.  Since some 
RS members published in medical or more general 
information science journals, this indicated further 
opportunities to take RS member research beyond JMLA 
to increase awareness of Section research and 
scholarship to the larger medical community.  The 
authors hope that this brief study revealed some of the 
value to be found, as well as some of the inherent 
limitations, in using bibliometric analysis.   

References 

1. Bornmann L, Leydesdorff L, Mutz R. The use of 
percentiles and percentile rank classes in the 
analysis of bibliometric data: Opportunities and 
limits. Journal of Informetrics. 2013;7(1):158-165. doi: 
10.1016/j.joi.2012.10.001. 

2. Alonso S, Cabrerizo FJ, Herrera-Viedma E, Herrera 
F. h-index: A review focused in its variants, 
computation and standardization for different 
scientific fields. Journal of Informetrics. 2009;3(4):273
-289. doi: 10.1016/j.joi.2009.04.001. 

3. Bornmann L, Marx W. How good is research really? 
Measuring the citation impact of publications with 
percentiles increases correct assessments and fair 
comparisons. EMBO Reports. 2013;14(3):226-230. 
doi: 10.1038/embor.2013.9. 

4. Seglen PO. Citation rates and journal impact factors 
are not suitable for evaluation of research. Acta 
Orthopaedica. 1998;69(3):224-229. doi: 
10.3109/17453679809000920. 

5. Wallace ML, Larivière V, Gingras Y. A small world of 
citations? The influence of collaboration networks on 
citation practices. PloS ONE. 2012; 2011;7
(3):e33339. doi: 10.1371/journal.pone.0033339. 

6. Demaerschalk, Bart, Wingerchuk, Dean, Wellik, Kay, 
Budavari, Adriane, Carter, Jonathan, Dodick, David.  
There Is MERIT to Evidence Based Neurology 
Training in Residency: P01.006.  Neurology. 2006;66
(5):A16. 

- - - 

Author Affiliations 

1 Director, Samuel J. Wood Library & C.V. Starr 
Biomedical Information Center, Weill Cornell Medicine, 
New York, NY; tew2004@med.cornell.edu 

2 Performance Data Analyst, Hospital Quality Institute, 
Sacramento, CA; ynoga@umich.edu  

3 Associate Librarian Emeritus, John Carroll University, 
University Heights, OH; rfenske@jcu.edu 

A Look at the Scholarly Output, continued Wheeler, Yaniv, and Fenske 

mailto:tew2004@med.cornell.edu
mailto:ynoga@umich.edu
mailto:rfenske@jcu.edu

