







































An examination of research topics in BMLA and JMLA - Marton 

 

   

Hypothesis, vol. 28, no. 1, Fall 2016                                                                      6                                                                                

 

 

TITLE 

An examination of research topics in the Journal of the Medical Library Association and 

the Bulletin of the Medical Library Association: quantifying the importance of research 

to medical librarians over time 

 

AUTHOR 

Christine Marton, PhD1 

 

INTRODUCTION 

Historically, librarians have undertaken observational studies of their users, specifically, of their 

preferences for and uses of specific information source formats, online databases, and 

information technologies. User satisfaction studies of library collections and services have also 

been popular. With the advent of desktop computing, computer networks and online databases 

in the latter half of the twentieth century, research studies of all kinds and across all disciplines 

have become more numerous. In today’s information society, research has grown into an integral 

activity no longer restricted to academics and research institutes. How has this trend affected 

the profession of health sciences librarianship? Has the number of research projects in this 

discipline increased in recent years? What study designs are being utilized? 

As the foremost professional association for health sciences librarians and informationists in the 

United States of America, the Medical Library Association (MLA) sets priorities for its membership 

and more broadly, for the profession. In recent decades, MLA has begun to prioritize research.   

In 1995, MLA developed a research policy statement, Using Scientific Evidence to Improve 

Information Practice, which called on health sciences librarians to be proactive in creating, 

managing, and using scientific evidence [1]. In 2008, MLA published The Research Imperative [2]. 

Based on semi-structured interviews with key stakeholders, including editors of the Journal of 

the Medical Library Association, the research policy emphasizes evidence-based library and 

information practice. Six themes were identified: creation of a research culture, domains of 

research, research skills set, roles of stakeholders, challenges, and measurement of progress.        

In 2008 and 2011 the Research Agenda Committee of the Research Section of MLA identified the 



An examination of research topics in BMLA and JMLA - Marton 

 

   

Hypothesis, vol. 28, no. 1, Fall 2016                                                                      7                                                                                

 

fifteen most important researchable questions facing our profession. [3, 4] More recently teams 

have been conducting systematic reviews to determine the state of knowledge in these 15 areas. 

[5,6]. Clearly, there is increasing recognition within the profession of health sciences librarianship 

of the importance of the role of librarians as researchers and evidence based practice. 

The Journal of the Medical Library Association (JMLA) is an open access, peer-reviewed journal 

published quarterly by the Medical Library Association. Current and previous full-text issues are 

freely available on PubMed Central. Previously known as the Bulletin of the Medical Library 

Association (BMLA), this journal serves as a reputable publication of research studies about 

health sciences librarianship. Both major citation indexes, Elsevier’s SCImago and Thomson 

Reuters’ Journal Citation Reports rank the JMLA higher than comparable health sciences librarian 

journals in their respective lists of journal impact factors for library and information sciences 

journals. Of the 209 journals in the Library & Information Science (LIS) subject category indexed 

by the Scopus database, the SCImago Journal & Country Rank ranked the JMLA in thirty-eighth 

position in 2015 with an SJR of 0.726, ahead of Health Information and Libraries Journal (43; 

0.650); Evidence Based Library and Information Practice (56, 0.586); Medical Reference Services 

Quarterly (65; 0.500); and Journal of Hospital Librarianship (125; 0.217). [9] Of the 86 journals in 

the LIS subject category indexed in the Journal Citation Reports database in 2015, the JMLA was 

ranked 37 with a journal impact factor of 1.084, while Health Information and Libraries Journal 

was ranked 50, with a journal impact factor of 0.712. The other aforementioned health sciences 

librarianship journals are not indexed in the Journal Citation Reports database. Clearly, the JMLA 

is the foremost publication for health sciences librarianship; articles published in this journal 

serve as a strong representation of the published research in this discipline. [7-9] 

 

OBJECTIVE                   

This quantitative study seeks to determine whether health sciences librarians have undertaken 

more research activities in recent years by measuring the occurrences of research-related words 

in all issues of the peer-reviewed journal publication of the Medical Library Association and 

comparing the percentage of occurrences (prevalence) between the earlier publication, 

Bulletin of the Medical Library Association and the current publication, Journal of the Medical 

Library Association.     

                                                                                                                   



An examination of research topics in BMLA and JMLA - Marton 

 

   

Hypothesis, vol. 28, no. 1, Fall 2016                                                                      8                                                                                

 

LITERATURE REVIEW 

Several studies on librarianship research have been conducted by library science academics and 

academic librarians. These studies generally utilize a restricted time period, such as one year 

or five years, and examine a small number of library and information science (LIS) journals. 

Content analysis and bibliometrics are popular study designs.  

In their content analysis of librarianship research, Koufogiannakis, Slater and Crumley examined 

the LIS journal literature for a one-year period in 2001. Of the 217 LIS journals reviewed, they 

included 107 journals in their study, of which 91 contained relevant data. Of the 2,664 journal 

articles examined, they classified 807 as research articles. The top LIS journals for research in 

2001 were the Journal of the American Society of Information Science and Technology (JASIST); 

Scientometrics; Information Processing & Management; College & Research Libraries; Bulletin of 

the Medical Library Association (tied with Journal of Library Administration); Libraries and 

Culture; Journal of Documentation; and Journal of Information Science (tied with Journal of 

Academic Librarianship). According to the authors’ classification scheme of six subject domains, 

“information access and retrieval” contained the greatest number of research articles and the 

most frequently published type of research was descriptive research. Non-experimental 

research, such as surveys, was found to be much more prevalent than experimental research. 

[10] 

Slutsky and Aytac reviewed science librarianship research from 2008–2012 in four LIS journals: 

Health Information & Libraries Journal (HILJ); Journal of the Medical Library Association (JMLA); 

Issues in Science & Technology Librarianship (ISTL); and Science & Technology Libraries (STL).  

They analyzed the texts of 574 articles and classified them as either research or non-research. 

Non-research articles were removed from the analysis. Slightly more than half of all articles were 

classified as research (n=311; 54.2%). Study variables included authorship, affiliation, type of 

research, research topic, and data collection and data analysis techniques. Bibliometric data 

analysis revealed that there has been a dramatic growth in research in these LIS journals for the 

years 2008-2012. The majority of research papers featured quantitative study designs; 

qualitative studies comprised only 10% of the research articles examined. Quantitative data 

analysis overwhelmingly consisted of descriptive statistics (88.7%). The most popular study 

designs were survey, content analysis, citation analysis, and interviews.  There was a significant 

difference among these four different publications with respect to the location of the study, 

context of research, research approach, and statistical analysis. As well, overall authorship was 



An examination of research topics in BMLA and JMLA - Marton 

 

   

Hypothesis, vol. 28, no. 1, Fall 2016                                                                      9                                                                                

 

highly collaborative; almost three-quarters of the research articles were written by two or more 

authors. Of interest, the JMLA contained the greatest number of research articles, followed by 

HILJ, ISTL, and STL. This finding lends support to the presumption that JMLA is a key journal, 

not only for health sciences librarianship research but for LIS research more broadly. [11] 

In a more recent study, Slutsky and Aytac conducted a bibliometric analysis of research articles 

published in the aforementioned STEM librarianship journals, ISTL and STL, over a ten-year 

period (2005-2014). They found a greater number of research articles in ISTL but higher Scopus 

citation metrics for STL. As well, the most frequent topic in STL was “bibliometrics and citation 

analysis” while in ISTL it was “libraries and librarianship,” with “library resources” as the 

foremost topic. There were more author collaborations in ISTL than STL. Of interest, JMLA was 

ranked eighth in the list of top 25 LIS journals cited in issues in ISTL and thirteenth in the top 

25 LIS journals cited in STL. [11] Clearly, JMLA is regarded as a reputable publication by science 

and technology librarians. [12] 

From these three bibliometric and citation analysis studies of the LIS journal literature, it is 

apparent that BMLA and its successor, JMLA, are well regarded research publications in the 

field of library and information science. Analyzing the prevalence of research studies and study 

designs in these two publications is a valid method for determining whether there has been an 

increase in research productivity in health sciences librarianship over time as well as an 

increase in the diversity of study designs beyond surveys and interviews. 

 

METHOD  

To draw this comparison, the frequency (counts) and prevalence (percentages) of articles including 

words and phrases that are commonly used to describe research studies and methodologies were 

obtained from all issues of the BMLA, which was published from 1911 to 2001 (volumes 1-89), and 

its successor, JMLA, which has been published since 2002 (volumes 90-104). All issues are available 

online in full text from PubMed Central (PMC). These two journals were queried using the search 

interface on the PMC webpage for the journals’ archive (www.ncbi.nlm.nih.gov/pmc/ journals/93/).  

  

http://www.ncbi.nlm.nih.gov/


An examination of research topics in BMLA and JMLA - Marton 

 

   

Hypothesis, vol. 28, no. 1, Fall 2016                                                                      10                                                                                

 

Search statements began with the first topic, which is the name of the journal: Bulletin of the 

Medical Library Association and its successor, Journal of the Medical Library Association.              

The journal name was entered as the first search term in the search box: "Bulletin of the Medical 

Library Association"[Journal] for one set of searches and "Journal of the Medical Library 

Association : JMLA"[Journal] for the other set of searches. Next, the first search term was paired 

with a second concept, beginning with the search term research, in double quotation marks, 

which was added to subsequent searches. For the second concept, the search term was entered 

in separate searches with either the field limit [All Fields] or the field limits [Title] OR [Abstract]. 

Thus, two searches were conducted for each second concept. The [All Fields] searches are broad: 

they capture the search terms as they appear in any part of the full text documents. The [Title] 

OR [Abstract] field limits restrict the searches to those two bibliographic fields, thereby improving 

the precision of the searches.  

Preliminary searches were conducted in late August, 2016. The original searches and new 

searches with additional second concept terms were conducted in early October, 2016 and 

also in mid-November 2016, when the fourth and final 2016 issue of JMLA was added to the 

journal repository site on PubMed Central. Tables 1-2 in the Results section below list all of the 

search statements and their corresponding number (counts and percentages) of search results. 

The searches progressed from broad to narrow as the search terms for the second concept 

became more specific.  

Searching all issues of both BMLA and JMLA provides quantitative measures of all research-

related words.  The number of search results serves as a basic measure of the pervasiveness of 

research in this publication, and more broadly, in the field of health sciences librarianship.    

Search term frequencies and percentages can be tabulated to facilitate pairwise quantitative 

comparisons that indicate whether there has been an increase in published research activities in 

health sciences librarianship over time. The use of inferential statistics, more specifically, the 

paired t-test, enables the calculation of the statistical significance of difference in means of 

search results for JMLA in relation to its predecessor, BMLA.  

  



An examination of research topics in BMLA and JMLA - Marton 

 

   

Hypothesis, vol. 28, no. 1, Fall 2016                                                                      11                                                                                

 

The search results data were first entered into two comprehensive tables in a Microsoft Word 

document: one table for the [All Fields] searches and a second table for the [Title] OR [Abstract] 

field limited searches. All table columns were copied to an Excel spreadsheet and the 

spreadsheet was then imported to the statistical package, SPSS (Version 11). The appropriate 

inferential statistic, the paired t-test, was calculated to determine the strength and significance 

of the difference in means for the search results on research-related terms between BMLA and 

JMLA for (1) [All Fields] searches and (2) [Title] OR [Abstract] searches. Percentages, instead of 

counts, were compared to take into account the larger number of journal issues for  BMLA in 

relation to JMLA, which is a function of the greater timespan for the publication of BMLA, and 

concomitant greater number of articles. 

Unlike prior bibliometric and content analyses studies on the pervasiveness of research in the 

field of librarianship, the context of these terms was not investigated. Thus, all occurrence of 

research-related terms in BMLA and JMLA are included irrespective of whether they were 

published in research articles or appear in editorials, reviews and other non-research columns 

of BMLA and JMLA. This is an acceptable approach because the objective is not to categorize 

the topics of research, nor to investigate characteristics of authorship or citation patterns, but 

solely to determine whether an increase in (1) research and (2) study designs has occurred in 

the profession of health sciences librarianship from 2002 onward, when BMLA was renamed 

JMLA. Restricting the searches to the [Title] OR [Abstract] fields improves, to some extent, the 

relevance of search results because of the greater specificity and relationship to topicality than 

the broad [All Fields] searches.  

 

RESULTS 

The initial search, “Bulletin of the Medical Library Associaton”[Journal] retrieved 7256 results 

while the initial search, "Journal of the Medical Library Association : JMLA"[Journal],  retrieved 

1498 search results. The greater publication timespan of BMLA is accountable for this large 

difference in number of search results on journal title. 

  



An examination of research topics in BMLA and JMLA - Marton 

 

   

Hypothesis, vol. 28, no. 1, Fall 2016                                                                      12                                                                                

 

 
GENERAL RESEARCH SEARCH TERMS 

 
 

SR PMC search statement: BMLA # % PMC  search statement: JMLA # % 

1 

"Bulletin of the Medical 
Library Association"[Journal] 
Alternatively:  
"Bulletin of the Medical 
Library Association"[Journal] 
OR "Bull Med Libr 
Assoc"[Journal] 

7256 100.0 

"Journal of the Medical 
Library Association : 
JMLA"[Journal] 
Alternatively:  
"Journal of the Medical 
Library Association"[Journal] 
OR "J Med Libr 
Assoc"[Journal] 

1498 100.0 

2 
"Bulletin of the Medical 
Library Association"[Journal] 
AND ("research"[All Fields]) 

2941 40.5 

("Journal of the Medical 
Library Association : 
JMLA"[Journal]) AND 
("research"[All Fields]) 

1101 73.5 

3 

"Bulletin of the Medical 
Library Association"[Journal] 
AND ("study"[All Fields] OR 
"studies"[All Fields]) 

2945 40.6 

("Journal of the Medical 
Library Association : 
JMLA"[Journal]) AND 
("study"[All Fields] OR 
"studies"[All Fields]) 

996 66.5 

4 

"Bulletin of the Medical 
Library Association"[Journal] 
AND ("study design"[All 
Fields]) 

32 0.4 

("Journal of the Medical 
Library Association : 
JMLA"[Journal]) AND 
("study design"[All Fields]) 

75 5.0 

 

Table 1a:  Number of search results from PubMed Central queries of BMLA and JMLA,  
All Fields queries – general research search terms 

 

In the following sub-sections of the Results section, the second concept, research-related, is 

operationalized as search terms for quantitative study designs (Table 2a,b); systematic review 

and meta-analysis (Table 3a,b);  mixed-method study designs (Table 4a,b), and qualitative study 

designs (Table 5a,b).  

  



An examination of research topics in BMLA and JMLA - Marton 

 

   

Hypothesis, vol. 28, no. 1, Fall 2016                                                                      13                                                                                

 

 

When the search term "research" is added and the [All Fields] field limit is employed, the number 

of search results is somewhat more than halved to 2941 search results for the Bulletin of the 

Medical Library Association and 1101 search results for the Journal of the Medical Library 

Association. More importantly, the percentage is 40.5 for BMLA and research versus 73.5 for 

JMLA and research, almost a doubling of term occurrences. (Table 1a) For the comparable [Title] 

OR [Abstract] searches, the increase is even more dramatic: 4.6% for BMLA and research versus 

18.8% for JMLA and research, a four-fold increase in term occurrence (Table 1b). 

For this set of searches on general research terms, the mean number of search results for All Fields 

BMLA searches is 27.2 while the mean number of searches for All Fields JMLA searches is 48.3.  

For Title OR Abstract searches, the mean number of search results for BMLA searches is 3.3 while 

for JMLA searches it is 13.1. Thus, there is an increase over time for occurrence of general 

research terms. 

 

SR PMC search statement: BMLA # % PMC  search statement: JMLA # % 

1 
"Bulletin of the  
Medical Library 
Association"[Journal] 

7256 100.0 
"Journal of the Medical Library 
Association : JMLA"[Journal] 

1498 100.0 

2 

"Bulletin of the  
Medical Library 
Association"[Journal] AND 
((“research"[Title] OR 
"research"[Abstract])) 

332 4.6 

("Journal of the Medical Library 
Association : JMLA"[Journal]) 
AND ((“research"[Title] OR 
"research"[Abstract])) 

281 18.8 

3 

"Bulletin of the  
Medical Library 
Association"[Journal] AND 
((“study"[Title] OR 
"study"[Abstract]) OR 
("studies"[Title] OR 
“studies"[Abstract])) 

379 5.2 

("Journal of the Medical Library 
Association : JMLA"[Journal]) 
AND ((“study"[Title] OR 
"study"[Abstract]) OR 
("studies"[Title] OR 
“studies"[Abstract])) 

301 20.1 

4 

"Bulletin of the Medical 
Library Association"[Journal] 
AND (("study design"[Title] 
OR "study design"[Abstract]) 
OR ("study designs"[Title] OR 
"study designs"[Abstract])) 

2 0 

("Journal of the Medical Library 
Association : JMLA"[Journal]) 
AND (("study design"[Title] OR 
"study design"[Abstract]) OR 
("study designs"[Title] OR  
"study designs"[Abstract])) 

5 0.3 

 

Table 1b:  Number of search results from PubMed Central queries of BMLA and JMLA,  
Title OR Abstract queries – general research search terms 



An examination of research topics in BMLA and JMLA - Marton 

 

   

Hypothesis, vol. 28, no. 1, Fall 2016                                                                      14                                                                                

 

 

QUANTITATIVE RESEARCH SEARCH TERMS 
 

 

SR PMC search statement: BMLA # % 
 
PMC  search statement: JMLA 
 

# % 

1 

"Bulletin of the Medical Library 
Association"[Journal] AND 
("quantitative"[All Fields] OR 
"quantify"[All Fields]) 

286 3.9 

("Journal of the Medical Library 
Association : JMLA"[Journal]) 
AND ("quantitative"[All Fields] 
OR "quantify"[All Fields]) 

182 12.2 

2 

"Bulletin of the Medical 
Library Association"[Journal] 
AND ("quantitative 
design"[All Fields]) 

0 0 

("Journal of the Medical Library 
Association : JMLA"[Journal]) 
AND ("quantitative design"[All 
Fields]) 

0 0 

3 

"Bulletin of the Medical 
Library Association"[Journal] 
AND ("survey"[All Fields] OR 
"surveys"[All Fields]) 

1615 22.3 

("Journal of the Medical Library 
Association : JMLA"[Journal]) 
AND ("survey"[All Fields] OR 
"surveys"[All Fields]) 

609 40.7 

4 

"Bulletin of the Medical 
Library Association"[Journal] 
AND ("questionnaire"[All 
Fields] OR 
"questionnaires"[All Fields]) 

655 9.0 

("Journal of the Medical Library 
Association : JMLA"[Journal]) 
AND ("questionnaire"[All Fields] 
OR "questionnaires"[All Fields]) 

215 14.4 

5 

"Bulletin of the Medical 
Library Association"[Journal] 
AND ("bibliometric"[All 
Fields]) 

49 0.7 
("Journal of the Medical Library 
Association : JMLA"[Journal]) 
AND ("bibliometric"[All Fields]) 

80 5.3 

6 

"Bulletin of the Medical 
Library Association"[Journal] 
AND ("citation analysis"[All 
Fields]) 

82 1.1 

("Journal of the Medical Library 
Association : JMLA"[Journal]) 
AND ("citation analysis"[All 
Fields]) 

85 5.7 

7 

"Bulletin of the Medical 
Library Association"[Journal] 
AND ("content analysis"[All 
Fields]) 

30 0.4 

("Journal of the Medical Library 
Association : JMLA"[Journal]) 
AND ("content analysis"[All 
Fields]) 

46 3.1 

 
Table 2a:  Number of search results from PubMed Central queries of BMLA and JMLA,  

All Fields queries –quantitative research search terms 
  



An examination of research topics in BMLA and JMLA - Marton 

 

   

Hypothesis, vol. 28, no. 1, Fall 2016                                                                      15                                                                                

 

 
The second set of research terms examined pertain to quantitative study designs. The search 

results are presented in Table 2a and Table 2b. According to Creswell, a key characteristic of 

quantitative study is determinism. “Examining the relationships between and among variables 

is central to answering questions and hypotheses through surveys and experiments. The 

reduction to a parsimonious set of variables, tightly controlled through design or statistical 

analysis, provides measures or observations for testing a theory. Objective data results from 

empirical observations and measures. Validity and reliability of scores on instruments, 

additional standards for making knowledge claims, lead to meaningful interpretations of 

data.”[13] 

In Tables 2a and 2b, several quantitative study designs are examined in relation to the 

occurrence of appropriate search terms in BMLA and JMLA. To begin, general terms, quantify 

and quantitative design, are utilized. Then, the searches are narrowed to specific quantitative 

methods, such as survey and questionnaire. Lastly, even narrower terms are employed, such 

as citation analysis. Although the term quantitative design is not to be found in the search 

results, there are many search results for surveys and questionnaires, and smaller numbers for 

bibliometric(s), content analysis(es) and citation analysis(es).  

For each quantitative method, the percentage of search results for JMLA exceeds that of BMLA. 

The mean number of search results for BMLA All Fields searches on this set of quantitative 

study designs is 5.4 while the mean number of search results for JMLA All Fields searches is 

11.6. For the Title OR Abstract searches, BMLA searches have a mean of 0.8 while JMLA 

searches have a mean of 2.4. These statistical findings demonstrate an increase in the number 

of occurrences of quantitative research-related terms over time. 

 



An examination of research topics in JMLA and BMLA - Marton 

 

   

Hypothesis, vol. 28, no. 1, Fall 2016                                                                      16                                                                                

 

 
 

SR 
 

PMC search statement: BMLA 
 

# % 
 

PMC  search statement: JMLA 
 

# % 

1 

"Bulletin of the Medical Library 
Association"[Journal] AND 
(("quantitative"[Title] OR 
"quantitative"[Abstract]) OR 
("quantify"[Title] OR 
“quantify"[Abstract])) 

27 0.4 

("Journal of the Medical Library 
Association : JMLA"[Journal]) AND 
(("quantitative"[Title] OR 
"quantitative"[Abstract]) OR 
("quantify"[Title] OR 
“quantify"[Abstract])) 

20 1.3 

2 

"Bulletin of the Medical Library 
ssociation"[Journal] AND  
(("quantitative design"[Title] OR 
"quantitative design"[Abstract]) 
OR ("quantitative designs"[Title] 
OR “quantitative 
designs"[Abstract])) 

0 0 

("Journal of the Medical Library 
Association : JMLA"[Journal]) AND 
(("quantitative design"[Title] OR 
"quantitative design"[Abstract]) OR 
("quantitative designs"[Title] OR 
“quantitative designs"[Abstract])) 

0 0 

3 

"Bulletin of the Medical Library 
Association"[Journal] AND 
(("survey"[Title] OR 
"survey"[Abstract]) OR 
("surveys"[Title] OR 
“surveys"[Abstract])) 

262 3.6 

("Journal of the Medical Library 
Association : JMLA"[Journal]) AND 
(("survey"[Title] OR 
"survey"[Abstract]) OR 
("surveys"[Title] OR 
“surveys"[Abstract])) 

147 9.8 

4 

"Bulletin of the Medical Library 
Association"[Journal] AND 
(("questionnaire"[Title] OR 
"questionnaire"[Abstract]) OR 
("questionnaires"[Title] OR 
“questionnaires"[Abstract])) 

86 1.2 

("Journal of the Medical Library 
Association : JMLA"[Journal]) AND 
(("questionnaire"[Title] OR 
"questionnaire"[Abstract]) OR 
("questionnaires"[Title] OR 
“questionnaires"[Abstract])) 

34 2.3 

5 

"Bulletin of the Medical Library 
Association"[Journal] AND 
(("bibliometric"[Title] OR 
"bibliometric"[Abstract]) OR 
("bibliometrics"[Title] OR 
“bibliometrics"[Abstract])) 

17 0.2 

("Journal of the Medical Library 
Association : JMLA"[Journal]) AND 
(("bibliometric"[Title] OR 
"bibliometric"[Abstract]) OR 
("bibliometrics"[Title] OR 
“bibliometrics"[Abstract])) 

20 1.3 

6 

"Bulletin of the Medical Library 
Association"[Journal] AND 
(("citation anaysis"[Title] OR 
"citation analysis"[Abstract]) OR 
("citation analyses"[Title] OR 
“citation analyses"[Abstract])) 

13 0.2 

("Journal of the Medical Library 
Association : JMLA"[Journal]) AND 
(("citation analysis"[Title] OR "citation 
analysis"[Abstract]) OR ("citation 
analyses"[Title] OR “citation 
analyses"[Abstract])) 

28 1.7 

7 

"Bulletin of the Medical Library 
Association"[Journal] AND 
(("content analysis"[Title] OR 
"content analysis"[Abstract]) OR 
("content analyses"[Title] OR 
“content analyses"[Abstract])) 

7 0.1 

("Journal of the Medical Library 
Association : JMLA"[Journal]) AND 
(("content analysis"[Title] OR 
"content analysis"[Abstract]) OR 
("content analyses"[Title] OR 
“content analyses"[Abstract])) 

9 0.6 

 
Table 2b:  Number of search results from PubMed Central queries of BMLA and JMLA,  

Title OR Abstract queries –quantitative research search terms 



An examination of research topics in JMLA and BMLA - Marton 

 

   

Hypothesis, vol. 28, no. 1, Fall 2016                                                                      17                                                                                

 

 
SYSTEMATIC REVIEWS AND META-ANALYSES 

A systematic review provides a comprehensive analysis of all existing primary studies on a very 

well-defined research question. It evaluates the methods employed in these studies, summarizes 

the results, presents important findings, identifies reasons for differences in findings across 

studies, and identifies the limitations of current knowledge. Combining the results mathematically 

through the use of statistical methods that sumarize all of the findings from the primary studies 

is referred to as a meta-analysis. [14] 

 

SR PMC search statement: BMLA # % PMC  search statement: JMLA # % 

1 

"Bulletin of the Medical Library 

Association"[Journal] AND 

("systematic review"[All Fields] 

OR "systematic reviews"[All 

Fields]) 

19 0.3 

("Journal of the Medical Library 

Association : JMLA"[Journal]) 

AND ("systematic review"[All 

Fields] OR "systematic 

reviews"[All Fields]) 

287 19.2 

2 

"Bulletin of the Medical Library 

Association"[Journal] AND 

("meta analysis"[All Fields] OR 

"meta analyses"[All Fields]) 

31 0.4 

("Journal of the Medical Library 

Association : JMLA"[Journal]) 

AND ("meta analysis"[All Fields] 

OR "meta analyses"[All Fields]) 

95 6.3 

 

Table 3a:  Number of search results from PubMed Central queries of BMLA and JMLA,  
All Fields queries – systematic review and meta-analysis search terms 

 

 

For the All Fields searches of this set of research terms, the mean value of BMLA searches is 0.35 

while the mean value of JMLA searches is 12.8. For the Title OR Abstract searches, the mean value 

of BMLA searches is 0.1 while the mean value of JMLA searches is 1.4.  These statistical findings 

demonstrate an increase in the number of occurrences of these two research-related terms: 

systematic review(s) and meta-analysis(es) over time. 

  



An examination of research topics in JMLA and BMLA - Marton 

 

   

Hypothesis, vol. 28, no. 1, Fall 2016                                                                      18                                                                                

 

 

SR PMC search statement: BMLA # % PMC  search statement: JMLA # % 

1 

"Bulletin of the Medical Library 

Association"[Journal] AND 

(("systematic review"[Title] OR 

"systematic review"[Abstract]) OR 

("systematic reviews"[Title] OR 

“systematic reviews"[Abstract])) 

5 0.1 

("Journal of the Medical Library 

Association : JMLA"[Journal]) AND 

(("systematic review"[Title] OR 

"systematic review"[Abstract]) OR 

("systematic reviews"[Title] OR 

“systematic reviews"[Abstract])) 

35 2.3 

2 

"Bulletin of the Medical Library 

Association"[Journal] AND 

(("meta analysis"[Title] OR "meta 

analysis"[Abstract]) OR ("meta 

analyses"[Title] OR “meta 

analyses"[Abstract])) 

6 0.1 

("Journal of the Medical Library 

Association : JMLA"[Journal]) AND 

(("meta analysis"[Title] OR "meta 

analysis"[Abstract]) OR ("meta 

analyses"[Title] OR “meta 

analyses"[Abstract])) 

7 0.5 

 

Table 3b:  Number of search results from PubMed Central queries of BMLA and JMLA,  
Title OR Abstract queries – systematic review and meta-analysis search terms 
 

 
 
MIXED METHODS RESEARCH 

Mixed methods study designs include both quantitative and qualitative methods. Either the 

quantitative or the qualitative component can be dominant, or they can co-exist equally. Creswell 

provides a useful matrix that illustrates the four decisions required to select a mixed methods 

approach. The use of a theoretical framework; the implementation sequence of quantitative and 

qualitative data collection; the priority given to quantitative and qualitation data collection and 

analysis, and the integration of quantitative and qualitative findings are key considerations for 

mixed methods designs. [15] 

  



An examination of research topics in JMLA and BMLA - Marton 

 

   

Hypothesis, vol. 28, no. 1, Fall 2016                                                                      19                                                                                

 

 

SR PMC search statement: BMLA # % PMC  search statement: JMLA # % 

1 

"Bulletin of the Medical Library 

Association"[Journal] AND 

("mixed method"[All Fields] OR 

"mixed methods"[All Fields]) 

0 0 

("Journal of the Medical Library 

Association : JMLA"[Journal]) 

AND ("mixed method"[All Fields] 

OR "mixed methods"[All Fields]) 

21 1.4 

2 

"Bulletin of the Medical Library 

Association"[Journal] AND 

("triangulation"[All Fields]) 

2 0 

("Journal of the Medical Library 

Association : JMLA"[Journal]) 

AND ("triangulation"[All Fields]) 

17 1.1 

 
Table 4a:  Number of search results from PubMed Central queries of BMLA and JMLA,  

All Fields queries – mixed methods search terms 
 
 

SR PMC search statement: BMLA # % PMC  search statement: JMLA # % 

1 

"Bulletin of the Medical Library 

Association"[Journal] AND 

(("mixed method"[Title] OR 

"mixed Method"[Abstract]) OR 

("mixed methods"[Title] OR 

"mixed methods"[Abstract])) 

0 0 

("Journal of the Medical Library 
Association : JMLA"[Journal]) AND 
(("mixed method"[Title] OR  
"mixed method"[Abstract]) OR 
("mixed methods"[Title] OR 
 "mixed methods"[Abstract])) 

5 0.3 

2 

"Bulletin of the Medical Library 

Association"[Journal] AND 

(("triangulation"[Title] OR 

"triangulation"[Abstract]) OR 

("triangulate"[Title] OR 

"triangulate"[Abstract])) 

0 0 

("Journal of the Medical Library 

Association : JMLA"[Journal]) AND 

(("triangulation"[Title] OR 

"triangulation"[Abstract]) OR 

("triangulate"[Title] OR 

"triangulate"[Abstract])) 

2 0.1 

 

Table 4b:  Number of search results from PubMed Central queries of BMLA and JMLA,  
Title OR Abstract queries – mixed methods search terms 

  



An examination of research topics in JMLA and BMLA - Marton 

 

   

Hypothesis, vol. 28, no. 1, Fall 2016                                                                      20                                                                                

 

QUALITATIVE RESEARCH SEARCH TERMS 

Creswell characterizes qualitative research as broad, holistic, reflective and interpretative. [16] 

For this fifth and final set of searches on research-related terms in BMLA and JMLA, general terms 

for qualitative research, qualitative and qualitative design, are entered first, followed by more 

specific search terms for individual qualitative approaches: interview(s), focus group(s), critical 

incident(s), and phenomenology. For the All Fields searches, the mean value for BMLA searches 

is 2.3 while the mean value for JMLA searches is 8.3, an almost four-fold difference. For the Title 

OR Abstract searches, the mean value for BMLA searches is 0.1 while the mean value for JMLA 

searches is 0.85, an eight-fold difference. These statistical findings demonstrate an increase in 

the number of occurrences of qualitative research term over time. 

 

SR PMC search statement: BMLA # % PMC  search statement: JMLA # % 

1 

"Bulletin of the Medical Library 
Association"[Journal] AND 
("qualitative"[All Fields] OR 
“descriptive”[All Fields]) 

518 7.1 

("Journal of the Medical Library 
Association : JMLA"[Journal]) AND 
("qualitative"[All Fields] OR 
“descriptive”[All Fields]) 

343 22.9 

2 
"Bulletin of the Medical Library 
Association"[Journal] AND 
("qualitative design"[All Fields]) 

0 0 
("Journal of the Medical Library 
Association : JMLA"[Journal]) AND 
("qualitative design"[All Fields]) 

1 0.3 

3 

"Bulletin of the Medical Library 
Association"[Journal] AND 
("interview"[All Fields] OR 
"interviews"[All Fields]) 

375 5.2 

("Journal of the Medical Library 
Association : JMLA"[Journal]) AND 
("interview"[All Fields] OR 
"interviews"[All Fields]) 

261 17.4 

4 

"Bulletin of the Medical Library 
Association"[Journal] AND 
("focus group"[All Fields] OR 
"focus groups"[All Fields]) 

48 0.7 

("Journal of the Medical Library 
Association : JMLA"[Journal]) AND 
("focus group"[All Fields] OR 
"focus groups"[All Fields]) 

119 7.9 

5 

"Bulletin of the Medical Library 
Association"[Journal] AND 
("critical incident"[All Fields] OR 
"critical incidents"[All Fields]) 

34 0.5 

("Journal of the Medical Library 
Association : JMLA"[Journal]) AND 
("critical incident"[All Fields] OR 
"critical incidents"[All Fields]) 

12 0.8 

6 

"Bulletin of the Medical Library 
Association"[Journal] AND 
("phenomenology"[All Fields] OR 
"phenomenological"[All Fields]) 

5 0.1 

("Journal of the Medical Library 
Association : JMLA"[Journal]) AND 
("phenomenology"[All Fields] OR 
"phenomenological"[All Fields]) 

4 0.3 

 

Table 5a:  Number of search results from PubMed Central queries of BMLA and JMLA,  
All Fields queries – qualitative research search terms 

  

 



An examination of research topics in JMLA and BMLA - Marton 

 

   

Hypothesis, vol. 28, no. 1, Fall 2016                                                                      21                                                                                

 

 

 

SR PMC search statement: BMLA # % PMC  search statement: JMLA # % 

1 

"Bulletin of the Medical Library 
Association"[Journal] AND 
(("qualitative"[Title] OR 
"qualitative"[Abstract]) OR 
("descriptive”[Title] OR 
“descriptive"[Abstract])) 

33 0.5 

("Journal of the Medical Library 
Association : JMLA"[Journal]) AND 
(("qualitative"[Title] OR 
"qualitative"[Abstract]) OR 
("descriptive”[Title] OR 
“descriptive"[Abstract])) 

50 3.3 

2 

"Bulletin of the Medical Library 
Association"[Journal] AND 
(("qualitative design"[Title] OR 
"qualitative design"[Abstract]) OR 
("qualitative designs”[Title] OR 
“qualitative designs"[Abstract])) 

0 0 

("Journal of the Medical Library 
Association : JMLA"[Journal]) AND 
(("qualitative design"[Title] OR 
"qualitative design"[Abstract]) OR 
("qualitative designs”[Title] OR 
“qualitative designs"[Abstract])) 

0 0 

3 

"Bulletin of the Medical Library 
Association"[Journal] AND 
(("interview"[Title] OR 
"interview"[Abstract]) OR 
("interviews"[Title] OR 
“interviews"[Abstract])) 

35 0.5 

("Journal of the Medical Library 
Association : JMLA"[Journal]) AND 
(("interview"[Title] OR 
"interview"[Abstract]) OR 
("interviews"[Title] OR 
“interviews"[Abstract])) 

52 3.5 

4 

"Bulletin of the Medical Library 
Association"[Journal] AND (("focus 
group"[Title] OR "focus 
group"[Abstract]) OR ("focus 
groups"[Title] OR “focus 
groups"[Abstract])) 

4 0.1 

("Journal of the Medical Library 
Association : JMLA"[Journal]) AND 
(("focus group"[Title] OR "focus 
group"[Abstract]) OR ("focus 
groups"[Title] OR “focus 
groups"[Abstract])) 

17 1.1 

5 

"Bulletin of the Medical Library 
Association"[Journal] AND 
(("critical incident"[Title] OR "critical 
incident"[Abstract]) OR ("critical 
incidents"[Title] OR “critical 
incidents"[Abstract])) 

1 0 

("Journal of the Medical Library 
Association : JMLA"[Journal]) AND 
(("critical incident"[Title] OR 
"critical incident"[Abstract]) OR 
("critical incidents"[Title] OR 
“critical incidents"[Abstract])) 

1 0.1 

6 

"Bulletin of the Medical Library 
Association"[Journal] AND 
((“phenomenology"[Title] OR 
"phenomenology"[Abstract]) OR 
("phenomenological"[Title] OR 
"phenomenological"[Abstract])) 

0 0 

("Journal of the Medical Library 
Association : JMLA"[Journal]) AND 
((“phenomenology"[Title] OR 
"phenomenology"[Abstract]) OR 
("phenomenological"[Title] OR 
"phenomenological"[Abstract])) 

1 0.1 

 

Table 5b:  Number of search results from PubMed Central queries of BMLA and JMLA,  
Title OR Abstract queries – qualitative research search terms 



An examination of research topics in JMLA and BMLA - Marton 

 

   

Hypothesis, vol. 28, no. 1, Fall 2016                                                                      22                                                                                

 

 

SUMMARY OF FINDINGS 

Progressively narrowing the searches through the addition of the second concept, the research-

related terms, predictively reduces the number of search results, with more specific search terms 

reducing the number of search results considerably. Moreover, when the second search concept 

is restricted to [Title] OR [Abstract], there is a five to ten-fold reduction in the number of search 

results in comparison to the [All Fields] searches. The higher percentage of research-related 

search results for JMLA searches relative to BMLA searches is found consistently for all 

searches, for both the [All Fields] searches and the [Title] OR [Abstract] searches. 

Clearly, there is a marked increase in general and specific research-related terms in JMLA relative 

to its predecessor, BMLA. Paired t-tests conducted in the statistical package, SPSS (version 11) 

demonstrate that these differences are statistically significant for both the All Fields searches 

(t=4.092, df=19, p=.001) and the Title OR Abstract searches (t=2.615, df=19, p=.017). Overall, 

the quantitative data analysis of occurrences of research-related terms indicates that there is an 

increase in published research activity in the health sciences librarianship profession over time, 

with the largest noticeable increase for systematic reviews.  

 

DISCUSSION 

Conducting searches within issues of the Bulletin of the Medical Library Association and the 

Journal of the Medical Library Association reveal several interesting findings concerning the 

prominence of research in this leading health sciences librarianship journal over time, as 

measured by percentage of search terms. First, the word research appears in abundance in 

both BMLA and JMLA. The word study and its plural, studies, is also frequently found in BMLA 

and JMLA. The large number of search results suggests a prominent role for research in health 

sciences librarianship. Paired t-tests comparising BMLA searches to JMLA searches 

demonstrate statistically significant increases in occurrences of research-related terms over 

time for general research terms, quantitative research terms, qualitative research terms, and 

mixed-methods research terms. All research-related search terms are mentioned more 

frequently in JMLA than BMLA suggesting that health sciences librarianship research has 

increased since 2002, which supports a finding by Slutsky and Aytac concerning the increase in 

research in librarianship. As well, a greater diversity of study designs are being utilized. 



An examination of research topics in JMLA and BMLA - Marton 

 

   

Hypothesis, vol. 28, no. 1, Fall 2016                                                                      23                                                                                

 

 

Surveys are popular. Surveys are commonly questionnaires. However, they can also take the form 

of interviews that utilize a pre-determined set of questions and list of responses. Well-known 

qualitative methods, such as open-ended interviews and focus groups are also popular. The 

predominance of quantitative, non-experimental designs, confirms findings from earlier research 

on LIS journals conducted by Koufogiannakis, Slater, and Crumley and by Slutsky and Aytac.              

Less well known qualitative study designs, such as the critical incident technique and 

phenomenological approaches, are infrequently found in both BMLA and JMLA.  

Systematic reviews, a form of secondary research, has gained considerably in visibility. The 

significant increase in mention of systematic reviews is an interesting trend that is reflective of 

the current interest in evidence-based medicine (EBM) and evidence-based practice (EBP). 

 

LIMITATIONS 

There are several limitations associated with this type of study. First, number of search results is 

a very basic quantitative measure that is largely devoid of context. This method can be refined 

by searching each individual issue of BMLA and JMLA and counting the number of search results 

per search term. As well, the occurrence of each search term in individual articles would 

represent a further refinement of this measure and determine to some extent the context in 

which these research-related terms are utilized. Second, health sciences librarians have other 

publication options, such as Health Information and Libraries Journal, Medical Reference Services 

Quarterly and Journal of Hospital Librarianship, to name but a few journal titles in this discipline. 

As well, they can publish their research in health sciences journals or information science 

journals. Thus, a quantitative study that focuses solely on counting the number of search results 

for research-related terms in one journal, albeit a prominent journal in its field, will only capture 

a fraction of the research studies published in the journal literature by health sciences librarians.  

 

FUTURE DIRECTIONS 

Future studies to investigate this research topic further include replication studies utilizing the 

research method employed in this study for investigation in other health sciences librarianship 

journals to determine if the same trends are present. As well, this research method could be 

utilized in future studies but with a more fine-grained approach to searching for research-related 



An examination of research topics in JMLA and BMLA - Marton 

 

   

Hypothesis, vol. 28, no. 1, Fall 2016                                                                      24                                                                                

 

terms in specific issues of BMLA and JMLA. Additional research-related search terms could be 

considered for inclusion, such as thematic analysis, inductive analysis, theoretical framework, 

and inferential statistics, among others. Another avenue to explore is authorship metrics in 

relation to study design. For example, Wheeler, Yaniv and Fenske determined the most influential 

authors who are members of the MLA Research Section, according to number of citations in the 

Web of Science Citation Report. Carol Lefebvre’s articles on systematic reviews were in the top 

ten papers in the LIS discipline, suggesting a relationship between citation count and study 

design. [17] Author affiliation is also of interest. For example, Hardin and Stankus reported the 

institutional affiliations of the published academic science, engineering, agricultural, and medical 

librarians over a ten-year period (2000-2010). The LIS journals examined were Science & 

Technology Libraries; Issues in Science and Technology Librarianship; Journal of Agricultural and 

Food Information; Journal of the Medical Library Association; and Medical Reference Services 

Quarterly. The top US affiliations in this set of LIS journals were: Illinois, Purdue, Texas, Penn State 

and Cornell. [18] Exploring relationships between author h-index, author affiliation, article 

citation count, journal ranking, and study design may yield interesting findings. 

 

REFERENCES 

1. Grefsheim SF, Rankin JA, Perry GJ, McKibbon KA. Affirming our commitment to research: the 
Medical Library Association’s research policy statement: the process and findings. Journal of 
the Medical Library Association. 2008;96(2):114-120. doi: 10.3163/1536-5050.96.2.114. 

2. Dalrymple PW, Bastille JD, Bradley J, Dee CR, Humphreys BL, Marshall JG, Weller AC, Webb 
RE. Using scientific evidence to improve information practice. Chicago, IL: Medical Library 
Association, 1995. 

3. Eldredge JD, Harris MR, Ascher MT. Defining the Medical Library Association research agenda: 
methodology and final results from a consensus process. Journal of the Medical Library 
Association.2009;97(3):178-185. doi: 10.3163/1536-5050.97.3.006. 

4. Eldredge JD, Ascher MT, Homes JN, Harris MR. The new Medical Library Association research 
agenda: final results from a three-phase Delphi study. Journal of the Medical Library 
Association. 2012;100(3):214-218. doi: 10.3163/1536-5050.100.3.012 

5. Ascher MT, Eldredge JD, Holmes HN, Harris MR. 2012. Medical Library Association. Research 
Section. Research Agenda Committee. MLA Research Agenda: Appraising the Best Available 
Evidence.  http://repository.unm.edu/handle/1928/24634 

https://dx.doi.org/10.3163%2F1536-5050.96.2.114
https://dx.doi.org/10.3163%2F1536-5050.97.3.006
https://dx.doi.org/10.3163%2F1536-5050.100.3.012
https://scholar.google.com/scholar?oi=bibs&cluster=11436103200852211256&btnI=1&hl=en
https://scholar.google.com/scholar?oi=bibs&cluster=11436103200852211256&btnI=1&hl=en
http://repository.unm.edu/handle/1928/24634


An examination of research topics in JMLA and BMLA - Marton 

 

   

Hypothesis, vol. 28, no. 1, Fall 2016                                                                      25                                                                                

 

6. Eldredge JD, Ascher MT, Holmes HN. Medical Library Association Research Section. Research 
Agenda Committee. MLA Research Agenda: Systematic Review Project Team Updates 
Presentation. MLA Annual Meeting. May 17, 2015. http://repository.unm.edu/handle/1928/27127 

7. SCImago. (2016). About us. November 8, 2011. <http://www.scimagojr.com/aboutus.php> 

8. SCImago. (2016). Journal rankings: library and information sciences. November 8, 2011.  
<http://www.scimagojr.com/ journalrank.php?category=3309> 

9. 2015 Journal Citation Reports® Social Sciences Edition (Thomson Reuters, 2016) 

10. Koufogiannakis D, Slater L, Crumley E. A content analysis of librarianship research. Journal of 
information science. 2004;30(3):227-239. doi: 10.1177/0165551504044668 

11. Slutsky B, Aytac S. Publication patterns of science, technology, and medical librarians: Review 
of the 2008-2012 published research. Science & Technology Libraries. 2014;33(4):369-382. 
doi: 10.1080/0194262X.2014.952486 

12. Slutsky B, Aytac S. Bibliometric analysis and comparison of two STEM LIS Journals: Science & 
Technology Libraries and Issues in Science & Technology Librarianship (2005–2014). Science 
& Technology Libraries. 2016;35(2):152-171. doi: 10.1080/0194262X.2016.1171191 

13. Creswell JW. Research Design: Qualitative, quantitative, and mixed methods approaches. 2nd 
ed. Thousand Oaks, CA: Sage Publications; 2002. P. 153. 

14. Garg AX, Hackam D, Tonelli M. Systematic Review and Meta-analysis: When One Study Is Just 
not Enough. Clinical Journal of the American Society of Nephorology. 2008, 3(1): 253–260. 
doi: 10.2215/CJN.01430307. P. 253. 

15. Creswell JW. Research Design: Qualitative, quantitative, and mixed methods approaches. 2nd 
ed. Thousand Oaks, CA: Sage Publications; 2002. P. 182. 

16. Creswell JW. Research Design: Qualitative, quantitative, and mixed methods approaches. 2nd 
ed. Thousand Oaks, CA: Sage Publications; 2002. P. 211. 

17. Wheeler TR, Yaniv N, Fenske RE. (2015). A look at the scholarly output of the Medical Library 
Association Research Section. Hypothesis. 2015;27(1):3-7. 

18. Hardin A, Stankus T. The affiliations of U.S. academic librarians in the most prominent journals 
of science, engineering, agricultural, and medical librarianship, 2000-2010.  Science & 
Technology Libraries. 2011;30(2): 143-156. doi: 10.1080/0194262X.2011.575285 

 
 
Author Affiliations  
1 Adjunct Instructor, Faculty of Information, University of Toronto, Toronto, Ontario; 
christine.marton@utoronto.ca 
 

http://www.scimagojr.com/aboutus.php
http://www.scimagojr.com/%20journalrank.php?category=3309
https://scholar.google.com/citations?view_op=view_citation&hl=en&user=GI865CUAAAAJ&cstart=20&sortby=pubdate&citation_for_view=GI865CUAAAAJ:u-x6o8ySG0sC
http://dx.doi.org/10.1080/0194262X.2014.952486
http://dx.doi.org/10.1080/0194262X.2016.1171191
http://dx.doi.org/10.1080/0194262X.2011.575285
mailto:christine.marton@utoronto.ca

