An examination of research topics in BMLA and JMLA - Marton Hypothesis, vol. 28, no. 1, Fall 2016 6 TITLE An examination of research topics in the Journal of the Medical Library Association and the Bulletin of the Medical Library Association: quantifying the importance of research to medical librarians over time AUTHOR Christine Marton, PhD1 INTRODUCTION Historically, librarians have undertaken observational studies of their users, specifically, of their preferences for and uses of specific information source formats, online databases, and information technologies. User satisfaction studies of library collections and services have also been popular. With the advent of desktop computing, computer networks and online databases in the latter half of the twentieth century, research studies of all kinds and across all disciplines have become more numerous. In today’s information society, research has grown into an integral activity no longer restricted to academics and research institutes. How has this trend affected the profession of health sciences librarianship? Has the number of research projects in this discipline increased in recent years? What study designs are being utilized? As the foremost professional association for health sciences librarians and informationists in the United States of America, the Medical Library Association (MLA) sets priorities for its membership and more broadly, for the profession. In recent decades, MLA has begun to prioritize research. In 1995, MLA developed a research policy statement, Using Scientific Evidence to Improve Information Practice, which called on health sciences librarians to be proactive in creating, managing, and using scientific evidence [1]. In 2008, MLA published The Research Imperative [2]. Based on semi-structured interviews with key stakeholders, including editors of the Journal of the Medical Library Association, the research policy emphasizes evidence-based library and information practice. Six themes were identified: creation of a research culture, domains of research, research skills set, roles of stakeholders, challenges, and measurement of progress. In 2008 and 2011 the Research Agenda Committee of the Research Section of MLA identified the An examination of research topics in BMLA and JMLA - Marton Hypothesis, vol. 28, no. 1, Fall 2016 7 fifteen most important researchable questions facing our profession. [3, 4] More recently teams have been conducting systematic reviews to determine the state of knowledge in these 15 areas. [5,6]. Clearly, there is increasing recognition within the profession of health sciences librarianship of the importance of the role of librarians as researchers and evidence based practice. The Journal of the Medical Library Association (JMLA) is an open access, peer-reviewed journal published quarterly by the Medical Library Association. Current and previous full-text issues are freely available on PubMed Central. Previously known as the Bulletin of the Medical Library Association (BMLA), this journal serves as a reputable publication of research studies about health sciences librarianship. Both major citation indexes, Elsevier’s SCImago and Thomson Reuters’ Journal Citation Reports rank the JMLA higher than comparable health sciences librarian journals in their respective lists of journal impact factors for library and information sciences journals. Of the 209 journals in the Library & Information Science (LIS) subject category indexed by the Scopus database, the SCImago Journal & Country Rank ranked the JMLA in thirty-eighth position in 2015 with an SJR of 0.726, ahead of Health Information and Libraries Journal (43; 0.650); Evidence Based Library and Information Practice (56, 0.586); Medical Reference Services Quarterly (65; 0.500); and Journal of Hospital Librarianship (125; 0.217). [9] Of the 86 journals in the LIS subject category indexed in the Journal Citation Reports database in 2015, the JMLA was ranked 37 with a journal impact factor of 1.084, while Health Information and Libraries Journal was ranked 50, with a journal impact factor of 0.712. The other aforementioned health sciences librarianship journals are not indexed in the Journal Citation Reports database. Clearly, the JMLA is the foremost publication for health sciences librarianship; articles published in this journal serve as a strong representation of the published research in this discipline. [7-9] OBJECTIVE This quantitative study seeks to determine whether health sciences librarians have undertaken more research activities in recent years by measuring the occurrences of research-related words in all issues of the peer-reviewed journal publication of the Medical Library Association and comparing the percentage of occurrences (prevalence) between the earlier publication, Bulletin of the Medical Library Association and the current publication, Journal of the Medical Library Association. An examination of research topics in BMLA and JMLA - Marton Hypothesis, vol. 28, no. 1, Fall 2016 8 LITERATURE REVIEW Several studies on librarianship research have been conducted by library science academics and academic librarians. These studies generally utilize a restricted time period, such as one year or five years, and examine a small number of library and information science (LIS) journals. Content analysis and bibliometrics are popular study designs. In their content analysis of librarianship research, Koufogiannakis, Slater and Crumley examined the LIS journal literature for a one-year period in 2001. Of the 217 LIS journals reviewed, they included 107 journals in their study, of which 91 contained relevant data. Of the 2,664 journal articles examined, they classified 807 as research articles. The top LIS journals for research in 2001 were the Journal of the American Society of Information Science and Technology (JASIST); Scientometrics; Information Processing & Management; College & Research Libraries; Bulletin of the Medical Library Association (tied with Journal of Library Administration); Libraries and Culture; Journal of Documentation; and Journal of Information Science (tied with Journal of Academic Librarianship). According to the authors’ classification scheme of six subject domains, “information access and retrieval” contained the greatest number of research articles and the most frequently published type of research was descriptive research. Non-experimental research, such as surveys, was found to be much more prevalent than experimental research. [10] Slutsky and Aytac reviewed science librarianship research from 2008–2012 in four LIS journals: Health Information & Libraries Journal (HILJ); Journal of the Medical Library Association (JMLA); Issues in Science & Technology Librarianship (ISTL); and Science & Technology Libraries (STL). They analyzed the texts of 574 articles and classified them as either research or non-research. Non-research articles were removed from the analysis. Slightly more than half of all articles were classified as research (n=311; 54.2%). Study variables included authorship, affiliation, type of research, research topic, and data collection and data analysis techniques. Bibliometric data analysis revealed that there has been a dramatic growth in research in these LIS journals for the years 2008-2012. The majority of research papers featured quantitative study designs; qualitative studies comprised only 10% of the research articles examined. Quantitative data analysis overwhelmingly consisted of descriptive statistics (88.7%). The most popular study designs were survey, content analysis, citation analysis, and interviews. There was a significant difference among these four different publications with respect to the location of the study, context of research, research approach, and statistical analysis. As well, overall authorship was An examination of research topics in BMLA and JMLA - Marton Hypothesis, vol. 28, no. 1, Fall 2016 9 highly collaborative; almost three-quarters of the research articles were written by two or more authors. Of interest, the JMLA contained the greatest number of research articles, followed by HILJ, ISTL, and STL. This finding lends support to the presumption that JMLA is a key journal, not only for health sciences librarianship research but for LIS research more broadly. [11] In a more recent study, Slutsky and Aytac conducted a bibliometric analysis of research articles published in the aforementioned STEM librarianship journals, ISTL and STL, over a ten-year period (2005-2014). They found a greater number of research articles in ISTL but higher Scopus citation metrics for STL. As well, the most frequent topic in STL was “bibliometrics and citation analysis” while in ISTL it was “libraries and librarianship,” with “library resources” as the foremost topic. There were more author collaborations in ISTL than STL. Of interest, JMLA was ranked eighth in the list of top 25 LIS journals cited in issues in ISTL and thirteenth in the top 25 LIS journals cited in STL. [11] Clearly, JMLA is regarded as a reputable publication by science and technology librarians. [12] From these three bibliometric and citation analysis studies of the LIS journal literature, it is apparent that BMLA and its successor, JMLA, are well regarded research publications in the field of library and information science. Analyzing the prevalence of research studies and study designs in these two publications is a valid method for determining whether there has been an increase in research productivity in health sciences librarianship over time as well as an increase in the diversity of study designs beyond surveys and interviews. METHOD To draw this comparison, the frequency (counts) and prevalence (percentages) of articles including words and phrases that are commonly used to describe research studies and methodologies were obtained from all issues of the BMLA, which was published from 1911 to 2001 (volumes 1-89), and its successor, JMLA, which has been published since 2002 (volumes 90-104). All issues are available online in full text from PubMed Central (PMC). These two journals were queried using the search interface on the PMC webpage for the journals’ archive (www.ncbi.nlm.nih.gov/pmc/ journals/93/). http://www.ncbi.nlm.nih.gov/ An examination of research topics in BMLA and JMLA - Marton Hypothesis, vol. 28, no. 1, Fall 2016 10 Search statements began with the first topic, which is the name of the journal: Bulletin of the Medical Library Association and its successor, Journal of the Medical Library Association. The journal name was entered as the first search term in the search box: "Bulletin of the Medical Library Association"[Journal] for one set of searches and "Journal of the Medical Library Association : JMLA"[Journal] for the other set of searches. Next, the first search term was paired with a second concept, beginning with the search term research, in double quotation marks, which was added to subsequent searches. For the second concept, the search term was entered in separate searches with either the field limit [All Fields] or the field limits [Title] OR [Abstract]. Thus, two searches were conducted for each second concept. The [All Fields] searches are broad: they capture the search terms as they appear in any part of the full text documents. The [Title] OR [Abstract] field limits restrict the searches to those two bibliographic fields, thereby improving the precision of the searches. Preliminary searches were conducted in late August, 2016. The original searches and new searches with additional second concept terms were conducted in early October, 2016 and also in mid-November 2016, when the fourth and final 2016 issue of JMLA was added to the journal repository site on PubMed Central. Tables 1-2 in the Results section below list all of the search statements and their corresponding number (counts and percentages) of search results. The searches progressed from broad to narrow as the search terms for the second concept became more specific. Searching all issues of both BMLA and JMLA provides quantitative measures of all research- related words. The number of search results serves as a basic measure of the pervasiveness of research in this publication, and more broadly, in the field of health sciences librarianship. Search term frequencies and percentages can be tabulated to facilitate pairwise quantitative comparisons that indicate whether there has been an increase in published research activities in health sciences librarianship over time. The use of inferential statistics, more specifically, the paired t-test, enables the calculation of the statistical significance of difference in means of search results for JMLA in relation to its predecessor, BMLA. An examination of research topics in BMLA and JMLA - Marton Hypothesis, vol. 28, no. 1, Fall 2016 11 The search results data were first entered into two comprehensive tables in a Microsoft Word document: one table for the [All Fields] searches and a second table for the [Title] OR [Abstract] field limited searches. All table columns were copied to an Excel spreadsheet and the spreadsheet was then imported to the statistical package, SPSS (Version 11). The appropriate inferential statistic, the paired t-test, was calculated to determine the strength and significance of the difference in means for the search results on research-related terms between BMLA and JMLA for (1) [All Fields] searches and (2) [Title] OR [Abstract] searches. Percentages, instead of counts, were compared to take into account the larger number of journal issues for BMLA in relation to JMLA, which is a function of the greater timespan for the publication of BMLA, and concomitant greater number of articles. Unlike prior bibliometric and content analyses studies on the pervasiveness of research in the field of librarianship, the context of these terms was not investigated. Thus, all occurrence of research-related terms in BMLA and JMLA are included irrespective of whether they were published in research articles or appear in editorials, reviews and other non-research columns of BMLA and JMLA. This is an acceptable approach because the objective is not to categorize the topics of research, nor to investigate characteristics of authorship or citation patterns, but solely to determine whether an increase in (1) research and (2) study designs has occurred in the profession of health sciences librarianship from 2002 onward, when BMLA was renamed JMLA. Restricting the searches to the [Title] OR [Abstract] fields improves, to some extent, the relevance of search results because of the greater specificity and relationship to topicality than the broad [All Fields] searches. RESULTS The initial search, “Bulletin of the Medical Library Associaton”[Journal] retrieved 7256 results while the initial search, "Journal of the Medical Library Association : JMLA"[Journal], retrieved 1498 search results. The greater publication timespan of BMLA is accountable for this large difference in number of search results on journal title. An examination of research topics in BMLA and JMLA - Marton Hypothesis, vol. 28, no. 1, Fall 2016 12 GENERAL RESEARCH SEARCH TERMS SR PMC search statement: BMLA # % PMC search statement: JMLA # % 1 "Bulletin of the Medical Library Association"[Journal] Alternatively: "Bulletin of the Medical Library Association"[Journal] OR "Bull Med Libr Assoc"[Journal] 7256 100.0 "Journal of the Medical Library Association : JMLA"[Journal] Alternatively: "Journal of the Medical Library Association"[Journal] OR "J Med Libr Assoc"[Journal] 1498 100.0 2 "Bulletin of the Medical Library Association"[Journal] AND ("research"[All Fields]) 2941 40.5 ("Journal of the Medical Library Association : JMLA"[Journal]) AND ("research"[All Fields]) 1101 73.5 3 "Bulletin of the Medical Library Association"[Journal] AND ("study"[All Fields] OR "studies"[All Fields]) 2945 40.6 ("Journal of the Medical Library Association : JMLA"[Journal]) AND ("study"[All Fields] OR "studies"[All Fields]) 996 66.5 4 "Bulletin of the Medical Library Association"[Journal] AND ("study design"[All Fields]) 32 0.4 ("Journal of the Medical Library Association : JMLA"[Journal]) AND ("study design"[All Fields]) 75 5.0 Table 1a: Number of search results from PubMed Central queries of BMLA and JMLA, All Fields queries – general research search terms In the following sub-sections of the Results section, the second concept, research-related, is operationalized as search terms for quantitative study designs (Table 2a,b); systematic review and meta-analysis (Table 3a,b); mixed-method study designs (Table 4a,b), and qualitative study designs (Table 5a,b). An examination of research topics in BMLA and JMLA - Marton Hypothesis, vol. 28, no. 1, Fall 2016 13 When the search term "research" is added and the [All Fields] field limit is employed, the number of search results is somewhat more than halved to 2941 search results for the Bulletin of the Medical Library Association and 1101 search results for the Journal of the Medical Library Association. More importantly, the percentage is 40.5 for BMLA and research versus 73.5 for JMLA and research, almost a doubling of term occurrences. (Table 1a) For the comparable [Title] OR [Abstract] searches, the increase is even more dramatic: 4.6% for BMLA and research versus 18.8% for JMLA and research, a four-fold increase in term occurrence (Table 1b). For this set of searches on general research terms, the mean number of search results for All Fields BMLA searches is 27.2 while the mean number of searches for All Fields JMLA searches is 48.3. For Title OR Abstract searches, the mean number of search results for BMLA searches is 3.3 while for JMLA searches it is 13.1. Thus, there is an increase over time for occurrence of general research terms. SR PMC search statement: BMLA # % PMC search statement: JMLA # % 1 "Bulletin of the Medical Library Association"[Journal] 7256 100.0 "Journal of the Medical Library Association : JMLA"[Journal] 1498 100.0 2 "Bulletin of the Medical Library Association"[Journal] AND ((“research"[Title] OR "research"[Abstract])) 332 4.6 ("Journal of the Medical Library Association : JMLA"[Journal]) AND ((“research"[Title] OR "research"[Abstract])) 281 18.8 3 "Bulletin of the Medical Library Association"[Journal] AND ((“study"[Title] OR "study"[Abstract]) OR ("studies"[Title] OR “studies"[Abstract])) 379 5.2 ("Journal of the Medical Library Association : JMLA"[Journal]) AND ((“study"[Title] OR "study"[Abstract]) OR ("studies"[Title] OR “studies"[Abstract])) 301 20.1 4 "Bulletin of the Medical Library Association"[Journal] AND (("study design"[Title] OR "study design"[Abstract]) OR ("study designs"[Title] OR "study designs"[Abstract])) 2 0 ("Journal of the Medical Library Association : JMLA"[Journal]) AND (("study design"[Title] OR "study design"[Abstract]) OR ("study designs"[Title] OR "study designs"[Abstract])) 5 0.3 Table 1b: Number of search results from PubMed Central queries of BMLA and JMLA, Title OR Abstract queries – general research search terms An examination of research topics in BMLA and JMLA - Marton Hypothesis, vol. 28, no. 1, Fall 2016 14 QUANTITATIVE RESEARCH SEARCH TERMS SR PMC search statement: BMLA # % PMC search statement: JMLA # % 1 "Bulletin of the Medical Library Association"[Journal] AND ("quantitative"[All Fields] OR "quantify"[All Fields]) 286 3.9 ("Journal of the Medical Library Association : JMLA"[Journal]) AND ("quantitative"[All Fields] OR "quantify"[All Fields]) 182 12.2 2 "Bulletin of the Medical Library Association"[Journal] AND ("quantitative design"[All Fields]) 0 0 ("Journal of the Medical Library Association : JMLA"[Journal]) AND ("quantitative design"[All Fields]) 0 0 3 "Bulletin of the Medical Library Association"[Journal] AND ("survey"[All Fields] OR "surveys"[All Fields]) 1615 22.3 ("Journal of the Medical Library Association : JMLA"[Journal]) AND ("survey"[All Fields] OR "surveys"[All Fields]) 609 40.7 4 "Bulletin of the Medical Library Association"[Journal] AND ("questionnaire"[All Fields] OR "questionnaires"[All Fields]) 655 9.0 ("Journal of the Medical Library Association : JMLA"[Journal]) AND ("questionnaire"[All Fields] OR "questionnaires"[All Fields]) 215 14.4 5 "Bulletin of the Medical Library Association"[Journal] AND ("bibliometric"[All Fields]) 49 0.7 ("Journal of the Medical Library Association : JMLA"[Journal]) AND ("bibliometric"[All Fields]) 80 5.3 6 "Bulletin of the Medical Library Association"[Journal] AND ("citation analysis"[All Fields]) 82 1.1 ("Journal of the Medical Library Association : JMLA"[Journal]) AND ("citation analysis"[All Fields]) 85 5.7 7 "Bulletin of the Medical Library Association"[Journal] AND ("content analysis"[All Fields]) 30 0.4 ("Journal of the Medical Library Association : JMLA"[Journal]) AND ("content analysis"[All Fields]) 46 3.1 Table 2a: Number of search results from PubMed Central queries of BMLA and JMLA, All Fields queries –quantitative research search terms An examination of research topics in BMLA and JMLA - Marton Hypothesis, vol. 28, no. 1, Fall 2016 15 The second set of research terms examined pertain to quantitative study designs. The search results are presented in Table 2a and Table 2b. According to Creswell, a key characteristic of quantitative study is determinism. “Examining the relationships between and among variables is central to answering questions and hypotheses through surveys and experiments. The reduction to a parsimonious set of variables, tightly controlled through design or statistical analysis, provides measures or observations for testing a theory. Objective data results from empirical observations and measures. Validity and reliability of scores on instruments, additional standards for making knowledge claims, lead to meaningful interpretations of data.”[13] In Tables 2a and 2b, several quantitative study designs are examined in relation to the occurrence of appropriate search terms in BMLA and JMLA. To begin, general terms, quantify and quantitative design, are utilized. Then, the searches are narrowed to specific quantitative methods, such as survey and questionnaire. Lastly, even narrower terms are employed, such as citation analysis. Although the term quantitative design is not to be found in the search results, there are many search results for surveys and questionnaires, and smaller numbers for bibliometric(s), content analysis(es) and citation analysis(es). For each quantitative method, the percentage of search results for JMLA exceeds that of BMLA. The mean number of search results for BMLA All Fields searches on this set of quantitative study designs is 5.4 while the mean number of search results for JMLA All Fields searches is 11.6. For the Title OR Abstract searches, BMLA searches have a mean of 0.8 while JMLA searches have a mean of 2.4. These statistical findings demonstrate an increase in the number of occurrences of quantitative research-related terms over time. An examination of research topics in JMLA and BMLA - Marton Hypothesis, vol. 28, no. 1, Fall 2016 16 SR PMC search statement: BMLA # % PMC search statement: JMLA # % 1 "Bulletin of the Medical Library Association"[Journal] AND (("quantitative"[Title] OR "quantitative"[Abstract]) OR ("quantify"[Title] OR “quantify"[Abstract])) 27 0.4 ("Journal of the Medical Library Association : JMLA"[Journal]) AND (("quantitative"[Title] OR "quantitative"[Abstract]) OR ("quantify"[Title] OR “quantify"[Abstract])) 20 1.3 2 "Bulletin of the Medical Library ssociation"[Journal] AND (("quantitative design"[Title] OR "quantitative design"[Abstract]) OR ("quantitative designs"[Title] OR “quantitative designs"[Abstract])) 0 0 ("Journal of the Medical Library Association : JMLA"[Journal]) AND (("quantitative design"[Title] OR "quantitative design"[Abstract]) OR ("quantitative designs"[Title] OR “quantitative designs"[Abstract])) 0 0 3 "Bulletin of the Medical Library Association"[Journal] AND (("survey"[Title] OR "survey"[Abstract]) OR ("surveys"[Title] OR “surveys"[Abstract])) 262 3.6 ("Journal of the Medical Library Association : JMLA"[Journal]) AND (("survey"[Title] OR "survey"[Abstract]) OR ("surveys"[Title] OR “surveys"[Abstract])) 147 9.8 4 "Bulletin of the Medical Library Association"[Journal] AND (("questionnaire"[Title] OR "questionnaire"[Abstract]) OR ("questionnaires"[Title] OR “questionnaires"[Abstract])) 86 1.2 ("Journal of the Medical Library Association : JMLA"[Journal]) AND (("questionnaire"[Title] OR "questionnaire"[Abstract]) OR ("questionnaires"[Title] OR “questionnaires"[Abstract])) 34 2.3 5 "Bulletin of the Medical Library Association"[Journal] AND (("bibliometric"[Title] OR "bibliometric"[Abstract]) OR ("bibliometrics"[Title] OR “bibliometrics"[Abstract])) 17 0.2 ("Journal of the Medical Library Association : JMLA"[Journal]) AND (("bibliometric"[Title] OR "bibliometric"[Abstract]) OR ("bibliometrics"[Title] OR “bibliometrics"[Abstract])) 20 1.3 6 "Bulletin of the Medical Library Association"[Journal] AND (("citation anaysis"[Title] OR "citation analysis"[Abstract]) OR ("citation analyses"[Title] OR “citation analyses"[Abstract])) 13 0.2 ("Journal of the Medical Library Association : JMLA"[Journal]) AND (("citation analysis"[Title] OR "citation analysis"[Abstract]) OR ("citation analyses"[Title] OR “citation analyses"[Abstract])) 28 1.7 7 "Bulletin of the Medical Library Association"[Journal] AND (("content analysis"[Title] OR "content analysis"[Abstract]) OR ("content analyses"[Title] OR “content analyses"[Abstract])) 7 0.1 ("Journal of the Medical Library Association : JMLA"[Journal]) AND (("content analysis"[Title] OR "content analysis"[Abstract]) OR ("content analyses"[Title] OR “content analyses"[Abstract])) 9 0.6 Table 2b: Number of search results from PubMed Central queries of BMLA and JMLA, Title OR Abstract queries –quantitative research search terms An examination of research topics in JMLA and BMLA - Marton Hypothesis, vol. 28, no. 1, Fall 2016 17 SYSTEMATIC REVIEWS AND META-ANALYSES A systematic review provides a comprehensive analysis of all existing primary studies on a very well-defined research question. It evaluates the methods employed in these studies, summarizes the results, presents important findings, identifies reasons for differences in findings across studies, and identifies the limitations of current knowledge. Combining the results mathematically through the use of statistical methods that sumarize all of the findings from the primary studies is referred to as a meta-analysis. [14] SR PMC search statement: BMLA # % PMC search statement: JMLA # % 1 "Bulletin of the Medical Library Association"[Journal] AND ("systematic review"[All Fields] OR "systematic reviews"[All Fields]) 19 0.3 ("Journal of the Medical Library Association : JMLA"[Journal]) AND ("systematic review"[All Fields] OR "systematic reviews"[All Fields]) 287 19.2 2 "Bulletin of the Medical Library Association"[Journal] AND ("meta analysis"[All Fields] OR "meta analyses"[All Fields]) 31 0.4 ("Journal of the Medical Library Association : JMLA"[Journal]) AND ("meta analysis"[All Fields] OR "meta analyses"[All Fields]) 95 6.3 Table 3a: Number of search results from PubMed Central queries of BMLA and JMLA, All Fields queries – systematic review and meta-analysis search terms For the All Fields searches of this set of research terms, the mean value of BMLA searches is 0.35 while the mean value of JMLA searches is 12.8. For the Title OR Abstract searches, the mean value of BMLA searches is 0.1 while the mean value of JMLA searches is 1.4. These statistical findings demonstrate an increase in the number of occurrences of these two research-related terms: systematic review(s) and meta-analysis(es) over time. An examination of research topics in JMLA and BMLA - Marton Hypothesis, vol. 28, no. 1, Fall 2016 18 SR PMC search statement: BMLA # % PMC search statement: JMLA # % 1 "Bulletin of the Medical Library Association"[Journal] AND (("systematic review"[Title] OR "systematic review"[Abstract]) OR ("systematic reviews"[Title] OR “systematic reviews"[Abstract])) 5 0.1 ("Journal of the Medical Library Association : JMLA"[Journal]) AND (("systematic review"[Title] OR "systematic review"[Abstract]) OR ("systematic reviews"[Title] OR “systematic reviews"[Abstract])) 35 2.3 2 "Bulletin of the Medical Library Association"[Journal] AND (("meta analysis"[Title] OR "meta analysis"[Abstract]) OR ("meta analyses"[Title] OR “meta analyses"[Abstract])) 6 0.1 ("Journal of the Medical Library Association : JMLA"[Journal]) AND (("meta analysis"[Title] OR "meta analysis"[Abstract]) OR ("meta analyses"[Title] OR “meta analyses"[Abstract])) 7 0.5 Table 3b: Number of search results from PubMed Central queries of BMLA and JMLA, Title OR Abstract queries – systematic review and meta-analysis search terms MIXED METHODS RESEARCH Mixed methods study designs include both quantitative and qualitative methods. Either the quantitative or the qualitative component can be dominant, or they can co-exist equally. Creswell provides a useful matrix that illustrates the four decisions required to select a mixed methods approach. The use of a theoretical framework; the implementation sequence of quantitative and qualitative data collection; the priority given to quantitative and qualitation data collection and analysis, and the integration of quantitative and qualitative findings are key considerations for mixed methods designs. [15] An examination of research topics in JMLA and BMLA - Marton Hypothesis, vol. 28, no. 1, Fall 2016 19 SR PMC search statement: BMLA # % PMC search statement: JMLA # % 1 "Bulletin of the Medical Library Association"[Journal] AND ("mixed method"[All Fields] OR "mixed methods"[All Fields]) 0 0 ("Journal of the Medical Library Association : JMLA"[Journal]) AND ("mixed method"[All Fields] OR "mixed methods"[All Fields]) 21 1.4 2 "Bulletin of the Medical Library Association"[Journal] AND ("triangulation"[All Fields]) 2 0 ("Journal of the Medical Library Association : JMLA"[Journal]) AND ("triangulation"[All Fields]) 17 1.1 Table 4a: Number of search results from PubMed Central queries of BMLA and JMLA, All Fields queries – mixed methods search terms SR PMC search statement: BMLA # % PMC search statement: JMLA # % 1 "Bulletin of the Medical Library Association"[Journal] AND (("mixed method"[Title] OR "mixed Method"[Abstract]) OR ("mixed methods"[Title] OR "mixed methods"[Abstract])) 0 0 ("Journal of the Medical Library Association : JMLA"[Journal]) AND (("mixed method"[Title] OR "mixed method"[Abstract]) OR ("mixed methods"[Title] OR "mixed methods"[Abstract])) 5 0.3 2 "Bulletin of the Medical Library Association"[Journal] AND (("triangulation"[Title] OR "triangulation"[Abstract]) OR ("triangulate"[Title] OR "triangulate"[Abstract])) 0 0 ("Journal of the Medical Library Association : JMLA"[Journal]) AND (("triangulation"[Title] OR "triangulation"[Abstract]) OR ("triangulate"[Title] OR "triangulate"[Abstract])) 2 0.1 Table 4b: Number of search results from PubMed Central queries of BMLA and JMLA, Title OR Abstract queries – mixed methods search terms An examination of research topics in JMLA and BMLA - Marton Hypothesis, vol. 28, no. 1, Fall 2016 20 QUALITATIVE RESEARCH SEARCH TERMS Creswell characterizes qualitative research as broad, holistic, reflective and interpretative. [16] For this fifth and final set of searches on research-related terms in BMLA and JMLA, general terms for qualitative research, qualitative and qualitative design, are entered first, followed by more specific search terms for individual qualitative approaches: interview(s), focus group(s), critical incident(s), and phenomenology. For the All Fields searches, the mean value for BMLA searches is 2.3 while the mean value for JMLA searches is 8.3, an almost four-fold difference. For the Title OR Abstract searches, the mean value for BMLA searches is 0.1 while the mean value for JMLA searches is 0.85, an eight-fold difference. These statistical findings demonstrate an increase in the number of occurrences of qualitative research term over time. SR PMC search statement: BMLA # % PMC search statement: JMLA # % 1 "Bulletin of the Medical Library Association"[Journal] AND ("qualitative"[All Fields] OR “descriptive”[All Fields]) 518 7.1 ("Journal of the Medical Library Association : JMLA"[Journal]) AND ("qualitative"[All Fields] OR “descriptive”[All Fields]) 343 22.9 2 "Bulletin of the Medical Library Association"[Journal] AND ("qualitative design"[All Fields]) 0 0 ("Journal of the Medical Library Association : JMLA"[Journal]) AND ("qualitative design"[All Fields]) 1 0.3 3 "Bulletin of the Medical Library Association"[Journal] AND ("interview"[All Fields] OR "interviews"[All Fields]) 375 5.2 ("Journal of the Medical Library Association : JMLA"[Journal]) AND ("interview"[All Fields] OR "interviews"[All Fields]) 261 17.4 4 "Bulletin of the Medical Library Association"[Journal] AND ("focus group"[All Fields] OR "focus groups"[All Fields]) 48 0.7 ("Journal of the Medical Library Association : JMLA"[Journal]) AND ("focus group"[All Fields] OR "focus groups"[All Fields]) 119 7.9 5 "Bulletin of the Medical Library Association"[Journal] AND ("critical incident"[All Fields] OR "critical incidents"[All Fields]) 34 0.5 ("Journal of the Medical Library Association : JMLA"[Journal]) AND ("critical incident"[All Fields] OR "critical incidents"[All Fields]) 12 0.8 6 "Bulletin of the Medical Library Association"[Journal] AND ("phenomenology"[All Fields] OR "phenomenological"[All Fields]) 5 0.1 ("Journal of the Medical Library Association : JMLA"[Journal]) AND ("phenomenology"[All Fields] OR "phenomenological"[All Fields]) 4 0.3 Table 5a: Number of search results from PubMed Central queries of BMLA and JMLA, All Fields queries – qualitative research search terms An examination of research topics in JMLA and BMLA - Marton Hypothesis, vol. 28, no. 1, Fall 2016 21 SR PMC search statement: BMLA # % PMC search statement: JMLA # % 1 "Bulletin of the Medical Library Association"[Journal] AND (("qualitative"[Title] OR "qualitative"[Abstract]) OR ("descriptive”[Title] OR “descriptive"[Abstract])) 33 0.5 ("Journal of the Medical Library Association : JMLA"[Journal]) AND (("qualitative"[Title] OR "qualitative"[Abstract]) OR ("descriptive”[Title] OR “descriptive"[Abstract])) 50 3.3 2 "Bulletin of the Medical Library Association"[Journal] AND (("qualitative design"[Title] OR "qualitative design"[Abstract]) OR ("qualitative designs”[Title] OR “qualitative designs"[Abstract])) 0 0 ("Journal of the Medical Library Association : JMLA"[Journal]) AND (("qualitative design"[Title] OR "qualitative design"[Abstract]) OR ("qualitative designs”[Title] OR “qualitative designs"[Abstract])) 0 0 3 "Bulletin of the Medical Library Association"[Journal] AND (("interview"[Title] OR "interview"[Abstract]) OR ("interviews"[Title] OR “interviews"[Abstract])) 35 0.5 ("Journal of the Medical Library Association : JMLA"[Journal]) AND (("interview"[Title] OR "interview"[Abstract]) OR ("interviews"[Title] OR “interviews"[Abstract])) 52 3.5 4 "Bulletin of the Medical Library Association"[Journal] AND (("focus group"[Title] OR "focus group"[Abstract]) OR ("focus groups"[Title] OR “focus groups"[Abstract])) 4 0.1 ("Journal of the Medical Library Association : JMLA"[Journal]) AND (("focus group"[Title] OR "focus group"[Abstract]) OR ("focus groups"[Title] OR “focus groups"[Abstract])) 17 1.1 5 "Bulletin of the Medical Library Association"[Journal] AND (("critical incident"[Title] OR "critical incident"[Abstract]) OR ("critical incidents"[Title] OR “critical incidents"[Abstract])) 1 0 ("Journal of the Medical Library Association : JMLA"[Journal]) AND (("critical incident"[Title] OR "critical incident"[Abstract]) OR ("critical incidents"[Title] OR “critical incidents"[Abstract])) 1 0.1 6 "Bulletin of the Medical Library Association"[Journal] AND ((“phenomenology"[Title] OR "phenomenology"[Abstract]) OR ("phenomenological"[Title] OR "phenomenological"[Abstract])) 0 0 ("Journal of the Medical Library Association : JMLA"[Journal]) AND ((“phenomenology"[Title] OR "phenomenology"[Abstract]) OR ("phenomenological"[Title] OR "phenomenological"[Abstract])) 1 0.1 Table 5b: Number of search results from PubMed Central queries of BMLA and JMLA, Title OR Abstract queries – qualitative research search terms An examination of research topics in JMLA and BMLA - Marton Hypothesis, vol. 28, no. 1, Fall 2016 22 SUMMARY OF FINDINGS Progressively narrowing the searches through the addition of the second concept, the research- related terms, predictively reduces the number of search results, with more specific search terms reducing the number of search results considerably. Moreover, when the second search concept is restricted to [Title] OR [Abstract], there is a five to ten-fold reduction in the number of search results in comparison to the [All Fields] searches. The higher percentage of research-related search results for JMLA searches relative to BMLA searches is found consistently for all searches, for both the [All Fields] searches and the [Title] OR [Abstract] searches. Clearly, there is a marked increase in general and specific research-related terms in JMLA relative to its predecessor, BMLA. Paired t-tests conducted in the statistical package, SPSS (version 11) demonstrate that these differences are statistically significant for both the All Fields searches (t=4.092, df=19, p=.001) and the Title OR Abstract searches (t=2.615, df=19, p=.017). Overall, the quantitative data analysis of occurrences of research-related terms indicates that there is an increase in published research activity in the health sciences librarianship profession over time, with the largest noticeable increase for systematic reviews. DISCUSSION Conducting searches within issues of the Bulletin of the Medical Library Association and the Journal of the Medical Library Association reveal several interesting findings concerning the prominence of research in this leading health sciences librarianship journal over time, as measured by percentage of search terms. First, the word research appears in abundance in both BMLA and JMLA. The word study and its plural, studies, is also frequently found in BMLA and JMLA. The large number of search results suggests a prominent role for research in health sciences librarianship. Paired t-tests comparising BMLA searches to JMLA searches demonstrate statistically significant increases in occurrences of research-related terms over time for general research terms, quantitative research terms, qualitative research terms, and mixed-methods research terms. All research-related search terms are mentioned more frequently in JMLA than BMLA suggesting that health sciences librarianship research has increased since 2002, which supports a finding by Slutsky and Aytac concerning the increase in research in librarianship. As well, a greater diversity of study designs are being utilized. An examination of research topics in JMLA and BMLA - Marton Hypothesis, vol. 28, no. 1, Fall 2016 23 Surveys are popular. Surveys are commonly questionnaires. However, they can also take the form of interviews that utilize a pre-determined set of questions and list of responses. Well-known qualitative methods, such as open-ended interviews and focus groups are also popular. The predominance of quantitative, non-experimental designs, confirms findings from earlier research on LIS journals conducted by Koufogiannakis, Slater, and Crumley and by Slutsky and Aytac. Less well known qualitative study designs, such as the critical incident technique and phenomenological approaches, are infrequently found in both BMLA and JMLA. Systematic reviews, a form of secondary research, has gained considerably in visibility. The significant increase in mention of systematic reviews is an interesting trend that is reflective of the current interest in evidence-based medicine (EBM) and evidence-based practice (EBP). LIMITATIONS There are several limitations associated with this type of study. First, number of search results is a very basic quantitative measure that is largely devoid of context. This method can be refined by searching each individual issue of BMLA and JMLA and counting the number of search results per search term. As well, the occurrence of each search term in individual articles would represent a further refinement of this measure and determine to some extent the context in which these research-related terms are utilized. Second, health sciences librarians have other publication options, such as Health Information and Libraries Journal, Medical Reference Services Quarterly and Journal of Hospital Librarianship, to name but a few journal titles in this discipline. As well, they can publish their research in health sciences journals or information science journals. Thus, a quantitative study that focuses solely on counting the number of search results for research-related terms in one journal, albeit a prominent journal in its field, will only capture a fraction of the research studies published in the journal literature by health sciences librarians. FUTURE DIRECTIONS Future studies to investigate this research topic further include replication studies utilizing the research method employed in this study for investigation in other health sciences librarianship journals to determine if the same trends are present. As well, this research method could be utilized in future studies but with a more fine-grained approach to searching for research-related An examination of research topics in JMLA and BMLA - Marton Hypothesis, vol. 28, no. 1, Fall 2016 24 terms in specific issues of BMLA and JMLA. Additional research-related search terms could be considered for inclusion, such as thematic analysis, inductive analysis, theoretical framework, and inferential statistics, among others. Another avenue to explore is authorship metrics in relation to study design. For example, Wheeler, Yaniv and Fenske determined the most influential authors who are members of the MLA Research Section, according to number of citations in the Web of Science Citation Report. Carol Lefebvre’s articles on systematic reviews were in the top ten papers in the LIS discipline, suggesting a relationship between citation count and study design. [17] Author affiliation is also of interest. For example, Hardin and Stankus reported the institutional affiliations of the published academic science, engineering, agricultural, and medical librarians over a ten-year period (2000-2010). The LIS journals examined were Science & Technology Libraries; Issues in Science and Technology Librarianship; Journal of Agricultural and Food Information; Journal of the Medical Library Association; and Medical Reference Services Quarterly. The top US affiliations in this set of LIS journals were: Illinois, Purdue, Texas, Penn State and Cornell. [18] Exploring relationships between author h-index, author affiliation, article citation count, journal ranking, and study design may yield interesting findings. REFERENCES 1. Grefsheim SF, Rankin JA, Perry GJ, McKibbon KA. Affirming our commitment to research: the Medical Library Association’s research policy statement: the process and findings. Journal of the Medical Library Association. 2008;96(2):114-120. doi: 10.3163/1536-5050.96.2.114. 2. Dalrymple PW, Bastille JD, Bradley J, Dee CR, Humphreys BL, Marshall JG, Weller AC, Webb RE. Using scientific evidence to improve information practice. Chicago, IL: Medical Library Association, 1995. 3. Eldredge JD, Harris MR, Ascher MT. Defining the Medical Library Association research agenda: methodology and final results from a consensus process. Journal of the Medical Library Association.2009;97(3):178-185. doi: 10.3163/1536-5050.97.3.006. 4. Eldredge JD, Ascher MT, Homes JN, Harris MR. The new Medical Library Association research agenda: final results from a three-phase Delphi study. Journal of the Medical Library Association. 2012;100(3):214-218. doi: 10.3163/1536-5050.100.3.012 5. Ascher MT, Eldredge JD, Holmes HN, Harris MR. 2012. Medical Library Association. Research Section. Research Agenda Committee. MLA Research Agenda: Appraising the Best Available Evidence. http://repository.unm.edu/handle/1928/24634 https://dx.doi.org/10.3163%2F1536-5050.96.2.114 https://dx.doi.org/10.3163%2F1536-5050.97.3.006 https://dx.doi.org/10.3163%2F1536-5050.100.3.012 https://scholar.google.com/scholar?oi=bibs&cluster=11436103200852211256&btnI=1&hl=en https://scholar.google.com/scholar?oi=bibs&cluster=11436103200852211256&btnI=1&hl=en http://repository.unm.edu/handle/1928/24634 An examination of research topics in JMLA and BMLA - Marton Hypothesis, vol. 28, no. 1, Fall 2016 25 6. Eldredge JD, Ascher MT, Holmes HN. Medical Library Association Research Section. Research Agenda Committee. MLA Research Agenda: Systematic Review Project Team Updates Presentation. MLA Annual Meeting. May 17, 2015. http://repository.unm.edu/handle/1928/27127 7. SCImago. (2016). About us. November 8, 2011. 8. SCImago. (2016). Journal rankings: library and information sciences. November 8, 2011. 9. 2015 Journal Citation Reports® Social Sciences Edition (Thomson Reuters, 2016) 10. Koufogiannakis D, Slater L, Crumley E. A content analysis of librarianship research. Journal of information science. 2004;30(3):227-239. doi: 10.1177/0165551504044668 11. Slutsky B, Aytac S. Publication patterns of science, technology, and medical librarians: Review of the 2008-2012 published research. Science & Technology Libraries. 2014;33(4):369-382. doi: 10.1080/0194262X.2014.952486 12. Slutsky B, Aytac S. Bibliometric analysis and comparison of two STEM LIS Journals: Science & Technology Libraries and Issues in Science & Technology Librarianship (2005–2014). Science & Technology Libraries. 2016;35(2):152-171. doi: 10.1080/0194262X.2016.1171191 13. Creswell JW. Research Design: Qualitative, quantitative, and mixed methods approaches. 2nd ed. Thousand Oaks, CA: Sage Publications; 2002. P. 153. 14. Garg AX, Hackam D, Tonelli M. Systematic Review and Meta-analysis: When One Study Is Just not Enough. Clinical Journal of the American Society of Nephorology. 2008, 3(1): 253–260. doi: 10.2215/CJN.01430307. P. 253. 15. Creswell JW. Research Design: Qualitative, quantitative, and mixed methods approaches. 2nd ed. Thousand Oaks, CA: Sage Publications; 2002. P. 182. 16. Creswell JW. Research Design: Qualitative, quantitative, and mixed methods approaches. 2nd ed. Thousand Oaks, CA: Sage Publications; 2002. P. 211. 17. Wheeler TR, Yaniv N, Fenske RE. (2015). A look at the scholarly output of the Medical Library Association Research Section. Hypothesis. 2015;27(1):3-7. 18. Hardin A, Stankus T. The affiliations of U.S. academic librarians in the most prominent journals of science, engineering, agricultural, and medical librarianship, 2000-2010. Science & Technology Libraries. 2011;30(2): 143-156. doi: 10.1080/0194262X.2011.575285 Author Affiliations 1 Adjunct Instructor, Faculty of Information, University of Toronto, Toronto, Ontario; christine.marton@utoronto.ca http://www.scimagojr.com/aboutus.php http://www.scimagojr.com/%20journalrank.php?category=3309 https://scholar.google.com/citations?view_op=view_citation&hl=en&user=GI865CUAAAAJ&cstart=20&sortby=pubdate&citation_for_view=GI865CUAAAAJ:u-x6o8ySG0sC http://dx.doi.org/10.1080/0194262X.2014.952486 http://dx.doi.org/10.1080/0194262X.2016.1171191 http://dx.doi.org/10.1080/0194262X.2011.575285 mailto:christine.marton@utoronto.ca