
















































AWARDS 
 

 

 

Hypothesis 
Vol. 33 No.1  

Fall/Winter 2021 

2021 Research Caucus Research Awards  
 
Congratulations to the Winning Research Papers and Posters from MLA ’21 Virtual! 
 
The MLA Research Caucus is pleased to announce the winners for best research 
papers and posters presented at the MLA 2021 Virtual Meeting. Thank you to all the 
judges who volunteered their expertise to help select these deserving awardees both in 
the pre-judging phase and at the virtual conference. To learn more about the awards 
and selection process, visit the MLA Meeting Poster and Paper Awards page from the 
Research Caucus community information pages at 
http://www.mlanet.org/p/cm/ld/fid=938 
 

MLA ’21 WINNING CONTRIBUTED PAPERS 

 
First Place - Health Sciences Librarians’ Engagement in Open Science: A Scoping 
Review 
 

• Kevin Read – Associate Librarian, University of Saskatchewan 
• Dean Giustini – UBC Biomedical Branch Librarian, University of British Columbia 
• Ariel Deardoff – Data Services Librarian, UCSF Library 
• Lisa Federer – NLM Data Science and Open Science Librarian, National Library 

of Medicine 
• Melissa L. Rethlefsen - Executive Director, Health Sciences Library & Informatics 

Center at The University of New Mexico Health Sciences Center 
 

 
Objectives: To identify health sciences librarians’ (HSLs) engagement in open 
science (OS) through the delivery of library services, support, and programs for 
researchers. 
 
Methods: We performed a scoping review guided by Arksey and O’Malley’s 
framework and Joanna Briggs’ Manual for Scoping Reviews. Our search methods 
consisted of searching five databases (MEDLINE, Embase, CINAHL, LISTA, and 
Web of Science Core Collection), tracking citations, contacting experts, and targeted 
web searching. We used Zotero to manage citations, and Covidence for screening. 
To determine study eligibility, we applied predetermined inclusion and exclusion 
criteria, achieving consensus among reviewers when there was disagreement. 
Finally, we extracted data in duplicate and performed qualitative analysis to map key 
themes. 

http://www.mlanet.org/p/cm/ld/fid=938


AWARDS 
 

 

 

Hypothesis 
Vol. 33 No.1  

Fall/Winter 2021 

 
Results: We identified 54 included studies after reviewing 8173 citations and 319 full 
text studies. Research methods included descriptive or narrative approaches (76%), 
surveys, questionnaires and interviews (15%) or mixed methods (9%). Publication 
types included case studies (46%), journal articles (39%), conference posters and 
abstracts (9%), book chapters (4%), and reports (2%). Using FOSTER’s Open 
Science Taxonomy, we labeled studies with one or more of six themes: open access 
(54%), open data (43%), open science (24%), and open education, open source and 
citizen science (17%). Key drivers in OS were scientific integrity and transparency, 
openness as a guiding principle in research, and funder mandates making research 
openly-accessible. HSLs engaged in advocacy for OS in many ways, with most 
examples coming from academic institutions. HSLs assumed key roles by 
advocating for and promoting OS, and by collaborating on policy development, 
especially in support of OA and open data. 
 
Conclusions: HSLs play key roles in advancing OS worldwide. However, more 
formal studies are needed to assess the impact of HSLs’ engagement in OS to 
determine best practices. Future studies should identify OS researchers’ needs, and 
evaluate the library service models best designed to meet them. HSLs should 
promote broader adoption of OS within their research communities, and develop 
strategic plans aligned with institutional partners. Further, HSLs can promote OS by 
adopting more rigorous and transparent research practices of their own. Future 
research should consider examining HSLs’ engagement in OS through social justice 
and equity perspectives. 

 
 
Second Place - Considerations for Conducting Evidence Syntheses: Evaluating 
the Performance of Various Electronic Methods for De-duplicating References 
 

• Sandra McKeown – Health Sciences Librarian, Queen’s University 
• Zuhaib M. Mir – Resident Physician, Queen’s University 

 
Objectives: The objective of this study was to determine the accuracy and 
performance of commonly-used electronic methods for de-deduplicating references 
retrieved from searching multiple bibliographic databases. 
 
Methods: A heterogeneous sample of references was obtained by conducting a 
similar topical search in MEDLINE, Embase, PsycINFO and Cochrane CENTRAL. 
Manual abstraction was used to identify duplicate references among the search 
results and develop a gold standard for comparison. De-duplication methods 



AWARDS 
 

 

 

Hypothesis 
Vol. 33 No.1  

Fall/Winter 2021 

included Ovid multifile search, EndNote desktop (versions X9 and 20), Mendeley, 
Zotero, Covidence and Rayyan. False negative and false positive duplicate 
references for each method were identified and then the accuracy, sensitivity, and 
specificity of each was calculated. 
 
Results: Default de-duplication settings in Ovid multifile search and dedicated review 
software (Covidence and Rayyan) significantly outperformed reference management 
software. Ovid, Covidence and EndNote 20 possessed the highest specificity (1.00, 
1.00 and 0.998, respectively) for identifying duplicate references, while Rayyan 
demonstrated the highest sensitivity (0.96). The accuracy of EndNote desktop 
improved from the X9 version (0.76) to the newer 20 version ( 0.92). 
 
Conclusions: This study highlights the strengths and weaknesses of commonly-used 
electronic methods for de-duplicating references and suggests strategies to avoid 
unintentionally removing eligible studies and introducing bias into syntheses. When 
selecting database platforms and utilizing software programs for the review process, 
de-duplication functionality and performance is an important consideration for 
improving the efficiency and quality of evidence syntheses. 

 
Honorable Mention - Health Science Librarians’ Engagement in Work-Related 
Reflection: Results of a Qualitative Exploration of Why They Engage in Reflective 
Practice 
 

• Jolene M. Miller – Director, Mulford Health Science Library, University of Toledo 
 

Objectives: Published research on health science librarians use of reflection at work 
(reflective practice) tends to be results of action research and surveys. The current 
study fills a gap in the literature, exploring the following questions: As librarians 
engage in intentional reflection to improve work performance, what are they actually 
doing? Why do they invest time and energy in reflection in the first place? The 
purpose of this qualitative study is to better understand how health science librarians 
experience and understand reflective practice. 
 
Methods: In 2020, 18 health science librarians who use reflection at work were 
recruited using email invitations sent to medical/health science librarianship email 
distribution lists based in the US and Canada. There were no limitations on how, to 
what extent, or how long participants had been using reflection. They were 
interviewed using a video conferencing platform about their use of reflection at work 
and what advice they would give to someone new to reflective practice. The 
interview transcripts were analyzed within a phenomenological framework for 



AWARDS 
 

 

 

Hypothesis 
Vol. 33 No.1  

Fall/Winter 2021 

themes using MAXQDA 2020 software. (Prior to recruitment, this study was 
reviewed and identified as exempt research by the University of Toledo Social, 
Behavioral, and Educational Institutional Review Board.) 
 
Results: Several themes emerged from the analyses: (1) participants reported a 
variety of benefits from engaging in reflection at work; (2) the importance of 
intentionality in reflection; (3) the role of emotion in reflective practices; and (4) the 
wide range of reflective strategies used by participants. The primary themes 
regarding advice to people new to reflection: (1) start small; and (2) find what works 
best for you. 
 
Conclusions: This project provides rich detail about the ways that health science 
librarians engage in using reflection at work. Perhaps the most important finding is 
that librarians used reflection in ways that work best for them in terms of their 
workload, work environment, personality, and extent of experience with reflection. 
Hopefully, this encourages health science librarians to consider using reflection to 
improve their work performance without fear that they are “doing it wrong.” 

 
MLA ’21 WINNING CONTRIBUTED POSTERS 
 
First Place - Developing and Conducting a DEI Language Inclusivity Assessment 
on a Health Science Library’s Website 
 

• Jane Morgan-Daniel – Community Engagement and Health Literacy Liaison 
Librarian, Health Science Center Libraries, University of Florida 

• Matthew Daley – Web Designer, Health Science Center Libraries, University of 
Florida 

• Mary E. Edwards – Liaison Librarian, Health Science Center Libraries, University 
of Florida 

• Lori H. Eubanks – Office Assistant, Health Science Center Libraries, University of 
Florida 

• Hannah F. Norton – Chair, Health Science Center Libraries, University of Florida 
• Nina Stoyan-Rosenzweig – Senior Associate in Libraries, Health Science Center 

Libraries, University of Florida 
 

Objectives: The Diversity, Equity, and Inclusion (DEI) Team at a university health 
science library sought to conduct a language inclusivity assessment of their library’s 
public-facing website. The project’s objectives were to 1) create a checklist for 
inclusive language, 2) review the library’s webpages using the checklist, 3) update 



AWARDS 
 

 

 

Hypothesis 
Vol. 33 No.1  

Fall/Winter 2021 

the webpages accordingly, and 4) circulate the checklist to the university’s other 
libraries for potential use. Language inclusivity was defined as “language that is free 
from words, phrases or tones that reflect prejudiced, stereotyped or discriminatory 
views of particular people or groups” (British Columbia Public Service). 
 
Methods: First, the team searched for literature relating to best practices for 
language inclusivity. Nineteen useful resources were located, from which the team 
compiled a 32-item checklist, covering demographic information collected in 
webforms, as well as identification of gendered language, stereotypes, ableist 
language, and out-of-date terminology. With this checklist, the team reviewed the 33 
webpages of the library’s website, not including LibGuides or links to external 
webpages. Each webpage was assessed by one of five team members, who used a 
spreadsheet pre-populated with the checklist to record language that needed to be 
changed. The team also made a note of DEI-related content that could potentially be 
added, based on the language inclusivity literature we had consulted before 
beginning the assessment. 
 
Results: Nineteen instances were recorded for which improvements needed to be 
made, encompassing the following topics: adding pronouns for employees who opt 
in; referring to invisible disabilities on our “users with disabilities” page; using 
student-first language instead of faculty-first; spelling out acronyms; and removing 
outdated language with negative connotations such as “earmarked”, as this term has 
been associated with enslavement. The team’s recommendations for website 
content additions included creating a DEI statement; developing a code of conduct 
for the library’s online and physical spaces; and adding information about parking, 
assistive technologies, and emergency evacuation procedures to our “users with 
disabilities” page. 
 
Conclusions: The website updates are currently underway and the checklist has 
been circulated to the six other university libraries as planned. The next stage of the 
project is to expand the website review to include the library’s Archives pages; this 
poses the additional challenge of recognizing and appropriately delineating historical 
language where appropriate, so our library’s Archivist has joined the project team. 
Following completion of the Archives pages, the DEI Team plans to use the checklist 
to begin assessing our library’s LibGuides, signage, and non-digital flyers and forms. 

 
Second Place - The Status of Scholarly Efforts of Librarians in Health Literacy: A 
Bibliometric Analysis 
 



AWARDS 
 

 

 

Hypothesis 
Vol. 33 No.1  

Fall/Winter 2021 

• Alexandria Q. Wilson – Research & Learning Services Librarian, University of 
Tennessee Graduate School of Medicine 

• Courtney Wombles – Graduate Research Assistant, University of Tennessee 
Graduate School of Medicine 

• Robert E. Heidel – Associate Professor of Biostatistics, University of Tennessee 
Graduate School of Medicine 

• Kelsey L. Grabeel – Assistant Director, University of Tennessee Graduate School 
of Medicine 

 
Objectives: Over the past 20 years, health literacy has increasingly impacted patient 
care. Recently, Healthy People 2030 updated the definition to include personal 
health literacy as well as organizational health literacy. As information professionals, 
librarians can be leaders of helping to improve health literacy through their roles in 
the classroom, public library, or in the healthcare environment. In order to determine 
the scholarly impact librarians may have had on health literacy, researchers 
examined the status and characteristics of health literacy (HL) publications authored 
by librarians from 2000 to 2019. 
 
Methods: Bibliometrics, the statistical analysis of publications, is used to assess the 
indicators of productivity, collaboration, and impact or visibility of librarians in health 
literacy-related research. Data was collected using the multidisciplinary database 
Scopus and articles were screened for inclusion criteria using Rayyan. Through 
bibliometrics, quantitative analysis will be performed, including examination of 
number of citations, number of publications per year and per country, the most 
prolific authors and their affiliations, most prolific journals and journal impact factors, 
number of articles published in library science-related and non-library science-
related journals, the origin of the first author’s country, number of articles per 
research type, etc. The itsa (Interrupted Time Series analysis) procedure in Stata 
was used to assess the trends in publications pre- and post-2013. 
 
Results: Of 321 search results, 219 unique references met the inclusion criteria of 
librarian authorship. Before 2013, the rate of HL publications increased at a rate of 
1.30 annually (95% CI 0.97–1.62, p<0.001). After 2013, the number of HL 
publications increased at a rate of 4.18 publications (95% CI 3.71–4.65, p<0.001) 
annually with the peak in 2019 (n=32). Journal of Consumer Health on the Internet 
was the most prolific journal. The majority of included references were authored by 
at least two authors with a collaboration index of 2.58. In terms of impact, 79% of 
included articles were cited at least once. 
 
Conclusions: In the last two decades, librarian involvement in HL publications has 



AWARDS 
 

 

 

Hypothesis 
Vol. 33 No.1  

Fall/Winter 2021 

exponentially increased, most markedly in the years following 2013. Librarian 
authors are highly collaborative, but few have published on health literacy multiple 
times. The visibility, volume, and collaboration efforts show how engaged librarians 
are in health literacy scholarship. Further research is needed to examine the 
research and impactful efforts made by librarians that aren’t published. 

 
Second Place - Find the Studies Without the Noise: Validation of the Performance 
of a Pragmatically Designed PubMed Methodological Filter 
 

• Nancy Firchow – Senior Librarian, California Environmental Protection Agency, 
Office of Environmental Health Hazard Assessment  

• Melinda Davies – Associate Research Informationist, Kaiser Permanent Center 
for Health Research 

 
Objectives: Comprehensive review searches can generate a high volume of results, 
including many non-empirical publications that present no original patient/participant 
research data and will be screened out during the title/abstract review phase. Using 
a set of included studies from previously published systematic reviews (SRs), we 
demonstrate the relative recall performance of a pragmatically-designed PubMed 
search filter, the Non-Empirical Publication (NEP) filter, intended to distinguish and 
exclude non-research publications from comprehensive searches and still retrieve 
publications of interest. 
 
Methods: We identified 32 SRs published in 2017 and indexed in PubMed which 
reported their included studies. The focus of the SRs included treatment, 
epidemiology, diagnosis, and other. The SRs included studies of a broad range of 
methodologies, including RCTs, non-randomized clinical trials, observational 
studies, and case reports. We measured the ability of the NEP filter to retrieve the 
Pubmed-included studies included in these reviews. 
 
Results: The 32 SRs included a total of 1172 studies, of which 1062 were found in 
PubMed by known-item searching. The NEP filter demonstrated a relative recall of 
0.98 across all 32 reviews. Relative recall for SR subsets was as follows: Treatment 
SRs - 0.97; Epidemiology SRs - 0.98; Diagnosis SRs - 0.98; Other SRs - 0.99 
 
Conclusions: The NEP filter performs very well in retaining studies of interest using a 
relative recall method. It holds its high recall rate across SRs of different types and 
across included studies of a broad range of methodologies. Work is continuing to 
establish sensitivity, precision and reduction in number needed to read. 

 



AWARDS 
 

 

 

Hypothesis 
Vol. 33 No.1  

Fall/Winter 2021 

Honorable Mention - Medical Student Question Formulation Skills Development: 
A Randomized Controlled Trial 
 

• Jonathan D. Eldredge – Associate Professor, University of New Mexico 
• Melissa Schiff – Professor, University of New Mexico 
• Jens Langsjoen – Hospitalist, University of New Mexico 

 
Objectives: Eldredge et al. reported at MLA 2020 on their quasi-experiment involving 
a new FAC (Focus, Amplify, Compose) rubric for assessing medical students’ 
Evidence Based Practice (EBP) question formulation performance. This new rubric, 
when linked to complementary hands-on training, improved student scores. This 
study builds on the previous study by measuring whether the rubric alone, or when 
combined with a 25-minute intervention, will be more effective in teaching second-
year medical students on how to formulate effective EBP questions. 
 
Methods: Randomized Controlled Trial. 80 second-year medical students were 
randomized into either an intervention group or a control group prior to the beginning 
of the block. The authors administered a pre-test to all students that presented them 
with a clinical vignette and asked them to formulate an EBP question. Following the 
pre-test, all students were shown the question formulation rubric and it was briefly 
explained. Students in the intervention group were taught how to formulate EBP 
questions using the rubric followed by another 30 minutes of EBP search training. 
Students in the control group only received the EBP search training in their small 
group labs. All 80 students took the post-test in which they formulated a question in 
response to a clinical vignette. Statistical analysis will involve a two-sample paired t-
test to test between-group differences. 
 
Results: The authors collected their pre-test and post-test data for the intervention 
and the control groups during December 2020. The authors will report their findings 
at the MLA 2021 Annual Meeting. 
 
Conclusions: The authors will offer their conclusions during the MLA 2021 Annual 
Meeting. 

 


	MLA ’21 WINNING CONTRIBUTED PAPERS
	MLA ’21 WINNING CONTRIBUTED POSTERS

