Research Section Spotlight
Hypothesis, vol. 30, no. 1, Fall/Winter 2018 33
Research Roadmap: Understanding the Research
Process (Part 1)
Kristine M Alpi, PhD, MPH, MLS, AHIP
University Librarian
Oregon Health & Science University Library
Oregon Health & Science University
Portland, OR
krisalpi@gmail.com
Jonathan D Eldredge, PhD, MLS, AHIP
Evidence Based and Translational Science Collaboration Coordinator
Health Sciences Library and Informatics Center
University of New Mexico
New Mexico, NM
jeldredge@salud.unm.edu
Heather N Holmes, MLS, AHIP
Associate Director of Libraries
MUSC Libraries
Medical University of South Carolina (MUSC)
Charleston, SC
holmesh@musc.edu
Katherine G Akers, PhD
Biomedical Research/Research Data Specialist
Shiffman Medical Library
Wayne State University
Detroit, MI
katherine.akers@wayne.edu
Kimberly R Powell, MLIS
Research Impact Informationist
Woodruff Health Sciences Center Library
Emory University
Atlanta, GA
krpowel@emory.edu
Margaret Hoogland, MLS, AHIP
Assistant Professor, Clinical Medical Librarian
Mulford Health Sciences Library
University of Toledo
Toledo, OH
mahoogla@gmail.com
Research Section Spotlight
Hypothesis, vol. 30, no. 1, Fall/Winter 2018 34
*Editor’s note: This piece has been split into two parts due to length. Part 1 covers advice and
guidance about doing research while Part 2 (which will be in the Hypothesis Spring/Summer
2019 issue) discusses how to share the products of your research.
Introduction
Since 1982, the MLA Research Section has fostered discussion, shared advice, connected
members to potential collaborators and research mentors, and kept its membership up-to-
date on library research efforts, particularly as they apply to a wide variety of academic
life sciences and community health efforts. Towards these endeavors, the 2017-2018
Research Section program planners proposed a Special Content session on understanding
the research process for the 2018 MLA Annual Meeting & Exhibition in Atlanta, GA. The
session was envisioned as a panel discussion for an audience of new to intermediate
librarian researchers that would offer advice around getting started, from formulating an
idea, to study design, data collection, and finally dissemination and the publication
process. Other MLA Sections--Leadership and Management Section, Hospital Library
Section, and Educational Media and Technologies Section--offered their support. Panelists
were recruited based on their previous involvement in continuing education and editorial
work regarding library research, and several belonged to multiple Sections sponsoring the
session. In discussion with the Research Section Program Chair [Billman] and the session
moderator [Powell], each panelist [Akers, Alpi, Eldredge, Holmes] selected to focus on an
area most relevant to their work, with the recognition that all the presenters had expertise
across the following topics comprising the presentation:
• Overview of the research process
• Finding and working with a research mentor
• Turning daily work into a research project
• Choosing research methods that fit you and your question
• Using technology and data collection methods efficiently
• Disseminating your research
• Understanding the publishing process
“Research Roadmap: Understanding the Research Process,” was presented on May 20,
2018, and 2018 MLA Annual meeting attendees can access the recorded session by logging
into the online program with their registration information
(https://www.eventscribe.com/2018/MLA/). This account of the session is presented so
those unable to view the recordings may benefit from the panelists’ advice and expertise.
Making a Member-Driven Presentation
https://www.eventscribe.com/2018/MLA/
Research Section Spotlight
Hypothesis, vol. 30, no. 1, Fall/Winter 2018 35
In planning the session, we wanted to respond to MLA members’ questions about the
research process. On April 5, 2018, Billman posted a call for questions in the MLA-FOCUS,
an email update sent to all MLA members. Questions received before the submission
deadline were addressed in the content presented. The session incorporated two question
and answer periods, one after the first two speakers and the second at the end. At the
beginning of the session, a slide encouraged the audience to “Submit Your Questions”
using a tinyurl online survey or on cards. Those not addressed by the presenters during
the session have responses from our panelists in the Appendix.
The Roadmap as Presented
Overview of the Research Process
We began by reviewing the Evidence Based Practice Process and the opportunity for
research after the existing literature has been critically appraised, and before its
application to the problem that spurred the initial search for evidence. Moving on to the
research process, we shared a four-step version for which we would provide our
experiences:
1. Formulate a Research Question (Objective)
2. Design an appropriate method to answer the question
3. Interpret your results
4. Communicate your results
As we developed our expertise through being mentored and mentoring others, and
because several of the questions we received were about finding mentors or collaborators,
we addressed mentorship first.
Finding and Working with a Research Mentor
A mentor can be defined as “an individual with advanced knowledge, usually more senior
in some regard, who is committed to providing upward career mobility and assistance for
the protégé …” Moreover, “The mentoring relationship has been characterized as an
intense, sometimes intimate professional relationship devoted to providing social support
and development for the protégé’s career.”[1] Johnson observes that mentoring
represents an enduring and reciprocal personal relationship in which the mentor serves
as a role model while offering a safe environment for the protégé to consider new ideas. In
this context, the mentor provides acceptance, protection, challenge and coaching for the
protégé.[2]
While many studies have investigated career or academic mentoring, few investigations
have rigorously explored the specialized world of mentoring other professional colleagues
in conducting research. And almost all investigations into research mentoring have
Research Section Spotlight
Hypothesis, vol. 30, no. 1, Fall/Winter 2018 36
focused upon faculty-student mentoring. Eldredge has mentored more than 20 colleagues
over the past two decades. A few of these colleagues have been new faculty members in
his medical school whereas most have been library or information practitioners wanting
guidance with research projects. These early experiences, coupled with limited time for
mentoring individuals, led to sponsoring “The Research Mentor” column in Hypothesis
during 2008-2014. These columns featured topics about which his protégés needed
recurring guidance ranging from creativity in research [3, 4, 5, 6], to defining authorship
[7, 8], to the pragmatic aspects of research [9, 10]. They also included interviews with
esteemed researchers [11, 12] and a reflection upon a past mentor’s long-term effect upon
his own career [13]. Additionally, the author participated in several of the University of
New Mexico’s Mentoring Institute’s Research Conferences [14].
Roles of the protégé
Generalizing from these experiences, the protégé needs to practice reciprocity in the
mentoring relationship. The research mentor brings a wealth of knowledge and skills on
how to conduct a research project. The gains to the protégé are obvious. Yet, what does
the protégé bring to the mentoring relationship? Inventorying the attributes possessed by
the protégé might strengthen the relationship:
1. Previous research education
The mentoring relationship should not be confused with a tutoring relationship in which
the mentor provides comprehensive one-on-one instruction on all aspects of research. The
protégé should bring previous learning on research methods to the relationship. The
protégé can learn research methods by taking or auditing courses at nearby or online
academic institutions, such as those in subjects such as psychology, biology, or one of the
social sciences. The new MLA Research Training Institute might provide an avenue for
some protégés to gain research methods education and identify mentors, but this new, not
yet evaluated, program does not offer the depth of training found in a graduate-level
research course.
2. Literature searching
Every research process involves literature searches in the formative stages due to the
need to perform iterative literature searches until the final research question reflects the
state of the existing research knowledge on the topic to be investigated. Designing the
research methodology provides another stage in which one or more literature searches
will inform how the research project will unfold. Finally, once the research is done,
additional literature searching will assist in interpreting and discussing the results.
Librarians who are capable literature searchers can assist a research project with their
unique skill sets.
3. Organization
Research Section Spotlight
Hypothesis, vol. 30, no. 1, Fall/Winter 2018 37
Librarians are legendary for their organizational skills, and many have learned project
management skills that have direct application to the complex dimensions of a research
project. Librarians also can use bibliographic management tools such as Zotero or
EndNote to organize the hundreds of references associated with any research project. A
recently published scoping review involved elaborate bibliographies and analytics,
highlighting the protégé’s contribution of an impressive skill set to the project [15].
4. Track record
Many librarians have contributed to research projects as part of their everyday work.
These roles often involve literature searching and bibliographic management. While most
involvement with research projects takes a more episodic form, some projects involve
larger commitments. These projects form a track record of librarian involvement in past
research projects that librarians can point to when approaching a research mentor for
guidance.
5. Commitment
The potential protégé needs to bring a commitment to completing the research project to
the mentoring relationship. Experienced researchers know that many well-intended
research projects are abandoned at various phases prior to completion. Ideally, the
protégé can present the potential mentor with a plan and timeline that will advance the
research project through completion and communication to the profession through poster,
presentation, or publication of the results. The multi-phased MLA Research Agenda
project [16] tested the commitment of all researchers involved yet when it reached
completion, it provided a great service to our profession [17].
6. Creativity
Established researchers sometimes descend so deeply into the weeds of the research
design and its careful implementation that we forget to consider the opportunities to lend
some creativity to the project. A noteworthy example surfaced when one of the authors
included three medical students in a randomized controlled trial on Point-of-Use tools.
The three medical students expressed their creativity by producing a top-notch training
video for participants in the study. In other ways, the students also added elements of
creativity unrecognized by the veteran researchers and thereby improved project
participation among providers [18].
7. Enthusiasm
A few years ago, one of the author asked an audience of translational researchers what
adjectives they would use to describe the research process. The adjectives offered were
“tedious” and “detailed” and “comprehensive.” There are admittedly boring moments in
any research project in which one can be tempted to walk away. Toward the end of the
project, however, excitement builds as results, particularly difficult to explain results,
Research Section Spotlight
Hypothesis, vol. 30, no. 1, Fall/Winter 2018 38
emerge in need of interpretation. A protégé with enthusiasm most likely will rely on it to
muddle through the less exciting aspects of a research project. One applied informatics
research study that comes to mind involved a lot of tedium, but the researchers all plowed
through the hours of careful checking in this fidelity study to produce surprising results
[19].
Initiating the mentoring relationship
We have focused on how the protégé seeking to recruit a research mentor needs to step
outside the limits of her or his own perceptions to consider the perspective of the
potential mentor(s). The MLA Mentoring/Expertise Directory
(http://www.mlanet.org/page/mentoring) is one way to find mentors who have already
indicated willingness to engage in mentoring relationships. In other cases, you may
identify someone in the literature or at your institution with expertise you aspire to
develop and there you will not know their willingness to mentor until you make the
approach. It is up to the mentorship dyad to set mutually agreeable terms of the
relationship. The respect is reciprocal, and it is also clear that the more experienced
researcher can gain insight and satisfaction, and sometimes co-authorship from these
relationships. In some cases, an enduring relationship develops that leads to new studies
as collaborators.
Turning Daily Work into a Research Project
There are many ways to turn daily work into a research project; it is a matter of looking
and asking thoughtful questions about what work is being done. More specifically, it
means looking at what we are, or perhaps should be, measuring. Valuable research can be
performed at many levels; not every question is best addressed by a large, multi-center
NIH-funded study. This is especially true in library research projects because much of
what we measure and seek to improve is part of our daily work. One of the key pieces of
evidence-based library and information practice (EBLIP) is applying local evidence to
evidence in the literature, and to our own professional knowledge. Considering the five
principles of EBLIP as presented by Koufogiannakis and Brettle [20] may help us better
understand ways we can turn daily work into research:
Articulate: A clear understanding of the problem or question must be reached. Part of
articulating the problem includes working out what is known already and why the
information is needed, as well as ensuring that the problem is set in the appropriate
context.
Assemble: Evidence should be assembled from multiple sources that are the most
appropriate to the question or problem at hand and should include research evidence,
local evidence, and professional knowledge.
http://www.mlanet.org/page/mentoring
Research Section Spotlight
Hypothesis, vol. 30, no. 1, Fall/Winter 2018 39
Assess: Evidence should be assessed for its quality (often known as appraisal or critical
appraisal), determining what the evidence says as a whole.
Agree: Determine a course of action and begin implementation of the decision. If working
with a group, try to achieve consensus based on the evidence and organizational goals.
Adapt: Evaluate the decision and how it has worked in practice. Revisit goals and reflect
on the success of implementation.
Applied research at the Medical University of South Carolina (MUSC)
Keeping these principles in mind and thinking through the work that we do daily at the
library, a relatively simple research project grew out of updating LibGuides. We were
looking at click stats on our guides and noticed some irregularities such as usage
considerably different from what we expected. That started us in the first principle of
EBLIP, asking (articulating) the question of why these irregularities were present. The
research project developed from seeing the numbers and articulating the question. Before
we knew it, we had developed the project. Ultimately, looking at the clicks in our
LibGuides gave us an opportunity to assemble the evidence, assess it, and decide (agree)
on what to do with what we learned.
Using what we learned, we decided on ways to reformat our Guides and to reconsider
placement of links based on perceived importance. If things we felt were important had
been prominently placed, but were not getting many clicks, we moved them a little further
down the page since fewer clicks seemed to indicate less importance to the user.
Conversely, when we found things placed lower getting a lot of use, we relocated them to a
more prominent position. It is a very simple project, and one that we can easily adapt
over time. We will revisit the statistics in the future to assess whether our repositioning
made any difference.
It is important to note that this simple project led to more questions being asked and
therefore more research needing to be done. A secondary outcome (that we had not
considered until we really looked at the data) was the realization that we needed to adjust
our instruction sessions to include more guidance on navigating LibGuides, as well as
accessing the resources therein. A follow-up study looking at data from and the questions
being asked through LibAnswers is also being launched. The results will also factor into
future LibGuides updates and we expect it to bring out even more questions and things to
study. This is one small example of taking our everyday work and turning it into research.
It is a project we know we will learn from, and we anticipate publishing our results to
share our procedures with others who may consider a similar project. For another
example from a different institution, see Lindsay, Oelschlegel & Earl [21].
Research Section Spotlight
Hypothesis, vol. 30, no. 1, Fall/Winter 2018 40
Identifying and overcoming obstacles
How do we find time to do research when we often feel like we do not have enough time to
do our day-to-day job? Further, how do we find time to do research when it is not
expected of our position, or when our superiors are not supportive? This author [Holmes]
feels that the best way to achieve time for research is to simply prioritize. That means
figuring out what can and cannot be shuffled, eliminated, or delegated from your daily
work. It also means setting aside dedicated time in your schedule to get the research done,
even if it is on Saturday afternoon [Holmes] or in thirty-minute blocks before work while
others are sleeping [Alpi]; see Chronicle of Higher Education [22] for more ideas. You must
set boundaries so that your dedicated time is not always interrupted or sacrificed, and you
also must enforce the boundaries.
If you are passionate enough about a topic and want to research it, then you must be
willing to pursue it on your own time. There are many librarians who are either not in
positions where there are expectations of publishing and presenting or who lack support
from their superiors or even their colleagues, yet they want to be involved in advancing
our profession. It is important for all library staff with questions to find their own way to
pursue their interests, even if it is on their own time. In fact, most librarians spend a
portion of their own time (and often their own money) to pursue their professional goals
and interests. The results that will come from taking the time to work on projects will help
your resume/curriculum vitae (CV) for future opportunities.
Another obstacle that we frequently encounter is lack of access to resources, whether it is
software or people or something else. Your mentor or your professional association can
help you find these resources, but your own professional networking efforts will probably
yield the biggest benefit. If someone’s research interests you, it is entirely appropriate to
contact the researcher to say you would like to know more about the work or perhaps
even suggest a collaboration. It helps to have a scholarly presence online (e.g. LinkedIn or
Google Scholar Profiles) so they can learn more about you before responding to your
inquiry. Collaboration is one of the keys to success, and it requires work on your end to
make those meetings happen. Collaborators can provide resources like software or
expertise, but they can also end up serving as a mentor or a network to yet another person
with whom you may be able to work with in the future.
While a lack of understanding of the research process is an obstacle for early researchers,
it is very much a learning opportunity. Looking to the literature is one of the best ways to
get an introduction to the process, and again working with a mentor can help with
guidance. Educational opportunities are increasing, and the MLA’s Research and
Evidence Based Practice Curriculum Committee is in the process of building an
education curriculum for learning about research. A final suggestion for learning about
the research process, and a general way to get more involved in research, is to engage with
Research Section Spotlight
Hypothesis, vol. 30, no. 1, Fall/Winter 2018 41
your Institutional Review Board (IRB)/ethics committee. If you are going to study your
users or analyze educational output from previous courses for a presentation or
publication to the outside world, you will need to have your proposed project reviewed by
the IRB. Understanding the background of your IRB members or looking into becoming an
IRB member yourself may be an excellent engagement opportunity to learn about
research at your institution, and how you might find a partner. No matter your role, you
will learn much more about the process, and when the time comes for your own research
with human participants, you will know what you have to do to prepare your own
submission for approval. Always be sure to check with your IRB prior to starting your
research so you will know if what you propose will qualify as research or be considered
quality improvement. Having this information at the beginning of your work will save you
a tremendous amount of time later and will be important if you plan to present or publish
your findings!
In summary, there are many ways to turn daily work into research. It is a matter of taking
time to really look at the work you are doing, deciding what is measurable, and
considering if those measurements can or will make a difference to how we work in the
future. Technology is changing how we do our jobs almost overnight, so it is important
that we continue to make ourselves and our work relevant. That cannot be done without
producing evidence, so we must find ways to work together to meet our goals and foster
our professional interests.
Choosing Research Methods that Fit You and Your Question
There is not a “right” choice of research method for any question. As researchers at any
level of experience, we work on a continuum of what may help us address our question
and be feasible to execute. There are several characteristics of a proposed study to
consider in deciding among methods—your question, the participants, timing, and
capacity.
The first consideration is your research question—what do you want to know or learn?
Research questions are also not set in stone. You may start with a very broad research
question and realize that you need to focus it in order to begin working on it. Participants
are an extremely important component as you have varying levels of access to the insights
they can provide. Consider whether you can learn what you want to know from direct
engagement with people (clients, students, library staff, etc.), artifacts representing people
(assignments, electronic health records), or documents (cataloging records, reports,
published articles, etc.). You as the researcher are also a participant, bringing your
experience and current point of view to the study design, your interpretation of the
results, and the conclusions you make. In qualitative research, you must be explicit about
this in terms of sharing your contextual bias and subjectivity—for more see Preissle [23].
Research Section Spotlight
Hypothesis, vol. 30, no. 1, Fall/Winter 2018 42
Timing and capacity often intertwine, and they tie back into your choice of participants.
For example, you may be interested in interviewing a lot of people, but you do not have
sufficient capacity to perform a lot of interviews, so you choose to do a survey or analyze
existing records. Research always takes much longer than we propose, so usually we come
up with an initial time estimate and then double or triple it. You must include time and
resources to pilot test ANY methodology. Piloting surveys is common best practice, but
you also need to pilot interview protocols, observation checklists, data extraction forms,
and any other tool that would be used to gather or analyze data. If you do not feel you
have time to pilot the tool, then you probably do not have sufficient time for the research.
Pilot testing does not have to be with the target participants, but should be with
participants who are not involved in the research and are relatively similar to your
intended population. Many research projects are cross-sectional, meaning they measure a
slice of a population at a single point in time, typically a one-time survey administration or
observation. For more robust evidence of change over time, particularly in educational
research, it would be ideal to perform longitudinal research, repeating measures on the
same population over multiple points in time. However, tracking participants over time
can be challenging.
Methodologies have inherent characteristics and limitations, but they are also more or less
effective in the hands of experienced practitioners. Look at studies that have addressed
questions similar to yours in the library/information science literature and other social
science fields such as education, psychology, management, etc. If the literature on a topic
is from case studies or single-institution surveys, consider expanding the view with other
study types or by including additional institutions for cross-institutional comparisons. If
you are interested in applying a method and need assistance, methods mentorship can be
internal or external to your institution or field. You do not have to seek the most
renowned experts; it may be easier learn from those with slightly more skill or experience
who can describe their approaches and refer as needed to those with greater expertise.
The next section discusses a few common methodologies you may wish to consider.
Common Methodologies in Research
a. Document review and analysis
Analyzing published documents is a very comfortable research strategy for librarians and
can be a study on its own or part of a larger mixed methods study. Examples include a)
evaluating information quality and readability; b) bibliometrics of publications or
networks; and c) systematic or scoping reviews. You may evaluate materials produced by
your own institution or those produced by others. If using your own materials, be aware
of your potential bias and consider a collaborator or second evaluator not involved in the
production of the materials being evaluated. Standard and well-documented methodology
Research Section Spotlight
Hypothesis, vol. 30, no. 1, Fall/Winter 2018 43
for the evaluation or data extraction such as a codebook is very important for
reproducibility! This is an area where the analysis can easily be either one-time or
longitudinal, looking at annual compilations/comparisons.
b. Observational methods
Observational studies have different units of analysis, such as individuals, groups of
people, or things. Observations in libraries may be easy to facilitate since libraries are
public places, whereas observations in other areas may require permission to access that
environment. The potential impact of being observed can be very high in an environment
where the observer stands out to those being observed. It may take several passive
observation periods to reduce that effect. Having an observational protocol and recording
form that has been pilot tested in a similar environment is helpful to make sure observers
are consistent. In addition to what is being observed that has been thought of, it is often
help to note other things you notice which may turn out to be confounders, things that
could explain the situation, but are not what you planned to study. There is value in
observing cohorts or comparison groups, whether contemporary or historical, so that you
have a sense of whether the observed phenomenon is consistent across groups or may
have been an outlier specific to one group. This question of generalizability of findings is
also an issue when gathering survey or interview data.
c. Gathering data from people: surveys and interviews
The primary limitation to surveys and interviews is that only certain people participate
and therefore findings may not represent the broader population from which you
recruited your participants. This is one reason you report on the baseline population you
attempted to survey when you calculate the response rate. For example, a survey of
medical students may have a response rate of 50% (which would be considered very
good). However, if all the participants are female, the survey is not representative of your
population if half of your students are male, and no females participated. One way to
understand a phenomenon more deeply and gain multiple points of view is to gather data
on the same issue directed to multiple types of respondents and then look at correlations
between respondents, e.g. asking instructors, students, and library staff about perceptions
of same issue. Surveys and interviews introduce recall challenges for participants in terms
of whether they remember accurately what they did to report on it for the survey or
whether those that had a negative experience are more likely to remember it and respond
accordingly. It is important to understand whether respondents truly understand what
you are asking. Ways to address this are to use previously studied and validated measures,
work on developing your question-writing skills, and use piloting combined with cognitive
interviewing when you are writing new questions that may have never been asked in your
study population.
Research Section Spotlight
Hypothesis, vol. 30, no. 1, Fall/Winter 2018 44
d. Combining multiple methods
Case study research often combines multiple methods to add validity. A recent case study
by North Carolina State University doctoral candidate J.J. Evans included a pilot study
followed by four types of data gathering and analysis: 1) Demographic Survey; 2) Semi-
structured Interviews; 3) Research Activity Log; and 4) Documents & Investigator’s
Research Journal. Case studies often include other types of document analysis to provide
historical and geographic context for the case or can include and draw conclusions from
multiple cases. For more on case studies, see a forthcoming editorial in the Journal of the
Medical Library Association.
e. Interventional (“experimental”) methods
We have opportunities for interventional studies in library practice. Although educational
studies come most often to mind, these do not have to be measuring real-time interactions
with people. For example, studying the impact of changes in policy that affect circulation,
collections, etc. can be done using analysis of existing records before and after the period
in which the change was made. Pre- and post-test evaluations in a population that have
received an intervention are common; they require attention to the effect of testing itself
as it is possible that taking the first test is what primed the person to do better on the
second test and the intervention did not produce the effect. Another option is historical
control groups, if they are sufficiently like the current group. There may be an
opportunity to do a group or individual randomized controlled trial in a course with the
same instructor or different instructors. For fairness, consider offering a crossover design
so that the control group eventually gets the intervention if the intervention is shown to
be valuable or even neutral. As you involve multiple investigators in delivering the
interventions, following the protocol to be sure you are measuring the impact of the same
intervention becomes very important.
f. Combining data from multiple institutions
One of the questions we received prior to the presentation was about our experience with
multiple institution studies to strengthen confidence in our findings. Designing a study to
include multiple institutions has several positives. It may be that you would not have had
the capacity to do your study alone if the partner institutions had not provided expertise
or funding. Having larger potential populations can may help with finding significant
effects by reducing Type II (Beta) errors. This would mean you do not find a statistically
significant effect when there is an effect, because you do not have enough power to detect
a difference. This is usually due to a low number of participants or the effect is smaller
than you anticipated. Or it may allow enough participants to identify subpopulations
affected differently by what you are studying. It may also improve generalizability since
effect is measured at multiple institutions if similar results are seen across the institutions.
Finally, it allows you to comment on the reproducibility of methodological tools and of
findings.
Research Section Spotlight
Hypothesis, vol. 30, no. 1, Fall/Winter 2018 45
Challenges
Now for the challenges—the three most salient are timing, instrument development, and
ethics approvals. Collaborators may find it difficult for all members to be available at the
same time, even in terms of scheduling meetings in different time zones. Timing also
comes into play with accessing participants, particularly if the institutions are on different
calendars or structures. For example, if you want to reach nursing students in their third
week of classes, that could be weeks apart at different institutions. It can also be difficult
to get multiple investigators to agree on the same protocol or survey questions. Although
there is the possibility of adding unique questions at some institutions, lengthening an
instrument can affect response rates, jeopardizing the larger survey participation or
introducing other variability. This can be addressed through joint instrument
development and selecting very similar institutions for the collaboration where question
wording would not need to be greatly changed. IRB/ethics board differences across
institutions may also cause delays, as many IRBs will not accept the judgment of another
IRB (although that is slowly changing). In our experience getting a multi-site study
approved by multiple IRBs, some had a separate minimal application form for studies
anticipated to be exempt, while others required a full application but then the study went
through expedited review. This meant that many of the questions asked by the full IRB
application form had never been considered by the research collaborators at the other
institutions.
Using technology and data collection methods efficiently
1. Audience and timing impact choice of methods
One you have considered your question, participants, timing, and personal capacity
related to potential methods, it is time to focus on the audience and their timing and your
institutional capacity. Who is the audience for your anticipated findings? What kind of
evidence will convince them? If your question is to respond to the needs of
administrators, it may need to address return on investment in addition to the effect itself.
It may not be enough to say something has an effect if you do not also include the costs of
achieving that impact. Questions that you are trying to answer to impact researchers or
clinicians who are used to evidence-based practice and high-quality research literature
may need to have control or comparison groups for them to trust in your findings. If you
pursue research to build a portfolio for your library promotion and tenure committee, be
sure you understand how they will value individual versus collaborative efforts, as that
may shape whether you execute a study on your own and acknowledge others versus
offering the opportunity for co-authorship with greater involvement.
In addition to the audience, consider whether the research is time-sensitive or time-
specific. For example, is there a limited window of time in which the study is possible (e.g.,
pre- and post-construction) or are the findings to be the basis of a time-sensitive decision?
Research Section Spotlight
Hypothesis, vol. 30, no. 1, Fall/Winter 2018 46
Do you have deadlines to produce publications for promotion? These can all lead you to
choose methods that can be planned and executed quickly although almost any
methodology can be expedited if you have enough help or funds and the ethics
committee/IRB finds the study to be exempt. Some examples of exempt research are
anonymous surveys, passive observation of public behavior without collection of
identifiers, and retrospective chart review. Even if you have help or funding, it matters
whether it is consistently available throughout, or in chunks, or at the time of day you
need. An example is whether to use student labor, your own labor, or funds to hire
external people to transcribe interviews. For observational studies, it might be whether
you can do the observations at the time needed such as evenings or weekends or offsite
daytime, or whether you need to hire or train others to do the observations, or you decide
to rely on self-report data since no one is available to do the observations.
2. Anonymous vs. confidential
One of the most common questions that you must clarify for the IRB submission is
whether the participants will be anonymous or confidential, and this often determines
whether a study will or will not be exempt from further review. Your study design, the risk
to participants, and the safeguards you must place on your data and whether you can
share it often depend on getting this distinction correct in your consent documentation. If
data collection is truly anonymous, you do not know who participated and hopefully no
one could figure it out from your data. If it is confidential, you know the participants
(people/institutions) and you take steps to conceal their identities. There are benefits and
drawbacks to both approaches.
Choosing to collect anonymous data is generally an easier write-up for the IRB application,
since there are fewer risks of identity exposure and fewer data protections to describe.
However, if your data is anonymous, you cannot follow up with respondents about their
responses, do longitudinal studies, or join the data with other data sources that contain
identifiers if you decide to expand the scope of your research project. Even anonymous
surveys require data protection considerations. For example, to be anonymous you must
not track or download the IP addresses of participants which are often automatically
gathered by online survey systems such as Survey Monkey or Qualtrics. If you recruited
by sending surveys to individual emails through the software system mailing functions,
you may also need to have the system unlink or mask the responding email addresses.
Survey tool functions are discussed later, with some shown in Table 1. If questions are
specific and the population studied is small enough, people may be identifiable from their
responses, and this may essentially turn your study into a confidential one where you
need to redact identifying information from open-ended questions or only share
aggregated responses for questions that have over five respondents in a certain category.
Confidential data collection involves assigning participants a unique identifier which is
maintained securely and separately from research data. Because people are identifiable
Research Section Spotlight
Hypothesis, vol. 30, no. 1, Fall/Winter 2018 47
in-person, by voice, and image, methodologies such as individual and focus group
interviews and audio or video analysis can usually only be treated as confidential data.
3. Analysis planning as part of the research design
Consider the relationship between your questions, your analysis skills, and your research
team members. If you have a question with multiple factors and competing explanations,
and you intend to gather and analyze quantitative data you will need some statistical
expertise. It is much better to discuss your questions and data collection on the front end
of the project with a statistician for statistical consulting so that you can come to a good
understanding of the number of participants you will need to reach a meaningful
conclusion and what type of data you might collect that allows you to test your question
most robustly. If your research team includes students or library student workers or
volunteers, consider what they can learn about data collection and analysis, and what
expertise or capacity they already have. For example, while it may be more efficient to pay
a transcription service to create transcripts of video interviews, if you have time and that
is something you want the students to gain experience with you may decide to have them
do some of the transcriptions.
In planning your research timeline, design, and budget, it is important to consider the
tools you have available and your familiarity with using them and what types of output
they produce. A common choice is deciding on what online survey tool to use (see Table
1). If you have designed and pilot tested your survey on paper, thinking about what
question types will make it easy for participants to answer questions. It can be frustrating
when the type of question you made is not available in one of the free tools. Be sure to
pilot test the online version of your survey to make sure it works properly, and that the
resulting data is captured successfully. It is worth finding out whether you have
institutional access to a tool such as Qualtrics or REDCap, as that will also make it possible
to share surveys across institutions to facilitate collaboration. You also want to test
question types by answering them in your survey tool to make sure that the output will be
easy to analyze. For example, if you use a multiple checkbox function in Google Forms, the
data will be in the same column in the spreadsheet separated by a delimiter and you will
have to then use a function to separate those responses and group them. While the native
Google display will show the aggregated data, you will not be able to pull out the data that
way. We compare the features of the most common online survey tools (Qualtrics,
REDCap, Google Forms, Survey Monkey) available to us as university-based health
sciences librarians in Table 1.
Survey Tool Free Account Subscription Account Notes
Research Section Spotlight
Hypothesis, vol. 30, no. 1, Fall/Winter 2018 48
Google
Forms
(drive.google
.com)
Includes <15 MB of
storage
Unlimited surveys,
responses, and
questions
>15 MB of storage
Month-to-month fee
Minimal data analysis available
regardless of the account type
Offers no support. Minimal
types of questions available
For questions with multiple
response options, view
individual, not group,
responses to get the full
picture.
Qualtrics
(www.qualtr
ics.com)
100 responses
10 outgoing emails
8 question types
1 active survey
Unlimited questions
Summary reports &
filtering
Survey logic &
randomization
Online reporting
No expiration
Limits to features are based
upon account type and
subscription plan
Expiration occurs when
subscription to product ends
Initially designed for business
and market research surveys
Works for other types of
research as well
REDCap
(projectredc
ap.org)
One week trial
15 templates
All collected or
entered data is
removed after the
trial version
concludes
Unlimited templates
Technical support for
implementing and
maintaining the product
Initially designed to support
researchers and to encourage
collaboration
Relatively easy to create a
survey without using tutorials
Video tutorials and step-by-
step instructions available in all
versions
Survey
Monkey
(surveymonk
ey.com)
Create surveys with
up to 10 questions or
elements (i.e.
question types,
descriptive text, or
images)
Collect up to 100
responses per
survey
Unlimited questions and
question types, descriptive
text, images, etc.
No limits to collecting
numbers of responses
Learning curve is small.
Not easy to share unpublished
surveys with non-subscribers
for testing or editing purposes.
Support features are minimal.
The knowledge base, while
useful, is not easy to search
Table 1. Comparison of Select Survey Tools based on publicly available information as of May
2018. Compiled by Margaret Hoogland.
Timing questions are more intensive for studies with a qualitative component, particularly
analyzing any text or images or videos for your studies. Will you ingest text into
computer-assisted qualitative data analysis software tools (e.g. NVivo, Atlas.ti, Dedoose)
for analyzing transcripts or other data resources, or will you annotate in Word or by hand
Research Section Spotlight
Hypothesis, vol. 30, no. 1, Fall/Winter 2018 49
on print copies? If you have multiple coders involved, where will you maintain the code
listing? Multi-institutional studies have more complexity in terms of sharing data storage,
access controls, agreeing on tools and preferences for types of analysis. Regardless of the
tools used, keep track of which members of the team performed which aspects of the
research, as many journal publishers require a statement about how each author
contributed to the research.
Learn about options for disseminating your research and how the journal article
publication process works in the Spring/Summer 2019 issue of Hypothesis.
Acknowledgements: This content was originally presented as a Special Content Session
"Research Roadmap: Understanding the Research Process" at the Medical Library
Association (MLA) Annual Meeting on May 20, 2018 co-sponsored by the Research
Section, Leadership and Management Section, Hospital Library Section, and Educational
Media and Technologies Section. The session was made possible by the coordinating
efforts of Brooke L. Billman of the MLA Research Section. We also thank the two
anonymous peer reviewers for their thoughtful feedback.
Disclosure: Heather Holmes is the chair of the MLA’s Research and Evidence-Based
Practice Curriculum Committee.
Works cited
1. Godshalk VM, Sosik JJ. Mentoring and leadership. In: The handbook of mentoring at work:
theory, research, and practice. Ragins BR, Kram KE, Eds. Los Angeles, CA: Sage, 2007: 150.
2. Johnson WB. On being a mentor. New York, NY: Lawrence Erlbaum Associates, 2007: 21-
22.
3. The research mentor: creativity in research: an introduction. Hypothesis 2010; 22 (2):
11-12.
4. Eldredge J. The research mentor: creativity in research, part 2: classifying creativity.
Hypothesis 2011; 23 (1): 10-13.
5. Eldredge J. The research mentor: creativity in research, Part 3: age and creative
potential. Hypothesis 2012; 23 (2): 14-19.
6. Eldredge J. The research mentor: interview with Phillip Eaton, M.D., on creativity in
research. Hypothesis 2012; 24 (1): 15-16.
7. Eldredge J. The research mentor: authorship part one: defining the article author.
Hypothesis 2009; 21 (3): 11-14.
8. Eldredge J. The research mentor: authorship part two: order of authors. Hypothesis
2010; 22 (1): 8-11.
9. Eldredge J. The research mentor: finding time for research. Hypothesis 2008; 20 (2): 5-7.
Research Section Spotlight
Hypothesis, vol. 30, no. 1, Fall/Winter 2018 50
10. Eldredge JD, Ascher MT, Holmes HN, Harris MR. The research mentor: top-ranked
research questions and systematic reviews. Hypothesis 2013; 24(2): 19-20.
11. Eldredge J. The research mentor: interview with library researcher Jo Dorsch.
Hypothesis 2008; 20 (3): 8-9.
12. Eldredge J. The research mentor: interview with library researcher Gary Byrd.
Hypothesis 2009; 21 (1): 11-13.
13. Eldredge JD. The research mentor: the long reach of a mentor’s influence. Hypothesis
2014; 26 (1): 14-9.
14. University of New Mexico. Mentoring Institute. Conferences. Available from:
. Accessed 20 July 2018.
15. Spencer AJ, Eldredge JD. Roles for librarians in systematic reviews: a scoping review. J
Med Libr Assoc 2018;106(1):46-56. doi: 10.5195/jmla.2018.82. Epub 2018 Jan 2.
16. Eldredge JD, Ascher MT, Holmes HN. An innovative model of evidence-based practice
for other professions. J Med Libr Assoc. 2015;103(2):100-2.
17. Eldredge JD, Ascher MT, Holmes HN. An innovative model of evidence-based practice
for other professions. J Med Libr Assoc. 2015;103(2):100-2.
18. Eldredge JD, Hall LJ, McElfresh KR, Warner TD, Stromberg TL, Trost J, Jelinek DA. Rural
providers' access to online resources: a randomized controlled trial. J Med Libr Assoc.
2016;104(1):33-41.
19. Eldredge JD, Kroth PJ, Murray-Krezan C, Hantak CM, Weagel EF, Hannigan GG. How
accurately does the VIVO Harvester reflect actual Clinical and Translational Sciences
Award-affiliated faculty member publications? J Med Libr Assoc. 2015;103(1):19-21.
20. Koufogiannakis D, Brettle A. A new framework for EBLIP. In: Being evidence based in
library and information practice (pp. 11-18). London: Facet Publishing, 2016.
21. Lindsay JM, Oelschlegel S, Earl M. Surveying hospital nurses to discover educational
needs and preferences. J Med Libr Assoc 2017;105(3):226-232. doi:10.5195/jmla.2017.85.
22. How to make time for research and writing. Chronicle of Higher Education. 2017 April
9. Available from: Accessed July 20, 2018.
23. Preissle J. Subjectivity Statement. In: The SAGE Encyclopedia of Qualitative Research
Methods. Ed. Lisa M. Given. Thousand Oaks, CA: SAGE, 2008. 846.
Appendix
Questions from Participants
These questions came electronically from MLA members prior to and during the session.
We addressed questions received verbally during the session within the body of the
article.
Prior to the Session Via MLA-FOCUS (n = 1) and email to Research Section and
Hospital Libraries Section email lists (n = 2)
https://www.chronicle.com/article/How-to-Make-Time-for-Research/239724
https://www.chronicle.com/article/How-to-Make-Time-for-Research/239724
Research Section Spotlight
Hypothesis, vol. 30, no. 1, Fall/Winter 2018 51
1. How can you move beyond research studies based primarily on your own work (which
tends to be case studies), into doing research that will have bigger impact on the field?
Or put another way, how do you come up with research studies that will be impactful,
but are still manageable to do within the confines of your job and institution?
One way to move beyond your case study is to aggregate multiple case studies from
similar institutions to strengthen the patterns of findings (see Robert K. Yin’s Case study
research and applications, 6th ed., 2018). Another is to use your case study as pilot data for
a different methodology that involves a control population or other attempts to capture or
address confounding variables.
2. How to find collaborators, especially for hospital librarians that are solo or may have
an idea, and don’t have time to pursue by themselves?
3. My question about research is how the heck can I do it as a solo librarian.
Addressing questions two and three together, one approach to advancing research as a
solo librarian is to mentor a library science student doing research as part of a field
experience in your library or to mentor student volunteers from other social science
disciplines in your library. Many students have research skills or requirements but
struggle to come up with a meaningful question or real data, both of which you can
provide. Author Alpi co-authors many of her research studies with student employees
from diverse majors or library students doing field experiences. You can reach out to your
local library school, state library association, or post your inquiry to the MLA Medical
Library Education Section (http://www.mlanet.org/page/section-landing-medical-
library-education-section) to reach instructors of health sciences library students in
online programs who may be interested. In a small library, choosing projects that can
survive a longer timeframe or partnering with other areas of the institution with more
capacity or the ability to advocate for answering the question, e.g. nursing research,
continuing education, or quality improvement.
During the Session Verbally or via the Online Form
1. How can you find a research mentor at an institution with promotion and tenure
requirements similar to your own?
The MLA Mentoring website allows you to look at the institution of the mentor, but
beyond examining the title of the librarian for indicators of rank, does not offer any other
information about those requirements. The website Academic Librarian Status
(https://academiclibrarianstatus.wordpress.com/) offers a list of academic institutions in
the United States and elsewhere, sorted by the professional status of their librarians.
Although the professional school libraries may not always have the same requirements as
the university libraries, this is a good starting point. Mentors may have worked at
multiple institutions with diverse requirements and therefore it is important to consider
past experience beyond the mentor’s current institutional affiliation.
http://www.mlanet.org/page/section-landing-medical-library-education-section
http://www.mlanet.org/page/section-landing-medical-library-education-section
https://academiclibrarianstatus.wordpress.com/
Research Section Spotlight
Hypothesis, vol. 30, no. 1, Fall/Winter 2018 52
2. I’m gung ho about my research project but my more experienced research partners are
not as excited. Any advice on finding research partners who are excited to work with
new researchers on their early projects?
The first question that comes to mind is have you asked why they are not excited—it is the
topic, the method, the role, the collaborators, or their own capacity? The piece of this
article about what you offer mentors may provide some insight. If your institution is
pressuring them to produce more advanced evidence, you may be better off finding peer
collaborators or mentors outside your institution who do not feel the same pressure. You
must be open to the idea that your research project is not as compelling to others as it is to
you—check the literature to see how your proposed idea may be received if you decide to
go forward.
3. What are your opinions of keeping daily stats of everyday work and do you know of
ways to turn those daily stats into a story that highlight librarian/library impact?
At NC State University we track public service daily work statistics and space usage using
an open-source program called Suma (https://www.lib.ncsu.edu/projects/suma) to
gather data. It has built-in analytic tools. We track activities by user type and when we see
changes or growth with a certain user population this invites us to dig deeper. For
example, if we are doing more transactions with house officers, we then might interview a
few to understand how we are impacting them. Since intern/resident support is an
important issue, telling a story of how we support them would resonate. We also tell
Library Stories based on experiences, see https://www.lib.ncsu.edu/stories/supporting-
ncsu-researchers-achieving-nih-public-access.
4. Clinicians and academic faculty barely have time or interest for my short 7 question
surveys, but the questions I have are best answered by surveys. How can I better
engage my test audience? Is there an alternative to surveys?
One of the best ways to engage an audience is to have co-authors who represent that
audience. Can you co-author the survey with key opinion leaders or have them distribute
the survey for you? If you have a library committee with members from your test
audience, they might be able to assist. Depending on the topic of your surveys, it may be
easier to do a mini-focus group with representatives of the test audience at a meeting.
Feel free to contact author Alpi with more details to receive more focused feedback.
5. When your paper is submitted, and you are about to revise and resubmit, can the
author ask for new reviewers?
Probably not. When you are asked to revise and resubmit your manuscript, the editor of
the journal most likely wants the original reviewers to re-evaluate the revised manuscript
to ensure that your revisions have sufficiently alleviated their specific concerns. Moreover,
inviting new reviewers at this point would likely only prolong the peer review process and
create more “hoops” for you to jump through. However, if you have serious concerns
about a reviewer—perhaps you feel that they did not carefully read your manuscript or
https://www.lib.ncsu.edu/projects/suma
Research Section Spotlight
Hypothesis, vol. 30, no. 1, Fall/Winter 2018 53
were unreasonably critical—you can privately express those concerns in an email or
formal letter to the editor, who will take your concerns into consideration when weighing
the reviewers’ opinions to arrive at a final decision regarding acceptance.
National Library of Medicine (US). Digital Collections [Internet].
Bethesda (MD): Medline keyboard in the reading room [196?]. Available from:
http://resource.nlm.nih.gov/101445854
http://resource.nlm.nih.gov/101445854