





































Research Section Spotlight 

   

Hypothesis, vol. 30, no. 1, Fall/Winter 2018                                                                          33                                  

Research Roadmap: Understanding the Research 
Process (Part 1) 
 

Kristine M Alpi, PhD, MPH, MLS, AHIP 
University Librarian 

Oregon Health & Science University Library 
Oregon Health & Science University 
Portland, OR 
krisalpi@gmail.com 
 

Jonathan D Eldredge, PhD, MLS, AHIP 
Evidence Based and Translational Science Collaboration Coordinator 

Health Sciences Library and Informatics Center 
University of New Mexico 
New Mexico, NM 
jeldredge@salud.unm.edu 
 

Heather N Holmes, MLS, AHIP  
Associate Director of Libraries 

MUSC Libraries 

Medical University of South Carolina (MUSC) 

Charleston, SC 
holmesh@musc.edu   
 

Katherine G Akers, PhD 
Biomedical Research/Research Data Specialist 

Shiffman Medical Library 

Wayne State University 

Detroit, MI 

katherine.akers@wayne.edu 
 

Kimberly R Powell, MLIS 
Research Impact Informationist 

Woodruff Health Sciences Center Library 

Emory University 

Atlanta, GA 

krpowel@emory.edu 
 

Margaret Hoogland, MLS, AHIP 
Assistant Professor, Clinical Medical Librarian 

Mulford Health Sciences Library 

University of Toledo 

Toledo, OH 

mahoogla@gmail.com 



Research Section Spotlight 

   

Hypothesis, vol. 30, no. 1, Fall/Winter 2018                                                                          34                                  

*Editor’s note: This piece has been split into two parts due to length. Part 1 covers advice and 

guidance about doing research while Part 2 (which will be in the Hypothesis Spring/Summer 

2019 issue) discusses how to share the products of your research. 

Introduction  

Since 1982, the MLA Research Section has fostered discussion, shared advice, connected 

members to potential collaborators and research mentors, and kept its membership up-to-

date on library research efforts, particularly as they apply to a wide variety of academic 

life sciences and community health efforts.  Towards these endeavors, the 2017-2018 

Research Section program planners proposed a Special Content session on understanding 

the research process for the 2018 MLA Annual Meeting & Exhibition in Atlanta, GA.  The 

session was envisioned as a panel discussion for an audience of new to intermediate 

librarian researchers that would offer advice around getting started, from formulating an 

idea, to study design, data collection, and finally dissemination and the publication 

process. Other MLA Sections--Leadership and Management Section, Hospital Library 

Section, and Educational Media and Technologies Section--offered their support.  Panelists 

were recruited based on their previous involvement in continuing education and editorial 

work regarding library research, and several belonged to multiple Sections sponsoring the 

session.  In discussion with the Research Section Program Chair [Billman] and the session 

moderator [Powell], each panelist [Akers, Alpi, Eldredge, Holmes] selected to focus on an 

area most relevant to their work, with the recognition that all the presenters had expertise 

across the following topics comprising the presentation: 

• Overview of the research process 

• Finding and working with a research mentor 

• Turning daily work into a research project 

• Choosing research methods that fit you and your question 

• Using technology and data collection methods efficiently 

• Disseminating your research 

• Understanding the publishing process 
 

 “Research Roadmap: Understanding the Research Process,” was presented on May 20, 

2018, and 2018 MLA Annual meeting attendees can access the recorded session by logging 

into the online program with their registration information 

(https://www.eventscribe.com/2018/MLA/).  This account of the session is presented so 

those unable to view the recordings may benefit from the panelists’ advice and expertise. 

 

 

Making a Member-Driven Presentation 

https://www.eventscribe.com/2018/MLA/


Research Section Spotlight 

   

Hypothesis, vol. 30, no. 1, Fall/Winter 2018                                                                          35                                  

In planning the session, we wanted to respond to MLA members’ questions about the 

research process.  On April 5, 2018, Billman posted a call for questions in the MLA-FOCUS, 

an email update sent to all MLA members. Questions received before the submission 

deadline were addressed in the content presented.  The session incorporated two question 

and answer periods, one after the first two speakers and the second at the end. At the 

beginning of the session, a slide encouraged the audience to “Submit Your Questions” 

using a tinyurl online survey or on cards. Those not addressed by the presenters during 

the session have responses from our panelists in the Appendix.  

The Roadmap as Presented   

Overview of the Research Process 

We began by reviewing the Evidence Based Practice Process and the opportunity for 

research after the existing literature has been critically appraised, and before its 

application to the problem that spurred the initial search for evidence.  Moving on to the 

research process, we shared a four-step version for which we would provide our 

experiences: 

1. Formulate a Research Question (Objective)  

2. Design an appropriate method to answer the question  

3. Interpret your results  

4. Communicate your results  

As we developed our expertise through being mentored and mentoring others, and 

because several of the questions we received were about finding mentors or collaborators, 

we addressed mentorship first.  

Finding and Working with a Research Mentor  

A mentor can be defined as “an individual with advanced knowledge, usually more senior 

in some regard, who is committed to providing upward career mobility and assistance for 

the protégé …” Moreover, “The mentoring relationship has been characterized as an 

intense, sometimes intimate professional relationship devoted to providing social support 

and development for the protégé’s career.”[1] Johnson observes that mentoring 

represents an enduring and reciprocal personal relationship in which the mentor serves 

as a role model while offering a safe environment for the protégé to consider new ideas. In  

this context, the mentor provides acceptance, protection, challenge and coaching for the 

protégé.[2] 

 

While many studies have investigated career or academic mentoring, few investigations 

have rigorously explored the specialized world of mentoring other professional colleagues 

in conducting research. And almost all investigations into research mentoring have 



Research Section Spotlight 

   

Hypothesis, vol. 30, no. 1, Fall/Winter 2018                                                                          36                                  

focused upon faculty-student mentoring. Eldredge has mentored more than 20 colleagues 

over the past two decades. A few of these colleagues have been new faculty members in 

his medical school whereas most have been library or information practitioners wanting 

guidance with research projects. These early experiences, coupled with limited time for 

mentoring individuals, led to sponsoring “The Research Mentor” column in Hypothesis 

during 2008-2014. These columns featured topics about which his protégés needed 

recurring guidance ranging from creativity in research [3, 4, 5, 6], to defining authorship 

[7, 8], to the pragmatic aspects of research [9, 10].  They also included interviews with 

esteemed researchers [11, 12] and a reflection upon a past mentor’s long-term effect upon 

his own career [13].  Additionally, the author participated in several of the University of 

New Mexico’s Mentoring Institute’s Research Conferences [14]. 

Roles of the protégé 

Generalizing from these experiences, the protégé needs to practice reciprocity in the 

mentoring relationship. The research mentor brings a wealth of knowledge and skills on 

how to conduct a research project. The gains to the protégé are obvious. Yet, what does 

the protégé bring to the mentoring relationship? Inventorying the attributes possessed by 

the protégé might strengthen the relationship: 

1. Previous research education 

The mentoring relationship should not be confused with a tutoring relationship in which 

the mentor provides comprehensive one-on-one instruction on all aspects of research. The 

protégé should bring previous learning on research methods to the relationship. The 

protégé can learn research methods by taking or auditing courses at nearby or online 

academic institutions, such as those in subjects such as psychology, biology, or one of the 

social sciences. The new MLA Research Training Institute might provide an avenue for 

some protégés to gain research methods education and identify mentors, but this new, not 

yet evaluated, program does not offer the depth of training found in a graduate-level 

research course. 

2. Literature searching 

Every research process involves literature searches in the formative stages due to the 

need to perform iterative literature searches until the final research question reflects the 

state of the existing research knowledge on the topic to be investigated. Designing the 

research methodology provides another stage in which one or more literature searches 

will inform how the research project will unfold. Finally, once the research is done,  

additional literature searching will assist in interpreting and discussing the results. 

Librarians who are capable literature searchers can assist a research project with their 

unique skill sets. 

3. Organization 



Research Section Spotlight 

   

Hypothesis, vol. 30, no. 1, Fall/Winter 2018                                                                          37                                  

Librarians are legendary for their organizational skills, and many have learned project 

management skills that have direct application to the complex dimensions of a research 

project. Librarians also can use bibliographic management tools such as Zotero or 

EndNote to organize the hundreds of references associated with any research project. A 

recently published scoping review involved elaborate bibliographies and analytics, 

highlighting the protégé’s contribution of an impressive skill set to the project [15]. 

4. Track record 

Many librarians have contributed to research projects as part of their everyday work. 

These roles often involve literature searching and bibliographic management. While most 

involvement with research projects takes a more episodic form, some projects involve 

larger commitments. These projects form a track record of librarian involvement in past 

research projects that librarians can point to when approaching a research mentor for 

guidance. 

5. Commitment 

The potential protégé needs to bring a commitment to completing the research project to 

the mentoring relationship. Experienced researchers know that many well-intended 

research projects are abandoned at various phases prior to completion. Ideally, the 

protégé can present the potential mentor with a plan and timeline that will advance the 

research project through completion and communication to the profession through poster, 

presentation, or publication of the results. The multi-phased MLA Research Agenda 

project [16] tested the commitment of all researchers involved yet when it reached 

completion, it provided a great service to our profession [17]. 

6. Creativity 

Established researchers sometimes descend so deeply into the weeds of the research 

design and its careful implementation that we forget to consider the opportunities to lend 

some creativity to the project. A noteworthy example surfaced when one of the authors 

included three medical students in a randomized controlled trial on Point-of-Use tools. 

The three medical students expressed their creativity by producing a top-notch training 

video for participants in the study. In other ways, the students also added elements of 

creativity unrecognized by the veteran researchers and thereby improved project 

participation among providers [18]. 

 

7. Enthusiasm 

A few years ago, one of the author asked an audience of translational researchers what 

adjectives they would use to describe the research process. The adjectives offered were 

“tedious” and “detailed” and “comprehensive.” There are admittedly boring moments in 

any research project in which one can be tempted to walk away. Toward the end of the 

project, however, excitement builds as results, particularly difficult to explain results, 



Research Section Spotlight 

   

Hypothesis, vol. 30, no. 1, Fall/Winter 2018                                                                          38                                  

emerge in need of interpretation. A protégé with enthusiasm most likely will rely on it to 

muddle through the less exciting aspects of a research project. One applied informatics 

research study that comes to mind involved a lot of tedium, but the researchers all plowed 

through the hours of careful checking in this fidelity study to produce surprising results 

[19]. 

Initiating the mentoring relationship  

We have focused on how the protégé seeking to recruit a research mentor needs to step 

outside the limits of her or his own perceptions to consider the perspective of the 

potential mentor(s).  The MLA Mentoring/Expertise Directory 

(http://www.mlanet.org/page/mentoring) is one way to find mentors who have already 

indicated willingness to engage in mentoring relationships.  In other cases, you may 

identify someone in the literature or at your institution with expertise you aspire to 

develop and there you will not know their willingness to mentor until you make the 

approach.  It is up to the mentorship dyad to set mutually agreeable terms of the 

relationship.  The respect is reciprocal, and it is also clear that the more experienced 

researcher can gain insight and satisfaction, and sometimes co-authorship from these 

relationships. In some cases, an enduring relationship develops that leads to new studies 

as collaborators.   

Turning Daily Work into a Research Project  

There are many ways to turn daily work into a research project; it is a matter of looking 

and asking thoughtful questions about what work is being done. More specifically, it 

means looking at what we are, or perhaps should be, measuring.  Valuable research can be 

performed at many levels; not every question is best addressed by a large, multi-center 

NIH-funded study. This is especially true in library research projects because much of 

what we measure and seek to improve is part of our daily work. One of the key pieces of 

evidence-based library and information practice (EBLIP) is applying local evidence to 

evidence in the literature, and to our own professional knowledge. Considering the five 

principles of EBLIP as presented by Koufogiannakis and Brettle [20] may help us better 

understand ways we can turn daily work into research:  

Articulate: A clear understanding of the problem or question must be reached. Part of 

articulating the problem includes working out what is known already and why the  

information is needed, as well as ensuring that the problem is set in the appropriate 

context. 

Assemble: Evidence should be assembled from multiple sources that are the most 

appropriate to the question or problem at hand and should include research evidence, 

local evidence, and professional knowledge. 

http://www.mlanet.org/page/mentoring


Research Section Spotlight 

   

Hypothesis, vol. 30, no. 1, Fall/Winter 2018                                                                          39                                  

Assess: Evidence should be assessed for its quality (often known as appraisal or critical 

appraisal), determining what the evidence says as a whole. 

Agree: Determine a course of action and begin implementation of the decision. If working 

with a group, try to achieve consensus based on the evidence and organizational goals. 

Adapt: Evaluate the decision and how it has worked in practice. Revisit goals and reflect 

on the success of implementation. 

Applied research at the Medical University of South Carolina (MUSC) 

Keeping these principles in mind and thinking through the work that we do daily at the 

library, a relatively simple research project grew out of updating LibGuides. We were 

looking at click stats on our guides and noticed some irregularities such as usage 

considerably different from what we expected. That started us in the first principle of 

EBLIP, asking (articulating) the question of why these irregularities were present. The 

research project developed from seeing the numbers and articulating the question. Before 

we knew it, we had developed the project. Ultimately, looking at the clicks in our 

LibGuides gave us an opportunity to assemble the evidence, assess it, and decide (agree) 

on what to do with what we learned. 

Using what we learned, we decided on ways to reformat our Guides and to reconsider 

placement of links based on perceived importance. If things we felt were important had 

been prominently placed, but were not getting many clicks, we moved them a little further 

down the page since fewer clicks seemed to indicate less importance to the user. 

Conversely, when we found things placed lower getting a lot of use, we relocated them to a 

more prominent position.  It is a very simple project, and one that we can easily adapt 

over time. We will revisit the statistics in the future to assess whether our repositioning 

made any difference. 

It is important to note that this simple project led to more questions being asked and 

therefore more research needing to be done.  A secondary outcome (that we had not 

considered until we really looked at the data) was the realization that we needed to adjust 

our instruction sessions to include more guidance on navigating LibGuides, as well as  

accessing the resources therein. A follow-up study looking at data from and the questions 

being asked through LibAnswers is also being launched. The results will also factor into 

future LibGuides updates and we expect it to bring out even more questions and things to 

study.  This is one small example of taking our everyday work and turning it into research. 

It is a project we know we will learn from, and we anticipate publishing our results to 

share our procedures with others who may consider a similar project.  For another 

example from a different institution, see Lindsay, Oelschlegel & Earl [21].  

 
 



Research Section Spotlight 

   

Hypothesis, vol. 30, no. 1, Fall/Winter 2018                                                                          40                                  

Identifying and overcoming obstacles 

How do we find time to do research when we often feel like we do not have enough time to 

do our day-to-day job? Further, how do we find time to do research when it is not 

expected of our position, or when our superiors are not supportive? This author [Holmes] 

feels that the best way to achieve time for research is to simply prioritize. That means 

figuring out what can and cannot be shuffled, eliminated, or delegated from your daily 

work. It also means setting aside dedicated time in your schedule to get the research done, 

even if it is on Saturday afternoon [Holmes] or in thirty-minute blocks before work while 

others are sleeping [Alpi]; see Chronicle of Higher Education [22] for more ideas. You must 

set boundaries so that your dedicated time is not always interrupted or sacrificed, and you 

also must enforce the boundaries.  

If you are passionate enough about a topic and want to research it, then you must be 

willing to pursue it on your own time. There are many librarians who are either not in 

positions where there are expectations of publishing and presenting or who lack support 

from their superiors or even their colleagues, yet they want to be involved in advancing 

our profession. It is important for all library staff with questions to find their own way to 

pursue their interests, even if it is on their own time. In fact, most librarians spend a 

portion of their own time (and often their own money) to pursue their professional goals 

and interests. The results that will come from taking the time to work on projects will help 

your resume/curriculum vitae (CV) for future opportunities. 

Another obstacle that we frequently encounter is lack of access to resources, whether it is 

software or people or something else. Your mentor or your professional association can 

help you find these resources, but your own professional networking efforts will probably 

yield the biggest benefit. If someone’s research interests you, it is entirely appropriate to 

contact the researcher to say you would like to know more about the work or perhaps 

even suggest a collaboration.  It helps to have a scholarly presence online (e.g. LinkedIn or 

Google Scholar Profiles) so they can learn more about you before responding to your 

inquiry. Collaboration is one of the keys to success, and it requires work on your end to 

make those meetings happen. Collaborators can provide resources like software or 

expertise, but they can also end up serving as a mentor or a network to yet another person 

with whom you may be able to work with in the future. 

While a lack of understanding of the research process is an obstacle for early researchers, 

it is very much a learning opportunity.  Looking to the literature is one of the best ways to 

get an introduction to the process, and again working with a mentor can help with 

guidance. Educational opportunities are increasing, and the MLA’s Research and 

Evidence Based Practice Curriculum Committee is in the process of building an 

education curriculum for learning about research.  A final suggestion for learning about 

the research process, and a general way to get more involved in research, is to engage with 



Research Section Spotlight 

   

Hypothesis, vol. 30, no. 1, Fall/Winter 2018                                                                          41                                  

your Institutional Review Board (IRB)/ethics committee. If you are going to study your 

users or analyze educational output from previous courses for a presentation or 

publication to the outside world, you will need to have your proposed project reviewed by 

the IRB.  Understanding the background of your IRB members or looking into becoming an 

IRB member yourself may be an excellent engagement opportunity to learn about 

research at your institution, and how you might find a partner. No matter your role, you 

will learn much more about the process, and when the time comes for your own research 

with human participants, you will know what you have to do to prepare your own 

submission for approval. Always be sure to check with your IRB prior to starting your 

research so you will know if what you propose will qualify as research or be considered 

quality improvement. Having this information at the beginning of your work will save you 

a tremendous amount of time later and will be important if you plan to present or publish 

your findings! 

In summary, there are many ways to turn daily work into research. It is a matter of taking 

time to really look at the work you are doing, deciding what is measurable, and 

considering if those measurements can or will make a difference to how we work in the 

future. Technology is changing how we do our jobs almost overnight, so it is important 

that we continue to make ourselves and our work relevant. That cannot be done without 

producing evidence, so we must find ways to work together to meet our goals and foster 

our professional interests. 

Choosing Research Methods that Fit You and Your Question   

There is not a “right” choice of research method for any question. As researchers at any 

level of experience, we work on a continuum of what may help us address our question 

and be feasible to execute.   There are several characteristics of a proposed study to 

consider in deciding among methods—your question, the participants, timing, and 

capacity.   

The first consideration is your research question—what do you want to know or learn?  

Research questions are also not set in stone.  You may start with a very broad research 

question and realize that you need to focus it in order to begin working on it. Participants 

are an extremely important component as you have varying levels of access to the insights 

they can provide.  Consider whether you can learn what you want to know from direct 

engagement with people (clients, students, library staff, etc.), artifacts representing people 

(assignments, electronic health records), or documents (cataloging records, reports, 

published articles, etc.).  You as the researcher are also a participant, bringing your 

experience and current point of view to the study design, your interpretation of the 

results, and the conclusions you make.  In qualitative research, you must be explicit about 

this in terms of sharing your contextual bias and subjectivity—for more see Preissle [23].  



Research Section Spotlight 

   

Hypothesis, vol. 30, no. 1, Fall/Winter 2018                                                                          42                                  

Timing and capacity often intertwine, and they tie back into your choice of participants.  

For example, you may be interested in interviewing a lot of people, but you do not have 

sufficient capacity to perform a lot of interviews, so you choose to do a survey or analyze 

existing records. Research always takes much longer than we propose, so usually we come 

up with an initial time estimate and then double or triple it.  You must include time and 

resources to pilot test ANY methodology.  Piloting surveys is common best practice, but 

you also need to pilot interview protocols, observation checklists, data extraction forms, 

and any other tool that would be used to gather or analyze data.   If you do not feel you 

have time to pilot the tool, then you probably do not have sufficient time for the research.  

Pilot testing does not have to be with the target participants, but should be with 

participants who are not involved in the research and are relatively similar to your 

intended population.  Many research projects are cross-sectional, meaning they measure a 

slice of a population at a single point in time, typically a one-time survey administration or 

observation.  For more robust evidence of change over time, particularly in educational 

research, it would be ideal to perform longitudinal research, repeating measures on the 

same population over multiple points in time.  However, tracking participants over time 

can be challenging.  

Methodologies have inherent characteristics and limitations, but they are also more or less 

effective in the hands of experienced practitioners.  Look at studies that have addressed 

questions similar to yours in the library/information science literature and other social 

science fields such as education, psychology, management, etc.  If the literature on a topic 

is from case studies or single-institution surveys, consider expanding the view with other 

study types or by including additional institutions for cross-institutional comparisons.  If 

you are interested in applying a method and need assistance, methods mentorship can be 

internal or external to your institution or field.  You do not have to seek the most 

renowned experts; it may be easier learn from those with slightly more skill or experience  

 

who can describe their approaches and refer as needed to those with greater expertise.  

The next section discusses a few common methodologies you may wish to consider.  
 

Common Methodologies in Research 
 

a. Document review and analysis  

Analyzing published documents is a very comfortable research strategy for librarians and 

can be a study on its own or part of a larger mixed methods study.  Examples include a) 

evaluating information quality and readability; b) bibliometrics of publications or 

networks; and c) systematic or scoping reviews.  You may evaluate materials produced by 

your own institution or those produced by others.  If using your own materials, be aware 

of your potential bias and consider a collaborator or second evaluator not involved in the 

production of the materials being evaluated.  Standard and well-documented methodology 



Research Section Spotlight 

   

Hypothesis, vol. 30, no. 1, Fall/Winter 2018                                                                          43                                  

for the evaluation or data extraction such as a codebook is very important for 

reproducibility!  This is an area where the analysis can easily be either one-time or 

longitudinal, looking at annual compilations/comparisons.  
 

b. Observational methods   

Observational studies have different units of analysis, such as individuals, groups of 

people, or things. Observations in libraries may be easy to facilitate since libraries are 

public places, whereas observations in other areas may require permission to access that 

environment.  The potential impact of being observed can be very high in an environment 

where the observer stands out to those being observed. It may take several passive 

observation periods to reduce that effect.  Having an observational protocol and recording 

form that has been pilot tested in a similar environment is helpful to make sure observers 

are consistent. In addition to what is being observed that has been thought of, it is often 

help to note other things you notice which may turn out to be confounders, things that 

could explain the situation, but are not what you planned to study.  There is value in 

observing cohorts or comparison groups, whether contemporary or historical, so that you  

have a sense of whether the observed phenomenon is consistent across groups or may 

have been an outlier specific to one group.  This question of generalizability of findings is 

also an issue when gathering survey or interview data.  
   

c. Gathering data from people: surveys and interviews  

The primary limitation to surveys and interviews is that only certain people participate 

and therefore findings may not represent the broader population from which you 

recruited your participants.  This is one reason you report on the baseline population you 

attempted to survey when you calculate the response rate.  For example, a survey of 

medical students may have a response rate of 50% (which would be considered very 

good).  However, if all the participants are female, the survey is not representative of your  

population if half of your students are male, and no females participated.  One way to 

understand a phenomenon more deeply and gain multiple points of view is to gather data 

on the same issue directed to multiple types of respondents and then look at correlations 

between respondents, e.g. asking instructors, students, and library staff about perceptions 

of same issue.  Surveys and interviews introduce recall challenges for participants in terms 

of whether they remember accurately what they did to report on it for the survey or 

whether those that had a negative experience are more likely to remember it and respond 

accordingly.  It is important to understand whether respondents truly understand what 

you are asking. Ways to address this are to use previously studied and validated measures, 

work on developing your question-writing skills, and use piloting combined with cognitive 

interviewing when you are writing new questions that may have never been asked in your 

study population.  



Research Section Spotlight 

   

Hypothesis, vol. 30, no. 1, Fall/Winter 2018                                                                          44                                  

d. Combining multiple methods  

Case study research often combines multiple methods to add validity. A recent case study 

by North Carolina State University doctoral candidate J.J. Evans included a pilot study 

followed by four types of data gathering and analysis: 1) Demographic Survey; 2) Semi-

structured Interviews; 3) Research Activity Log; and 4) Documents & Investigator’s 

Research Journal. Case studies often include other types of document analysis to provide 

historical and geographic context for the case or can include and draw conclusions from 

multiple cases.  For more on case studies, see a forthcoming editorial in the Journal of the 

Medical Library Association. 

e. Interventional (“experimental”) methods   

We have opportunities for interventional studies in library practice.  Although educational 

studies come most often to mind, these do not have to be measuring real-time interactions 

with people.  For example, studying the impact of changes in policy that affect circulation, 

collections, etc. can be done using analysis of existing records before and after the period 

in which the change was made.   Pre- and post-test evaluations in a population that have 

received an intervention are common; they require attention to the effect of testing itself 

as it is possible that taking the first test is what primed the person to do better on the 

second test and the intervention did not produce the effect.  Another option is historical 

control groups, if they are sufficiently like the current group.  There may be an 

opportunity to do a group or individual randomized controlled trial in a course with the 

same instructor or different instructors.  For fairness, consider offering a crossover design 

so that the control group eventually gets the intervention if the intervention is shown to 

be valuable or even neutral. As you involve multiple investigators in delivering the 

interventions, following the protocol to be sure you are measuring the impact of the same 

intervention becomes very important.  

f. Combining data from multiple institutions  

One of the questions we received prior to the presentation was about our experience with 

multiple institution studies to strengthen confidence in our findings. Designing a study to 

include multiple institutions has several positives.  It may be that you would not have had 

the capacity to do your study alone if the partner institutions had not provided expertise 

or funding. Having larger potential populations can may help with finding significant 

effects by reducing Type II (Beta) errors. This would mean you do not find a statistically 

significant effect when there is an effect, because you do not have enough power to detect 

a difference. This is usually due to a low number of participants or the effect is smaller 

than you anticipated. Or it may allow enough participants to identify subpopulations 

affected differently by what you are studying.  It may also improve generalizability since 

effect is measured at multiple institutions if similar results are seen across the institutions.  

Finally, it allows you to comment on the reproducibility of methodological tools and of 

findings. 



Research Section Spotlight 

   

Hypothesis, vol. 30, no. 1, Fall/Winter 2018                                                                          45                                  

Challenges 

Now for the challenges—the three most salient are timing, instrument development, and 

ethics approvals. Collaborators may find it difficult for all members to be available at the 

same time, even in terms of scheduling meetings in different time zones.  Timing also 

comes into play with accessing participants, particularly if the institutions are on different 

calendars or structures.  For example, if you want to reach nursing students in their third 

week of classes, that could be weeks apart at different institutions.  It can also be difficult 

to get multiple investigators to agree on the same protocol or survey questions.  Although 

there is the possibility of adding unique questions at some institutions, lengthening an 

instrument can affect response rates, jeopardizing the larger survey participation or 

introducing other variability.  This can be addressed through joint instrument 

development and selecting very similar institutions for the collaboration where question 

wording would not need to be greatly changed.  IRB/ethics board differences across 

institutions may also cause delays, as many IRBs will not accept the judgment of another 

IRB (although that is slowly changing).  In our experience getting a multi-site study 

approved by multiple IRBs, some had a separate minimal application form for studies 

anticipated to be exempt, while others required a full application but then the study went 

through expedited review.  This meant that many of the questions asked by the full IRB 

application form had never been considered by the research collaborators at the other 

institutions.  

Using technology and data collection methods efficiently   

1. Audience and timing impact choice of methods  

One you have considered your question, participants, timing, and personal capacity 

related to potential methods, it is time to focus on the audience and their timing and your 

institutional capacity.  Who is the audience for your anticipated findings? What kind of 

evidence will convince them?  If your question is to respond to the needs of 

administrators, it may need to address return on investment in addition to the effect itself.  

It may not be enough to say something has an effect if you do not also include the costs of 

achieving that impact.  Questions that you are trying to answer to impact researchers or 

clinicians who are used to evidence-based practice and high-quality research literature 

may need to have control or comparison groups for them to trust in your findings. If you 

pursue research to build a portfolio for your library promotion and tenure committee, be 

sure you understand how they will value individual versus collaborative efforts, as that 

may shape whether you execute a study on your own and acknowledge others versus 

offering the opportunity for co-authorship with greater involvement.  

In addition to the audience, consider whether the research is time-sensitive or time-

specific.  For example, is there a limited window of time in which the study is possible (e.g., 

pre- and post-construction) or are the findings to be the basis of a time-sensitive decision?  



Research Section Spotlight 

   

Hypothesis, vol. 30, no. 1, Fall/Winter 2018                                                                          46                                  

Do you have deadlines to produce publications for promotion?  These can all lead you to 

choose methods that can be planned and executed quickly although almost any 

methodology can be expedited if you have enough help or funds and the ethics 

committee/IRB finds the study to be exempt. Some examples of exempt research are 

anonymous surveys, passive observation of public behavior without collection of 

identifiers, and retrospective chart review.  Even if you have help or funding, it matters 

whether it is consistently available throughout, or in chunks, or at the time of day you 

need.  An example is whether to use student labor, your own labor, or funds to hire 

external people to transcribe interviews.  For observational studies, it might be whether 

you can do the observations at the time needed such as evenings or weekends or offsite 

daytime, or whether you need to hire or train others to do the observations, or you decide 

to rely on self-report data since no one is available to do the observations.   

2. Anonymous vs. confidential  

One of the most common questions that you must clarify for the IRB submission is 

whether the participants will be anonymous or confidential, and this often determines 

whether a study will or will not be exempt from further review. Your study design, the risk 

to participants, and the safeguards you must place on your data and whether you can 

share it often depend on getting this distinction correct in your consent documentation.  If 

data collection is truly anonymous, you do not know who participated and hopefully no  

one could figure it out from your data.  If it is confidential, you know the participants 

(people/institutions) and you take steps to conceal their identities.  There are benefits and 

drawbacks to both approaches.  
 

Choosing to collect anonymous data is generally an easier write-up for the IRB application, 

since there are fewer risks of identity exposure and fewer data protections to describe.  

However, if your data is anonymous, you cannot follow up with respondents about their 

responses, do longitudinal studies, or join the data with other data sources that contain 

identifiers if you decide to expand the scope of your research project.  Even anonymous 

surveys require data protection considerations.  For example, to be anonymous you must 

not track or download the IP addresses of participants which are often automatically 

gathered by online survey systems such as Survey Monkey or Qualtrics.  If you recruited 

by sending surveys to individual emails through the software system mailing functions, 

you may also need to have the system unlink or mask the responding email addresses.  

Survey tool functions are discussed later, with some shown in Table 1.  If questions are 

specific and the population studied is small enough, people may be identifiable from their 

responses, and this may essentially turn your study into a confidential one where you 

need to redact identifying information from open-ended questions or only share 

aggregated responses for questions that have over five respondents in a certain category.  

Confidential data collection involves assigning participants a unique identifier which is 

maintained securely and separately from research data.  Because people are identifiable 



Research Section Spotlight 

   

Hypothesis, vol. 30, no. 1, Fall/Winter 2018                                                                          47                                  

in-person, by voice, and image, methodologies such as individual and focus group 

interviews and audio or video analysis can usually only be treated as confidential data.  

 

3. Analysis planning as part of the research design  

Consider the relationship between your questions, your analysis skills, and your research 

team members.  If you have a question with multiple factors and competing explanations, 

and you intend to gather and analyze quantitative data you will need some statistical  

expertise.  It is much better to discuss your questions and data collection on the front end 

of the project with a statistician for statistical consulting so that you can come to a good 

understanding of the number of participants you will need to reach a meaningful 

conclusion and what type of data you might collect that allows you to test your question 

most robustly.  If your research team includes students or library student workers or 

volunteers, consider what they can learn about data collection and analysis, and what 

expertise or capacity they already have.  For example, while it may be more efficient to pay 

a transcription service to create transcripts of video interviews, if you have time and that 

is something you want the students to gain experience with you may decide to have them 

do some of the transcriptions.  
  

In planning your research timeline, design, and budget, it is important to consider the 

tools you have available and your familiarity with using them and what types of output 

they produce.  A common choice is deciding on what online survey tool to use (see Table 

1).  If you have designed and pilot tested your survey on paper, thinking about what 

question types will make it easy for participants to answer questions.  It can be frustrating 

when the type of question you made is not available in one of the free tools.  Be sure to 

pilot test the online version of your survey to make sure it works properly, and that the 

resulting data is captured successfully.  It is worth finding out whether you have 

institutional access to a tool such as Qualtrics or REDCap, as that will also make it possible 

to share surveys across institutions to facilitate collaboration.  You also want to test 

question types by answering them in your survey tool to make sure that the output will be 

easy to analyze. For example, if you use a multiple checkbox function in Google Forms, the 

data will be in the same column in the spreadsheet separated by a delimiter and you will 

have to then use a function to separate those responses and group them.  While the native 

Google display will show the aggregated data, you will not be able to pull out the data that 

way.  We compare the features of the most common online survey tools (Qualtrics, 

REDCap, Google Forms, Survey Monkey) available to us as university-based health 

sciences librarians in Table 1.  

 
 

Survey Tool Free Account Subscription Account Notes 



Research Section Spotlight 

   

Hypothesis, vol. 30, no. 1, Fall/Winter 2018                                                                          48                                  

Google 

Forms 

(drive.google

.com) 

Includes <15 MB of 

storage 

Unlimited surveys, 

responses, and 

questions 

  

 >15 MB of storage 

 Month-to-month fee 

Minimal data analysis available 

regardless of the account type 

Offers no support. Minimal 

types of questions available 

For questions with multiple 

response options, view 

individual, not group, 

responses to get the full 

picture. 

    Qualtrics 

(www.qualtr

ics.com) 

100 responses 

10 outgoing emails 

8 question types 

1 active survey 

Unlimited questions 

Summary reports & 

filtering 

Survey logic & 

randomization 

Online reporting 

No expiration 

Limits to features are based 

upon account type and 

subscription plan 

Expiration occurs when 

subscription to product ends 

  

Initially designed for business 

and market research surveys 

Works for other types of 

research as well 

  

REDCap 

(projectredc

ap.org) 

One week trial 

15 templates 

All collected or 

entered data is 

removed after the 

trial version 

concludes 

Unlimited templates 

Technical support for 

implementing and 

maintaining the product 

Initially designed to support 

researchers and to encourage 

collaboration   

Relatively easy to create a 

survey without using tutorials 

Video tutorials and step-by-

step instructions available in all 

versions 

Survey 

Monkey 

(surveymonk

ey.com) 

Create surveys with 

up to 10 questions or 

elements (i.e. 

question types, 

descriptive text, or 

images) 

Collect up to 100 

responses per 

survey 

Unlimited questions and 

question types, descriptive 

text, images, etc. 

No limits to collecting 

numbers of responses 

Learning curve is small. 

Not easy to share unpublished 

surveys with non-subscribers 

for testing or editing purposes. 

Support features are minimal.  

The knowledge base, while 

useful, is not easy to search 

Table 1. Comparison of Select Survey Tools based on publicly available information as of May 
2018. Compiled by Margaret Hoogland. 

Timing questions are more intensive for studies with a qualitative component, particularly 

analyzing any text or images or videos for your studies.  Will you ingest text into 

computer-assisted qualitative data analysis software tools (e.g. NVivo, Atlas.ti, Dedoose) 

for analyzing transcripts or other data resources, or will you annotate in Word or by hand 



Research Section Spotlight 

   

Hypothesis, vol. 30, no. 1, Fall/Winter 2018                                                                          49                                  

on print copies?  If you have multiple coders involved, where will you maintain the code 

listing?  Multi-institutional studies have more complexity in terms of sharing data storage, 

access controls, agreeing on tools and preferences for types of analysis.  Regardless of the 

tools used, keep track of which members of the team performed which aspects of the 

research, as many journal publishers require a statement about how each author 

contributed to the research.  

Learn about options for disseminating your research and how the journal article 

publication process works in the Spring/Summer 2019 issue of Hypothesis. 

 

Acknowledgements: This content was originally presented as a Special Content Session 

"Research Roadmap: Understanding the Research Process" at the Medical Library 

Association (MLA) Annual Meeting on May 20, 2018 co-sponsored by the Research 

Section, Leadership and Management Section, Hospital Library Section, and Educational 

Media and Technologies Section.  The session was made possible by the coordinating 

efforts of Brooke L. Billman of the MLA Research Section. We also thank the two 

anonymous peer reviewers for their thoughtful feedback.    

Disclosure: Heather Holmes is the chair of the MLA’s Research and Evidence-Based 

Practice Curriculum Committee.  

 
Works cited 

1. Godshalk VM, Sosik JJ. Mentoring and leadership. In: The handbook of mentoring at work: 

theory, research, and practice. Ragins BR, Kram KE, Eds. Los Angeles, CA: Sage, 2007: 150. 

2. Johnson WB. On being a mentor. New York, NY: Lawrence Erlbaum Associates, 2007: 21-

22. 

3. The research mentor: creativity in research: an introduction. Hypothesis 2010; 22 (2): 

11-12. 

4. Eldredge J. The research mentor: creativity in research, part 2: classifying creativity. 

Hypothesis 2011; 23 (1): 10-13. 

5. Eldredge J. The research mentor: creativity in research, Part 3: age and creative 

potential. Hypothesis 2012; 23 (2): 14-19. 

6. Eldredge J. The research mentor: interview with Phillip Eaton, M.D., on creativity in 

research. Hypothesis 2012; 24 (1): 15-16. 

7. Eldredge J. The research mentor: authorship part one: defining the article author. 

Hypothesis 2009; 21 (3): 11-14. 

8. Eldredge J. The research mentor: authorship part two: order of authors. Hypothesis 

2010; 22 (1): 8-11. 

9. Eldredge J. The research mentor: finding time for research. Hypothesis 2008; 20 (2): 5-7. 



Research Section Spotlight 

   

Hypothesis, vol. 30, no. 1, Fall/Winter 2018                                                                          50                                  

10. Eldredge JD, Ascher MT, Holmes HN, Harris MR. The research mentor: top-ranked 

research questions and systematic reviews. Hypothesis 2013; 24(2): 19-20.  

11. Eldredge J. The research mentor: interview with library researcher Jo Dorsch. 

Hypothesis 2008; 20 (3): 8-9. 

12.  Eldredge J. The research mentor: interview with library researcher Gary Byrd. 

Hypothesis 2009; 21 (1): 11-13. 

13. Eldredge JD. The research mentor: the long reach of a mentor’s influence. Hypothesis 

2014; 26 (1): 14-9. 

14. University of New Mexico. Mentoring Institute. Conferences. Available from: 

<https://mentor.unm.edu/conference/>.  Accessed 20 July 2018. 

15. Spencer AJ, Eldredge JD. Roles for librarians in systematic reviews: a scoping review. J 

Med Libr Assoc 2018;106(1):46-56. doi: 10.5195/jmla.2018.82. Epub 2018 Jan 2. 

16. Eldredge JD, Ascher MT, Holmes HN. An innovative model of evidence-based practice 

for other professions. J Med Libr Assoc. 2015;103(2):100-2. 

17. Eldredge JD, Ascher MT, Holmes HN. An innovative model of evidence-based practice 

for other professions. J Med Libr Assoc. 2015;103(2):100-2.  

18. Eldredge JD, Hall LJ, McElfresh KR, Warner TD, Stromberg TL, Trost J, Jelinek DA. Rural 

providers' access to online resources: a randomized controlled trial. J Med Libr Assoc. 

2016;104(1):33-41. 

19. Eldredge JD, Kroth PJ, Murray-Krezan C, Hantak CM, Weagel EF, Hannigan GG. How 

accurately does the VIVO Harvester reflect actual Clinical and Translational Sciences 

Award-affiliated faculty member publications? J Med Libr Assoc. 2015;103(1):19-21. 

20. Koufogiannakis D, Brettle A. A new framework for EBLIP. In: Being evidence based in 

library and information practice (pp. 11-18). London: Facet Publishing, 2016. 

21. Lindsay JM, Oelschlegel S, Earl M. Surveying hospital nurses to discover educational 

needs and preferences. J Med Libr Assoc 2017;105(3):226-232. doi:10.5195/jmla.2017.85. 

22. How to make time for research and writing. Chronicle of Higher Education.  2017 April 

9. Available from: <https://www.chronicle.com/article/How-to-Make-Time-for-

Research/239724> Accessed July 20, 2018. 

23. Preissle J. Subjectivity Statement. In: The SAGE Encyclopedia of Qualitative Research 

Methods. Ed. Lisa M. Given. Thousand Oaks, CA: SAGE, 2008. 846.  

 

Appendix 

Questions from Participants  

These questions came electronically from MLA members prior to and during the session. 
We addressed questions received verbally during the session within the body of the 
article.  

Prior to the Session Via MLA-FOCUS (n = 1) and email to Research Section and 
Hospital Libraries Section email lists (n = 2) 

https://www.chronicle.com/article/How-to-Make-Time-for-Research/239724
https://www.chronicle.com/article/How-to-Make-Time-for-Research/239724


Research Section Spotlight 

   

Hypothesis, vol. 30, no. 1, Fall/Winter 2018                                                                          51                                  

1. How can you move beyond research studies based primarily on your own work (which 
tends to be case studies), into doing research that will have bigger impact on the field? 
Or put another way, how do you come up with research studies that will be impactful, 
but are still manageable to do within the confines of your job and institution?  
 

One way to move beyond your case study is to aggregate multiple case studies from 
similar institutions to strengthen the patterns of findings (see Robert K. Yin’s Case study 
research and applications, 6th ed., 2018).  Another is to use your case study as pilot data for 
a different methodology that involves a control population or other attempts to capture or 
address confounding variables.  

 
2. How to find collaborators, especially for hospital librarians that are solo or may have 

an idea, and don’t have time to pursue by themselves?  
 
3. My question about research is how the heck can I do it as a solo librarian.  
 

Addressing questions two and three together, one approach to advancing research as a 
solo librarian is to mentor a library science student doing research as part of a field 
experience in your library or to mentor student volunteers from other social science 
disciplines in your library.  Many students have research skills or requirements but 
struggle to come up with a meaningful question or real data, both of which you can 
provide.  Author Alpi co-authors many of her research studies with student employees 
from diverse majors or library students doing field experiences. You can reach out to your 
local library school, state library association, or post your inquiry to the MLA Medical 
Library Education Section (http://www.mlanet.org/page/section-landing-medical-
library-education-section) to reach instructors of health sciences library students in 
online programs who may be interested.  In a small library, choosing projects that can 
survive a longer timeframe or partnering with other areas of the institution with more 
capacity or the ability to advocate for answering the question, e.g. nursing research, 
continuing education, or quality improvement.  

 

During the Session Verbally or via the Online Form 

1. How can you find a research mentor at an institution with promotion and tenure 
requirements similar to your own?   
 

The MLA Mentoring website allows you to look at the institution of the mentor, but 
beyond examining the title of the librarian for indicators of rank, does not offer any other 
information about those requirements. The website Academic Librarian Status 
(https://academiclibrarianstatus.wordpress.com/) offers a list of academic institutions in 
the United States and elsewhere, sorted by the professional status of their librarians.  
Although the professional school libraries may not always have the same requirements as 
the university libraries, this is a good starting point.  Mentors may have worked at 
multiple institutions with diverse requirements and therefore it is important to consider 
past experience beyond the mentor’s current institutional affiliation.  

http://www.mlanet.org/page/section-landing-medical-library-education-section
http://www.mlanet.org/page/section-landing-medical-library-education-section
https://academiclibrarianstatus.wordpress.com/


Research Section Spotlight 

   

Hypothesis, vol. 30, no. 1, Fall/Winter 2018                                                                          52                                  

2. I’m gung ho about my research project but my more experienced research partners are 
not as excited. Any advice on finding research partners who are excited to work with 
new researchers on their early projects? 
 

The first question that comes to mind is have you asked why they are not excited—it is the 
topic, the method, the role, the collaborators, or their own capacity? The piece of this 
article about what you offer mentors may provide some insight.  If your institution is 
pressuring them to produce more advanced evidence, you may be better off finding peer 
collaborators or mentors outside your institution who do not feel the same pressure.  You 
must be open to the idea that your research project is not as compelling to others as it is to 
you—check the literature to see how your proposed idea may be received if you decide to 
go forward.   
 
3. What are your opinions of keeping daily stats of everyday work and do you know of 

ways to turn those daily stats into a story that highlight librarian/library impact? 
 

At NC State University we track public service daily work statistics and space usage using 
an open-source program called Suma (https://www.lib.ncsu.edu/projects/suma) to 
gather data. It has built-in analytic tools.  We track activities by user type and when we see 
changes or growth with a certain user population this invites us to dig deeper.   For 
example, if we are doing more transactions with house officers, we then might interview a 
few to understand how we are impacting them.  Since intern/resident support is an 
important issue, telling a story of how we support them would resonate.  We also tell 
Library Stories based on experiences, see https://www.lib.ncsu.edu/stories/supporting-
ncsu-researchers-achieving-nih-public-access. 

4. Clinicians and academic faculty barely have time or interest for my short 7 question 
surveys, but the questions I have are best answered by surveys. How can I better 
engage my test audience? Is there an alternative to surveys? 
 

One of the best ways to engage an audience is to have co-authors who represent that 
audience.  Can you co-author the survey with key opinion leaders or have them distribute 
the survey for you?  If you have a library committee with members from your test 
audience, they might be able to assist.  Depending on the topic of your surveys, it may be 
easier to do a mini-focus group with representatives of the test audience at a meeting.  
Feel free to contact author Alpi with more details to receive more focused feedback. 
 
5. When your paper is submitted, and you are about to revise and resubmit, can the 

author ask for new reviewers?  
 

Probably not. When you are asked to revise and resubmit your manuscript, the editor of 
the journal most likely wants the original reviewers to re-evaluate the revised manuscript 
to ensure that your revisions have sufficiently alleviated their specific concerns. Moreover, 
inviting new reviewers at this point would likely only prolong the peer review process and 
create more “hoops” for you to jump through. However, if you have serious concerns 
about a reviewer—perhaps you feel that they did not carefully read your manuscript or 

https://www.lib.ncsu.edu/projects/suma


Research Section Spotlight 

   

Hypothesis, vol. 30, no. 1, Fall/Winter 2018                                                                          53                                  

were unreasonably critical—you can privately express those concerns in an email or 
formal letter to the editor, who will take your concerns into consideration when weighing 
the reviewers’ opinions to arrive at a final decision regarding acceptance. 

 

 

 

 

 

  

National Library of Medicine (US). Digital Collections [Internet]. 

Bethesda (MD): Medline keyboard in the reading room [196?]. Available from: 

http://resource.nlm.nih.gov/101445854 

 

 

 

 

http://resource.nlm.nih.gov/101445854

