978 Research Needs and Learning Format Preferences of Graduate Students at a Large Public University: An Exploratory Study Hilary Bussell, Jessica Hagman, and Christopher S. Guder* This article reports on a study of research needs and learning prefer- ences of graduate students at a public research university. A sequential exploratory mixed-method design was used, with a survey instrument developed from an initial qualitative stage. Significant differences were found between master’s and doctoral students’ and on-campus and on- line students’ confidence with several research skills. Graduate students overall prefer asynchronous online options and in-person workshops over synchronous online instruction and in-class presentations for learning research skills. The article concludes with a discussion of the value of the sequential exploratory mixed-method design for informing practice at an individual institution. Introduction: Background and Research Questions As part of their preparation to become independent scholars and professionals, gradu- ate students are expected to follow a research path that is largely self-structured and that pursues questions that are specialized and original.1 Graduate students may find it difficult to live up to this expectation for a variety of reasons. Because undergraduate research experiences vary widely, graduate students may not be prepared for graduate- level research when they matriculate.2 Once in graduate school, students do not always learn the skills they need to do independent research through their coursework.3 Many faculty members assume graduate students already know how to do research, and thus do not emphasize the acquisition of research skills in their roles as teachers and advi- sors.4 Graduate students are more likely to ask for help from their professors than from librarians, but academic programs do not always provide help in acquiring the skills to perform effective literature reviews and to engage in other research activities.5 Further, graduate students bring to their programs a variety of prior experiences, career goals, Hilary Bussell is Assistant Professor and Social Sciences Librarian at The Ohio State University Librar- ies; e-mail: bussell.21@osu.edu. Jessica Hagman is Social Media Coordinator and Subject Librarian and Christopher S. Guder is Reference and Instruction Librarian in Alden Library at Ohio University; e-mail: hagman@ohio.edu, guder@ohio.edu. ©2017 Hilary Bussell, Jessica Hagman, and Christopher S. Guder, Attribution-NonCommercial (http://creativecommons.org/licenses/by-nc/4.0/) CC BY-NC. doi:10.5860/crl.78.6.761 mailto:bussell.21@osu.edu mailto:hagman@ohio.edu mailto:guder@ohio.edu http://creativecommons.org/licenses/by-nc/4.0/ https://doi.org/10.5860/crl.78.6.761 Research Needs and Learning Format Preferences 979 and expectations. A doctoral student who is preparing for a career in academia will likely have different experiences and needs than a student in a professional master’s program when it comes to developing his or her research skills and agenda. These realities present a challenge for libraries in supporting graduate students. Common library instruction formats, such as one-shot sessions and orientations, are often ineffective for graduate students.6 Although librarians may work with graduate research methods classes, students often take these at the beginning of their time in graduate school, long before they begin work on a dissertation or thesis. Additionally, the fact that many graduate degree programs are now offered either partially or fully online means that there is a growing group of graduate students who may never set foot on campus but who have distinct research and information needs that libraries should support. This paper presents the results of a study undertaken to help one library better support its graduate students’ research needs in this complex landscape.7 Ohio University is a public research university in Athens, Ohio, with ten academic colleges offering graduate degree programs, several of which are taught fully or primarily online. The number of graduate students enrolled for the spring 2014 semester was 4,703, excluding medical students.8 It has been a strategic priority for the Ohio University Libraries to expand its support for graduate students, but Ohio University librarians have been uncertain about how to translate this into their day-to-day liaison activities. With only seventeen subject librarians, many of whom have multiple subject assignments and additional library responsibilities, it is difficult to support the breadth of graduate research projects across campus through individual reference consultations and course instruction. Anec- dotal data shared among librarians and gathered in the libraries’ reference transaction statistics suggest that graduate students across disciplines need help gaining many of the same research skills, and thus one approach might be for librarians to collaborate on developing a unified program for providing support and instruction to graduate students across campus. However, library workshops were abandoned several years ago due to perpetually low attendance, leaving librarians wary of trying this strategy again. Deciding that more data were needed to inform future efforts in this area, the authors of this paper undertook a sequential exploratory mixed-method study of graduate student research needs across their institution. The study was designed to provide evidence that could inform subject librarians and library administration on both the types of research needs graduate students have as well as their preferred format for learning research-related skills. The following questions were used to guide the study: • RQ1: What are the self-identified research needs of graduate students at a large public research university? □ RQ1A: Do these needs differ between on-campus and online students? □ RQ1B: Do these needs differ between master’s and doctoral students? • RQ2: How do graduate students prefer to learn research skills? □ RQ2A: Do these preferences differ between on-campus and online stu- dents? □ RQ2B: Do these preferences differ between master’s and doctoral students? In addition to discussing the results of this study, this paper will reflect on the value of the sequential exploratory mixed-method approach to understanding local user needs, which may be of use for other libraries considering employing a similar approach to understand a segment of their own user populations. Literature Review Librarians have long been interested in offering services specifically tailored to the graduate students at their institutions. In 1976, Eldridge Smith introduced a series 980 College & Research Libraries November 2017 of articles in College & Research Libraries on services to the “graduate community” by noting that it had become “insufficient” for university libraries to support graduate students by building robust library collections.9 In the intervening years, many librar- ies have sought to learn about the challenges that their graduate students have faced as they learn to find and use information in pursuit of their research projects. Of late, there has been a growing interest in understanding how academic libraries can sup- port graduate students in all aspects of their research, and in meeting the needs of the growing population of students who take classes online. Challenges in Finding and Using Information Recent studies of graduate students show a strong preference for easily accessible online resources and frustration with the complexity of accessing needed resources online and through library-based subscriptions. Graduate students often begin research tasks with a Google search and show affinity for Google Scholar as an easy-to-use and useful tool for identifying research sources.10 At the same time, graduate students report dissatisfaction with library-based research tools even as librarians have sought to improve the breadth of resources available online to their patrons. When asked about their research challenges, Georgetown graduate students expressed frustration at the process required to find the full text of journal articles and books in the stacks, a concern echoed by the nontraditional graduate students at Adelphi University and graduate students who had completed a literature review workshop at Oregon State University.11 Unfortunately, this lack of ease with library tools and preference for easily accessible information can cause students to ignore valuable resources that are difficult to access in favor of those that are easier to find.12 Once they have identified sources of information, researchers must decide how to organize information so it will be accessible as they write up the results. While there are now many citation management tools to automate this process, many graduate students say they do not use these tools, as they can be difficult to integrate into their workflow. In a multi-institution study of science researchers, Niu et al. found that only half of the respondents kept a “bibliographic database” of research materials.13 Humanities students interviewed at Cornell and Columbia universities indicated that while they wanted to use citation tools, they did not often actually use such tools during their own research projects.14 Melles and Unsworth surveyed participants in a literature review workshop and interviewed both graduate students and academics to understand their decision-making process about citation tools. They found a reluctance to begin using new tools, given the high time-cost of adapting their existing research materials and information management process.15 Citation management is a likely contender for a library-based workshop, given librarians’ familiarity with such tools and the students’ presumed need to develop their own information management systems. It appears, however, that not all students consider learning such a tool to be a priority. Challenges throughout the Research Cycle While most research on graduate student needs has focused on finding and managing information, some libraries have investigated ways to support students in all aspects of the research process. At the University of Michigan, for example, a survey of graduate students in the humanities revealed strong interest in learning more about the process of publication. Panel sessions with faculty members ultimately proved to be a draw for graduate students.16 The University of Denver has offered workshops on grant funding for graduate students. Focus groups held with attendees of these workshops indicated that the content was well received in an environment where students felt unable to find the funding they need for research and struggled to understand the process of winning Research Needs and Learning Format Preferences 981 grant funding.17 A survey of graduate students at nine different Canadian universities found that 90 percent of students were confident in their ability to manage the data they collected for research projects. Yet 14.2 percent said they needed to recollect lost data, and 17.2 percent said that they had lost data files.18 In a recent survey at Rutgers University, graduate students demonstrated interest in workshops on several research- related topics, including finding materials, identifying grant funding, data analysis, and formatting dissertation documents. Interestingly, the same study also included interviews with graduate program directors who expressed interest in workshops on the discovery of information and issues of plagiarism and ethics, topics that garnered little enthusiasm among students surveyed.19 This disconnect raises the question of whether students will find the training they need to conduct research in their courses or through their academic departments. For students who are not able to learn these research-related skills in their coursework or departments, library support services may serve as a means to building these academic skillsets. Library Outreach to Graduate Students Recognizing that graduate students may face challenges as they develop into in- dependent researchers, many libraries have offered learning opportunities to help graduate students develop the skills they need. At some libraries, this takes the form of workshops, like the Savvy Researcher series at the University of Illinois at Urbana- Champaign,20 the Graduate Library User Education series at the Georgia Institute of Technology,21 or the library training program for ABD distance doctoral students at Nova Southeastern University.22 Instruction may also be integrated into the students’ regular coursework, through online tutorials,23 or through librarians’ direct contact and consultation with students either in person or online as the number of online graduate students grows.24 For online students, the library website often serves as the primary means of contact with and information about library resources and offerings, which could leave students unaware of the full range of resources available to them. Like their counterparts in on- campus programs, graduate students in online programs report challenges in accessing the full text of journal articles. For students in the Graduate Social Work program at Marywood University, this proved to be true even when students reported that they had received instruction on how to use library resources. Also like their on-campus counterparts, online graduate students can be unlikely to ask for help from librarians, except as an option of last resort.25 Graduate Perceptions of Library Support In seeking to offer support for graduate students who face challenges throughout the research cycle, some libraries have run into difficulties connecting services with their intended audience, with graduate students ultimately proving to be “misunderstood, elusive or hard to reach.”26 There is evidence that graduate students do not think of the library as a place to turn for assistance when they meet research challenges. Fleming- May and Yuro’s focus groups with graduate students indicated that students were not sure librarians could offer appropriate assistance, believing that they should be able to do research-related work independently.27 This emphasis on self-sufficiency can also be seen among graduate students at Georgetown, who “prize their capacity to do the work independently, or perceive a lack of subject expertise in those available to help them,” and students at Oregon State University, who learned library-related research skills on their own.28 When asked in a survey about the importance of different library services, graduate students at Notre Dame ranked reference assistance and subject librarian services as least-used and of low importance.29 Ismail’s study of help-seeking 982 College & Research Libraries November 2017 by graduate students found that adult learners generally did not seek assistance from librarians, preferring instead to ask classmates or instructors.30 Even when relevant support is offered, students may not be aware of the services available to them through their library. Interviews with graduate students at the University of Colorado-Boulder that were intended to solicit desired service offerings showed that at least some of the requested support was already available through the library. Some of the participants indicated that they would not have thought of the library as a “place to turn for such support.”31 This lack of interest in or knowledge of library-based support may have less to do with the library or librarians, but instead be due to the students’ experiences with their department, which serves as the center of their academic lives.32 Any effort to offer support services to graduate students must contend with these attitudes and fit into graduate students’ academic lives as seamlessly as possible. The literature shows a wide range of ways that libraries have sought to support graduate students as they develop into independent researchers. Less explored, how- ever, is the question of to what extent graduate students struggle with each aspect of the research process, and whether the formats in which libraries offer instruction match the preferences of graduate students. The present study seeks to build on this literature in two ways: 1) by exploring the range of graduate students’ research needs; and 2) by identifying how graduate students, whether on-campus or online, in mas- ter’s or in PhD programs, prefer to receive instructional support as they develop into independent researchers. Methodology Mixed-Method Design The goal of the present study is to better understand the research needs and learning preferences of the graduate student population at Ohio University to determine the best means of addressing those needs. Previous library-based surveys of graduate students have focused on perceptions of the library’s resources, spaces, and services, or on the information-seeking and collection stages of the research process.33 Because the pres- ent study’s research questions are broader in nature, a research design was selected that would allow the researchers both to explore the authentic experiences of graduate students at Ohio University and to gather feedback from a large number of students. To that end, the study employed a sequential exploratory mixed-method design, in which an exploratory, qualitative stage was followed by a quantitative second stage. Sequential exploratory mixed-method design is frequently used by researchers seek- ing both to explore a phenomenon and to understand how the findings of qualitative research are distributed among members of a population, often by creating a new survey instrument.34 This method allowed the researchers to use focus groups and interviews to learn more about the experiences of graduate students as they learn to do research and to build a survey based on the qualitative data that shows the distribution of those experiences throughout the population. Qualitative Methodology: Focus Groups and Interviews Qualitative Data Collection The initial stage consisted of focus groups with on-campus students and individual interviews with online students. After securing Institutional Review Board (IRB) ap- proval, an e-mail was sent in June 2013 to all graduate students asking them to fill out a form on the library website if they wished to participate in the study. Interested students were contacted to arrange a time to attend a focus group or meet for an online interview. On-campus participants (n = 25, over 6 focus groups) received a $10 gift card to the university’s dining facilities, as well as refreshments during the focus group. Online Research Needs and Learning Format Preferences 983 students (n = 9) earned a $15 credit in their student account with the bursar’s office. The same questions were posed to both groups (see appendix A). The focus groups were recorded and transcribed by members of the research team. Interviews with online students were conducted via the chat function in Adobe Connect, which provided an automatic transcript. Qualitative Data Analysis The qualitative data were analyzed using the applied thematic analysis (ATA) ap- proach described by Guest, MacQueen, and Namey.35 Like grounded theory, ATA allows researchers to examine their textual data to build an understanding of the themes contained therein. However, ATA places less emphasis than grounded theory on building a theory using those data. Using ATA in this study enabled the research- ers to conduct an exploratory analysis of the interview and focus group results to understand what challenges participants faced and skills they needed to learn to do research. After transcribing the focus group recordings and collecting the text from the Adobe Connect interviews, each researcher reviewed each of the focus group and interview transcripts, identifying challenges faced by participants as they developed their own research skills. Other themes were also noted as they arose, such as problems students had encountered with their academic programs and intellectual and personality traits that they considered important for researchers to possess. The researchers met to compare analyses and flesh out understandings of how iden- tified themes overlapped or diverged. This led to the development of a codebook in which consistent language was used to describe each theme. Each interview and focus group transcript was then recoded using Dedoose, a web-based qualitative analysis tool. For the second analysis, the responses of each focus group participant were col- lected into single documents to analyze responses by individual student. Quantitative Methodology Survey and Sampling Using the themes developed from the qualitative stage, the researchers developed an 18-question survey in Qualtrics (see appendix B). The survey asked students to rate their confidence in their ability to complete a number of research-related tasks and their preferences for learning new skills related to their research. The survey also contained questions about the students’ use of technology and citation software for research, access issues and general satisfaction with the library, and how well they felt their undergraduate and graduate programs had prepared them for research. These latter questions, however, fall beyond the scope of the specific research ques- tions of this paper. During the spring 2014 semester, an e-mail list of all currently enrolled graduate students was obtained from Ohio University’s Office of the Registrar, along with data about academic college affiliation, program, degree sought, and on-campus or online status. Given this access to the entirety of the study population, this study used saturation sampling, a strategy in which all members of a population are sent the survey instrument rather than seeking a sample.36 Conducting an online survey means confronting nonresponse bias, or the possibility that respondents will differ from nonrespondents in ways that will render the results invalid.37 Therefore, it was decided that saturation sampling, rather than selecting a sample of the population, would yield the largest and most representative sample of the student population in our results. 984 College & Research Libraries November 2017 Participants The survey was distributed in April 2014. A link to the survey was e-mailed to every graduate student enrolled on the main campus (n = 2,840) and in online programs (n = 1,646), for a total of 4,486.38 The survey remained open for one week, with a midweek reminder sent out to those who had not yet completed the survey. Re- spondents were given the opportunity to register for the chance to win one of ten $50 Amazon Gift Cards. Of the 4,486 students contacted, 842 started the survey by completing the first ques- tion indicating their consent to participate. In total, 803 participants continued on to answer subsequent survey questions, resulting in a final response rate of 17.9 percent. Overall, the survey participants resemble the student population in terms of academic college affiliation, with some differences in on-campus versus online enrollment and degree sought. Table 1 shows the distribution of the survey participants by graduate college. Arts & Sciences students are overrepresented by 5.82 percent, while Business students are underrepresented by 4.95 percent. The other colleges show participation rates within 2 percent of the population rate. On-campus students are overrepresented compared to their peers in online programs, with on-campus students making up 12.03 percent more of the participants than the overall population (see table 2). Finally, doc- toral students are somewhat overrepresented among the survey participants, making up 7.37 percent more of the participants than their rate in the population (see table 3). TABLE 1 Distribution of Graduate Student Population and Survey Participants, by Academic College College Number of Graduate Students in College Percentage of Total Graduate Student Population Number of Survey Participants From College Percentage of Total Responses Over/Under Representation of Population in Survey Arts & Sciences 700 15.60% 172 21.42% 5.82% Business 680 15.16% 82 10.21% –4.95% Communication 192 4.28% 42 5.23% 0.95% Education 718 16.01% 143 17.81% 1.80% Engineering & Technology 426 9.50% 61 7.60% –1.90% Fine Arts 245 5.46% 35 4.36% –1.10% Graduate College* 132 2.94% 9 1.12% –1.82% Leadership & Public Affairs 69 1.54% 17 2.12% 0.58% Health Sciences & Professions 1,207 26.91% 210 26.15% –0.75% International Studies 117 2.61% 32 3.99% 1.38% Research Needs and Learning Format Preferences 985 Results Research Question 1: Identifying Research Needs The first research question seeks to discover graduate students’ self-identified research needs and whether these needs differ between students in on-campus and online programs, and in master’s and doctoral programs. Focus Group and Interview Results In focus groups with on-campus students and interviews with online students, par- ticipants were asked about challenges they have faced in learning to conduct research and the skills they believed they would need to learn to become effective and efficient researchers. The students described a wide range of challenges and skills; however, the largest areas of concern were finding and accessing library materials, developing a literature review, collecting and analyzing data, and developing the intellectual and personality attributes they would need to succeed, skills beyond the scope of traditional library instruction. Finding and Accessing Research Materials Twelve out of thirty-four students in the focus groups and interviews described dif- ficulties with accessing research materials via the libraries’ subscriptions. For three students, this was a technical problem with the proxy log-in system that allows access to materials from off campus. Eight students specifically mentioned having difficulty getting to the full text of articles or other research materials. These two issues are frequently addressed by librarians working at the reference desk or staffing the chat reference service. While some issues are beyond the library’s control—students not enrolled for classes, for example, cannot get access to library materials even if they plan TABLE 3 Distribution of Graduate Student Population and Survey Participants, by Degree Sought Number of Graduate Students by Degree Percentage of Total Graduate Student Population Number of Survey Participants by Degree Percentage of Total Responses Over/Under Representation of Population in Survey Non-degree 127 2.83% 6 0.75% –2.08% Masters 3,399 75.77% 566 70.49% –5.28% Doctoral 960 21.40% 231 28.77% 7.37% TABLE 2 Distribution of Graduate Student Population and Survey Participants, On-Campus and Online Students Number of Graduate Students Enrolled On-Campus or Online Percentage of Total Graduate Student Population Number of Survey Participants Enrolled On-Campus or Online Percentage of Total Responses Over/Under Representation of Population in Survey On-campus 2,840 63.98% 605 75.34% 12.03% Online students 1,646 36.69% 198 24.28% –12.03% 986 College & Research Libraries November 2017 to continue graduate-level work—such comments are a reminder that these challenges remain a barrier to research. Twelve students also described difficulties searching for information resources that they needed for research projects, with a number of students describing finding articles as a major challenge. They encountered difficulty when trying to decide where and how to search for relevant articles, “finding niche articles,” “finding enough peer-reviewed sources” within the time frame of a shorter-term class, or choosing keywords that would lead to the most relevant sources. One online student mentioned having developed skills for narrowing searches after his/her program organized an on-campus residency in which the subject librarian for the program provided ap- propriate instruction. Writing a Literature Review In addition to the challenge of locating articles, seven students described difficulties with knowing when to stop looking for articles and writing the results of their searches into research reports or literature reviews. Some students were overwhelmed by the prospect of “putting together” all of their literature into coherent narratives, or what one student described as “working it into a written sort of thing.” One student remarked that some students in their program were working through “hundreds of articles” as they began to work on their theses. But even with a large number of sources compiled, other students felt a “looming fear” that they would fail to include important work in their research projects, leading to a thesis committee asking them, “‘How did you not find this person?’” Accessing, Collecting, and Analyzing Data Five on-campus students talked about challenges accessing, collecting, and ana- lyzing data. For some students, accessing the data they needed to answer their own research questions was difficult: either the data were simply unavailable, not suitable for their project, or held by an organization that was unwilling to release the data to the student. Other students said they did not know how to analyze data they had collected, with one going so far to say as to have “no idea” how to analyze the data being entered into a data analysis program. One student mentioned limited access to data analysis software, given that his/her department had only one computer with this software and the library computers did not have this particular program. Other Challenges Beyond accessing information resources, writing a literature review, and working with data, students mentioned a range of other challenges and needed skills throughout the research process. These included finding funding for a research project, citing sources in the appropriate style, using various technology tools, and developing a poster to share research results. A surprising result from the qualitative portion of the study was the mention of what could be considered intellectual or personality attributes. When asked what skills they needed to develop to become effective and efficient research- ers, nine on-campus students in four of the focus groups brought up intellectual skills like the need to be patient, flexible, or bold. These may have come to mind given the students’ awareness of the differences in expectations for graduate-level research compared to expectations in undergraduate programs. Three students in the focus groups also brought up these expectations when discussing their research challenges. Research Needs and Learning Format Preferences 987 Survey Results Self-identified Research Needs Regardless of Group Survey participants were asked to rate their ability to complete 18 different research- related tasks using a 5-point Likert scale on which 1 was lowest confidence and 5 was highest confidence. Participants could also indicate that they were “not sure” of their ability to complete the skill. Respondents were not required to answer any of the questions, leading to a different number of responses for each question. Responses left blank or marked as “not sure” were excluded from this analysis. Table 4 shows the three skills that the participants rated highest and lowest in terms of their confidence in the described skill. The higher mean scores indicate that, overall, participants were most confident in their ability to cite sources in the appropriate style (M = 4.05), identify previously published research on their topics (M = 3.90), and access the full text of previously published research (M = 3.86), even though the focus group and interview participants reported difficulty finding and accessing the full text of resources. Lower mean scores indicate that participants were least confident in their ability to deter- mine where to publish their research (M = 2.48), successfully apply for grant funding for their research (M = 2.05), and identify sources of grant funding for their research (M = 2.04). On-campus vs. Online To help answer Research Question 1A (RQ1A), differences were identified between on-campus and online students’ confidence levels on research-related skills. To answer RQ1A, independent samples t-tests were used to investigate whether there were differ- ences between on-campus and online students’ confidence levels on research-related skills. Four skills showed statistically significant differences between the two groups at P < .05 (see table 5). Effect sizes for the differences (r) are also shown on table 5. Overall, online students showed greater confidence in their ability to develop a poster presentation and to decide when to end the search for previously published research. On-campus students rated themselves more confident in their ability to access the full text of previously published research and access books needed for research. While these differences are significant, the effect sizes are small, indicating that the amount of variance in the mean scores between the two groups can be only partially explained by the students’ status as online or on-campus students. Master’s vs. Doctoral To address Research Question 1B, differences between master’s and doctoral students’ confidence levels on research-related skills were compared using independent samples t-tests. There were significant differences in seven areas, with doctoral students report- ing significantly higher confidence levels than master’s students in each (see table 6). Again, the effect sizes are small. TABLE 4 Highest and Lowest Rated Skills, All Students Skill n Mean Cite sources in the appropriate style 783 4.05 Identify previously published research on your topic 782 3.90 Access the full text of previously published research 781 3.86 Determine where to publish your research 753 2.48 Successfully apply for grant funding for your research 685 2.05 Identify sources of grant funding for your research 705 2.04 988 College & Research Libraries November 2017 Research Question 2 Focus Group and Interview Results Learning Preferences In the focus groups with on-campus students and interviews with online students, participants described how they would prefer to learn new skills once they had de- termined they needed them to complete their research. Given that they are generally not on campus to attend events, it is not surprising that online students preferred an online option for learning new skills. Some of the on-campus students, however, also indicated that they would like to see online options for learning research-related skills. In discussing their preferred format for learning new skills, 14 students, evenly split across the on-campus and online groups, said that they would like to have an option to view a video to help them learn new research-related skills. TABLE 5 Skills with a Significant Difference between On-campus and Online Students (P < .05) Skill Campus n Mean Effect Size (r) Develop a poster presentation On-campus Online 583 188 3.45 3.80 .19 Access the full text of previously published research On-campus Online 586 195 3.90 3.72 .08 Access books needed for research On-campus Online 575 192 3.72 3.51 .02 Decide when to end the search for previously published research On-campus Online 564 189 3.04 3.30 .10 TABLE 6 Skills with a Significant Difference between Master’s and Doctoral Students Degree Sought n Mean Effect Size (r) Compile a literature review Master’s Doctoral 553 229 3.51 3.74 .10 Determine where to publish your research Master’s Doctoral 520 227 2.37 2.74 .15 Preparing a conference proposal Master’s Doctoral 522 220 2.94 3.39 .16 Cite sources in the appropriate style Master’s Doctoral 550 227 4.00 4.17 .08 Access the full text of previously published research Master’s Doctoral 547 228 3.77 4.06 .17 Collect data Master’s Doctoral 536 224 3.43 3.71 .12 Store or manage data Master’s Doctoral 534 224 3.42 3.61 .08 Research Needs and Learning Format Preferences 989 Survey Results Learning Preferences, All Students Participants ranked their preferences on five learning format options drawn from the qualitative data, from most preferred (scored as a 1) to least preferred (scored as a 5). This yielded an average score for each format that could be compared, with lower scores indicating that the option was more preferred among the participants (see table 7). Among all participants who ranked their learning format preferences (n = 712), “a video I could watch as needed” was ranked the most preferred among all the students (with the lowest overall score, M = 2.64), giving it a slight edge over “a website with images I could consult as needed” (M = 2.68) and “an in-person work- shop” (M = 2.86). Learning Preferences, On-Campus vs. Online To help answer Research Question 2A, the learning format preferences were compared between on-campus and online graduate students (see table 8). On-campus students showed the strongest preference for in-person workshops (M = 2.69), closely followed by a website (M = 2.71) and a video (M = 2.71). Online students showed a preference for a video (M = 2.32), with a website (M = 2.57) and a live online workshop (M = 2.80) following behind. Independent samples t-tests were run to determine whether these differences were significant. As shown on table 8, all formats except a website showed a significant dif- ference in mean scores when comparing on-campus versus online students. Learning Preferences, Master’s vs. Doctoral Learning format preferences were compared between master’s and doctoral students to help answer Research Question 2B. Master’s students showed a preference for a video (M = 2.59), followed by a website (M = 2.75) and an in-person workshop (M 990 College & Research Libraries November 2017 =2.95). Doctoral students showed a preference for a website (M = 2.54), followed by an in-person workshop (M = 2.66) and a video (M = 2.76). Independent samples t-tests were run to determine if the differences among these two groups were significant. In this case, only two learning formats (in-person work- shops and online workshop) showed significant differences at P > .05. Discussion Needs across the Research Spectrum The findings suggest that graduate students have research needs in a wide range of areas, from foundational skills such as choosing effective keywords, known item searching, and accessing article full text, to advanced skills like finding and analyzing data and disseminating research findings. While the qualitative findings reveal the scope of these needed skills, the quantitative findings give a more detailed picture of where these needs are distributed across different groups of graduate students. Breaking down the data in this way provides evidence for whether and how to tailor instructional offerings to the needs of specific groups. Both on-campus and online students discussed difficulties in finding and accessing research materials in the qualitative portion of the study, though the survey results indicated that, overall, students were more confident in their ability to find and cite sources in comparison to other research-related skills. The students differed, however, in their self-described need to find and analyze data and to develop certain intellectual and personality attributes to become effective researchers. These needs were only identified by students in the on-campus programs. While this does not necessarily mean that these skills are not relevant to the students in the online programs, these challenges appear not to be a priority to this group, as they were not mentioned by any online students in discussing difficulties they had faced. Instead, they focused on access and searching challenges. The quantitative findings show that master’s students have significantly less confi- dence than doctoral students in several research-related skills. These skills include both what could be considered more foundational (like citing sources) and more advanced (such as storing and managing data). Libraries should consider how they can better support master’s students specifically in gaining these skills. This can be trickier than it may first appear, since many institutions, like Ohio University, offer graduate courses Research Needs and Learning Format Preferences 991 that include both terminal master’s and doctoral students. Librarians could reach out directly to master’s students, especially those who will be writing a thesis or intend to apply to a doctoral program, by asking graduate thesis advisors to help spread the word about research skills support services to these students. Additionally, online students reported lower confidence levels than on-campus students in foundational skills related to accessing materials needed for research. This suggests that libraries should do more to help online students understand how to navigate online library resources and find the full-text link or link resolver in data- bases, as well as to promote Interlibrary Loan services and options for sending books to off-campus students. Some Online Learning Options Are Popular—But Not Synchronous Formats The qualitative findings suggest that on-campus as well as online students are open to learning research skills through online formats. This is backed up by the survey findings, with the two most preferred formats overall being videos that can be watched when needed and websites with text and images. These preferences persist when the results are broken down by group. While on-campus graduate students rated in-person workshops as their most preferred format, this was followed closely by websites and videos, which were the two formats most preferred by online students. Similarly, videos were rated the most preferred format by master’s students, and websites were rated most preferred by doctoral students. The popularity of text- and image-based websites as well as videos among graduate students provides support for creating online guides using platforms such as LibGuides or WordPress for this population. On the other hand, librarians should be wary of investing time or resources in live online workshops for graduate students. The live online workshop format was rated at or near the bottom by both online and on-campus students and by both master’s and doctoral students. This confirms anecdotal observations of many librarians at the researchers’ institution who have offered online workshops in the past. The fact that “a video that I could watch when needed” was rated the highest overall and in the top three preferred formats for all groups suggests that it is the synchronous nature of the live online workshop that graduate students do not like, rather than the online format itself. In-Person Workshops over In-Class Presentations Unlike their online cousin, in-person workshops were ranked as one of the more highly preferred formats by graduate students. Given the low attendance at workshops offered by Ohio University, the researchers found this result particularly surprising and worthy of further exploration, as will be discussed below in “Directions for Future Research.” By contrast, in-class presentations were one of the least preferred formats. It may be disheartening to see in-class presentations rated so low, as this is a common way for librarians to provide instruction, but this finding supports the researchers’ suspicion as well as literature indicating one-shot sessions are insufficient for graduate students. Considered in light of graduate students’ preference for asynchronous online options (such as videos and websites) and in-person workshops, these results suggest that graduate students want to be able to choose when and where they learn the skills they need for their research. Directions for Future Research The workshop model offers a promising solution to the problem, articulated at the outset of this paper, of how libraries can offer support and instruction to graduate students across campus in learning the skills they need for their research, without 992 College & Research Libraries November 2017 requiring each subject librarian to provide the entirety of this instruction for the stu- dents in their departments. As noted above, the fact that workshops often have low attendance makes it all the more surprising that this format was rated among the most highly preferred by graduate students. This finding suggests that it may not be the format itself but rather the content, timing, and promotion that contributes to low at- tendance. Future research could look at whether adjusting these variables, combined with offering instructional content based on the research needs identified in this study, leads to better attendance. It is also worth exploring other measures for success than attendance, like assessing how much attendees learned from the workshop by follow- ing up with them after some time has passed. This study offers evidence for libraries to continue exploring ways to implement an Information Commons model to support their graduate students. The qualitative find- ings suggest some graduate students do not receive adequate support and instruction from their academic programs in accessing, collecting, and analyzing data. While these skills could be taught to a limited degree in a workshop setting, graduate students would benefit from additional support in the form of a dedicated space with drop-in help and research computing programs they can use if their departments don’t offer them. Future research might consider whether this type of dedicated space, paired with a robust workshop series and drop-in help, perhaps in conjunction with other research-focused units on campus (such as the Office of Research or the Grants Office, among others), could contribute to the cultivation of the intellectual and personality attributes that respondents in the qualitative portion of the study identified as impor- tant to becoming effective researchers. In researching the feasibility of this approach at their campuses, future researchers may consider reaching out to the Graduate Col- lege or other graduate student-focused organizations during the survey-development stage. Although the present study asked participants about research skills that are not traditionally considered “library skills,” feedback from nonlibrarians who work with graduate populations could add to the robustness of a study by ensuring that a broad range of skills are addressed. Finally, a point of future analysis could be to compare differences in research needs and learning format preferences across disciplines. Although the survey included data on the academic college of each respondent, there were not enough responses from each of the 10 academic colleges to draw meaningful distinctions among them. A future study could avoid this pitfall by comparing across general disciplinary categories (social science, natural science, and others) as opposed to academic college. Reflections on the Research Design A unique aspect of this study was its sequential exploratory mixed-method design. An overall goal was to learn what graduate students identify as research needs in a way that avoided, as much as possible, imposing the researchers’ preconceived ideas. The researchers also wanted to design a study that would be effective in producing data about the unique population at Ohio University; thus, it was important to develop a survey based on qualitative themes identified by Ohio University graduate students, as opposed to one based on another institution’s population. Starting with focus groups and interviews helped reveal a range of challenges faced by Ohio University gradu- ate students, which the researchers then used as a basis for their survey. The range of these challenges was wider than the researchers anticipated, with graduate students expressing needs in areas that the researchers might have considered too foundational to ask about had they developed the survey from scratch. The survey results bore this out, enabling the researchers to identify which segments of the graduate student population need support in certain areas. Research Needs and Learning Format Preferences 993 There is a connection to be drawn between implementing this type of research de- sign and practicing evidence-based librarianship. Because it explores a specific user population’s needs without imposing a predetermined set of ideas, the sequential exploratory mixed-method design lends itself particularly well to gathering data to inform practice at a specific library. To that end, the authors have attempted to describe the methods and results of this study in a clear enough way that other librarians could replicate them. It is hoped that this will empower other librarians to use the research design described in this paper to gather evidence to inform practice at their own local institutions. Conclusion This study contributes to a growing body of literature on library support for graduate students. Results indicate that graduate students prefer to learn research skills through formats they can access on demand or attend when they need them, rather than during class. The fact that many graduate students take research methods classes at the start of their programs, and that doctoral students continue working on dissertations for years after finishing coursework, are additional reasons to develop opportunities for continued learning and retooling outside of coursework. Therefore, the researchers recommend that librarians focus on ways to provide research instruction to graduate students in formats that can be accessed or attended when needed, while at the same time focusing extra effort on marketing to key groups that have been shown to have lower confidence in various research skills (namely, master’s students and online students). Finally, the results of this study add to the case for libraries to partner with other units on campus in offering instruction for graduate students. Graduate students showed low confidence ratings in many areas that are not the traditional purview of library instruction, such as determining where to publish, securing grant funding, and data analysis. Although some universities may have units within the library that work in these areas, at Ohio University, as at many others, specialties in these areas reside in units across campus. It is important for libraries to use their central position to connect graduate students with specialists across campus. 994 College & Research Libraries November 2017 APPENDIX A. Focus Group and Interview Questions 1. Describe a recent research project that you completed for your graduate program. 2. What would you say were the biggest challenges in completing a research project? 3. Where did you learn how to do research? 4. Where do you go when you need help with a research project? 5. What kinds of skills do you think you need to develop to become a more effective or efficient graduate student or researcher? If there are any skills that you don’t think you need to learn yourself, what skills do you think are most important for a graduate student to be effective and efficient? 6. If you realized that there was a skill you needed to learn, such as [moderator: list something mentioned by the participants as a skill they would like to learn], where would you want to learn that skill? □ Possibilities: In class? Via an online workshop? Through a self-paced video tutorial? By text-based directions on a website? 7. What would be the best way for you to learn about opportunities to develop your research skills? □ Possibilities: via e-mail? A social networking site like Facebook or Twitter? On a website you visit frequently? Elsewhere? 8. What one thing do you think is most important for the library to offer to gradu- ate students? Research Needs and Learning Format Preferences 995 APPENDIX B. Survey Questionnaire Skills Questions Participants rated their confidence to complete the following tasks as poor, fair, good, very good or excellent (on a scale from 1-5) Writing & Sharing • Compile a literature review • Writing up the results of your research • Determine where to publish your research • Write about previous research without plagiarizing • Develop a poster for presentation • Prepare a conference proposal Research Sources • Cite sources in the appropriate style • Identify previously published research on your topic • Access the full text of previously published research • Keep track of or organize citations of previously published research • Access books I need for my research • Decide when to end the search for previously published research Funding • Identify Sources of grant funding for your research • Successfully apply for grant funding for your research Data • Analyze qualitative data • Analyze quantitative data • Collect data • Store or manage data Technology Questions Is there any software or computer program you will need to learn in order to complete research in your program, such as software for data analysis or visualization? Yes/No/Not Sure If yes: Please list the programs you will need to learn how to use to complete research in your program. Citation Management Questions Do you use a citation management tool to keep track of research sources? Examples of these tools include RefWorks, Zotero, Mendeley and EndNote. Yes/No/Not Sure If yes: Which tool(s) do you use? 996 College & Research Libraries November 2017 Learn Format Questions If you found yourself in a situation where you needed to learn a new skill in order to successfully complete a research project, how would you prefer to learn that skill? Please rank your preferred learning options from 1 to 5 with 1 being preferred in most cases and 5 being preferred least often. • A live online workshop • A video I could watch when needed • A website with images and text that I could consult as needed • An in-person workshop • A presentation in your regular classes Are there any preferred methods of learning a new skill that are not on this list? Yes/No If yes: What other method of learning a new skill would you prefer? Experience Questions Participants answered these questions on a five-point Likert scale from strongly disagree to strongly agree, with the exception of the question about not using particular resources. Please indicate whether you agree or disagree with the following statements about your experience using the Ohio University Libraries’ resources and services. • Ohio University Libraries staff are available to answer my questions. • Ohio University Libraries staff are able to answer my questions. • I have access to the information resources I need through the Ohio University Libraries • I feel well-informed about the resources and services available to me through the Ohio University Libraries Please indicate whether you agree or disagree with the following statements about your experience as a graduate student. • My program and coursework helped me learn the skills I need to become an effective and efficient researcher • My undergraduate program prepared me for graduate- level expectations of student research skills Have you ever decided not to use a particular information resource (such as a journal article or dissertation) because you were not able to access the document at the time you needed it? Yes/No Open-Ended Is there anything else you would like to let us know about learning research skills? Research Needs and Learning Format Preferences 997 Notes 1. Rachel Fleming-May and Lisa Yuro, “From Student to Scholar: The Academic Library and Social Sciences PhD Students’ Transformation,” portal: Libraries and the Academy 9 (2009): 209, doi:10.1353/pla.0.0040. 2. Hannah Gascho Rempel and Jeanne Davidson, “Providing Information Literacy Instruc- tion to Graduate Students through Literature Review Workshops,” Issues in Science and Technology Librarianship 53 (2008): 2, doi:10.5062/F44X55RG. 3. Chris M. Golde and Timothy M. Dore, “At Cross Purposes: What the Experiences of Today’s Doctoral Students Reveal about Doctoral Education” (Philadelphia, Penn.: Pew Charitable Trusts, 2001): 17, available online at http://eric.ed.gov/?id=ED450628 [accessed 16 May 2016]. 4. Rempel and Davidson, “Providing Information Literacy Instruction to Graduate Students,” 2; Fleming-May and Yuro, “From Student to Scholar,” 209–10. 5. Golde and Dore, “At Cross Purposes,” 17; David N. Boote and Penny Beile, “Scholars before Researchers: On the Centrality of the Dissertation Literature Review in Research Preparation,” Educational Researcher 34, no. 6 (2005): 9–10, doi:10.3102/0013189X034006003; Fleming-May and Yuro, “From Student to Scholar,” 210; Amalia Monroe-Gulick and Julie Petr, “Incoming Gradu- ate Students in the Social Sciences: How Much Do They Really Know about Library Research?” portal: Libraries and the Academy 12 (2012): 327, doi:10.1353/pla.2012.0032. 6. David Gibbs et al., “Assessing the Research Needs of Graduate Students at Georgetown University,” Journal of Academic Librarianship 38, no. 5 (2012): 272, doi:10.1016/j.acalib.2012.07.002. 7. Results from this study were previously published in Hilary Bussell, Jessica Hagman, and Christopher S. Guder, “Mixin’ It Up: Using a Mixed Methods Approach to Understand Gradu- ate Research Needs” (paper presented at the ACRL Conference, Portland, Oregon, March 25–28, 2015). 8. Ohio University Office of Institutional Research, Final Spring Enrollment Headcount, 2010–2016, available online at https://www.ohio.edu/instres/student/QuartEnroll/QuartEnroll- SPRING.html [accessed 27 June 2016]. 9. Eldridge Smith, “Library Services to the Graduate Community: Introduction,” College & Research Libraries 37, no. 3 (1976): 246, doi:10.5860/crl_37_03_246. 10. See Monica Vezzosi, “Doctoral Students’ Information Behaviour: An Exploratory Study at the University of Parma (Italy),” New Library World 110 (2009), doi:10.1108/03074800910928595; Barbara Blummer, Sara Lohnes Watulak, and Jeffrey Kenton, “The Research Experience for Education Graduate Students: A Phenomenographic Study,” Internet Reference Services Quarterly 17 (2013), doi:10.1080/10875301.2012.747462; Tanya Cothran, “Google Scholar Acceptance and Use among Graduate Students: A Quantitative Study,” Library and Information Science Research 33 (2011), doi:10.1016/j.lisr.2011.02.001; Hamid R. Jamali and Saeid Asadi, “Google and the Scholar: The Role of Google in Scientists’ Information-Seeking Behaviour,” Online Information Review 34 (2010), doi:10.1108/14684521011036990. 11. Gibbs et al., “Assessing the Research Needs,” 273; Eloise M. Bellard, “Information Literacy Needs of Nontraditional Graduate Students in Social Work,” Research Strategies 20 (2005): 500, doi:10.1016/j.resstr.2006.12.019; Hannah Gascho Rempel, “A Longitudinal Assessment of Graduate Student Research Behavior and the Impact of Attending a Library Literature Review Workshop,” College & Research Libraries 71 (2010): 542, doi:10.5860/crl-79. 12. Vanessa J. Earp, “Information Source Preferences of Education Graduate Students,” Be- havioral & Social Sciences Librarian 27 (2008): 85, doi:10.1080/01639260802194974. 13. Xi Niu et al., “National Study of Information Seeking Behavior of Academic Researchers in the United States,” Journal of the American Society for Information Science and Technology 61 (2010): 876, doi:10.1002/asi.21307. 14. Gabriela Castro Gessner et al., “Supporting Humanities Doctoral Student Success: A Col- laborative Project between Cornell University Library and Columbia University Libraries” (2011): 11, available online at https://www.2cul.org/activities/intervention [accessed 23 February 2013]. 15. Anne Melles and Kathryn Unsworth, “Examining the Reference Management Practices of Humanities and Social Science Postgraduate Students and Academics,” Australian Academic & Research Libraries 46, no. 4 (2015): 259, doi:10.1080/00048623.2015.1104790. 16. Barbara Alvarez, Jennifer L. Bonnet, and Meredith Kahn, “Publish, Not Perish: Supporting Graduate Students as Aspiring Authors,” Journal of Librarianship and Scholarly Communication 2 (2014): 4. doi:10.7710/2162-3309.1141. 17. Carrie Forbes, Gina Schlessleman-Tarango, and Peggy Keeran, “Expanding Support for Graduate Students: Library Workshops on Research Funding Opportunities,” College & Research Libraries (forthcoming), available online at http://crl.acrl.org/content/early/2016/03/22/crl16-854. http://10.1353/pla https://doi.org/10.5062/F44X55RG https://eric.ed.gov/?id=ED450628 https://doi.org/10.3102/0013189X034006003 http://10.1353/pla http://10.1016/j.acalib https://www.ohio.edu/instres/student/QuartEnroll/QuartEnrollSPRING.html https://www.ohio.edu/instres/student/QuartEnroll/QuartEnrollSPRING.html http://10.5860/crl https://doi.org/10.1108/03074800910928595 https://doi.org/10.1080/10875301.2012.747462 https://doi.org/10.1016/j.lisr.2011.02.001 https://doi.org/10.1108/14684521011036990 https://doi.org/10.1016/j.resstr.2006.12.019 http://10.5860/crl https://doi.org/10.1080/01639260802194974 http://10.1002/asi https://www.2cul.org/activities/intervention https://doi.org/10.1080/00048623.2015.1104790 https://doi.org/10.7710/2162-3309.1141 http://crl.acrl.org/content/early/2016/03/22/crl16-854.full.pdf 998 College & Research Libraries November 2017 full.pdf+html [accessed 21 June 2016]. 18. Lise Doucette and Bruce Fyfe, “Drowning in Research Data: Addressing Data Management and Literacy of Graduate Students” (paper presented at the ACRL Conference, Indianapolis, Indiana, April 10–13, 2013): 167, available online at www.ala.org/acrl/sites/ala.org.acrl/files/ content/conferences/confsandpreconfs/2013/papers/DoucetteFyfe_DroDrown.pdf [accessed 18 April 2013]. 19. Bonnie L. Fong et al., “Assessing and Serving the Workshop Needs of Graduate Students,” Journal of Academic Librarianship (in press): 3, doi:10.1016/j.acalib.2016.06.003. 20. Merinda Kaye Hensley, “The Savvy Researcher: Teaching Information Management Skills to Graduate Students” (paper presented at the LOEX Conference, Albuquerque, New Mexico, April 30–May 2, 2009). 21. Lori Critz et al., “Development of the Graduate Library User Education Series,” Reference Services Review 40 (2012), doi:10.1108/00907321211277341. 22. Johanna Tuñón and Laura Ramirez, “ABD or EdD? A Model of Library Training for Distance Doctoral Students,” Journal of Library Administration 50 (2010): 989–96, doi:10.1080/01930826.2010 .489004. 23. See Penny M. Beile and David N. Boote, “Library Instruction and Graduate Professional Development: Exploring the Effect of Learning Environments on Self-Efficacy and Learning Outcomes,” Alberta Journal of Educational Research 48 (2002): 364–67, available online at http:// eprints.rclis.org/16961/ [accessed 1 April 2016]; Swapna Kumar and Mary E. Edwards, “Informa- tion Literacy Skills and Embedded Librarianship in an Online Graduate Programme,” Journal of Information Literacy 7 (2013): 3–17, doi:10.11645/7.1.1722. 24. Andrea Baruzzi and Theresa Calcagno, “Academic Librarians and Graduate Students: An Exploratory Study,” portal: Libraries and the Academy 15 (2015): 401, doi:10.1353/pla.2015.0034. 25. Lizah Ismail, “Closing the Gap,” Reference & User Services Quarterly 53, no. 2 (2013): 167, doi:10.5860/rusq.53n2.164. 26. Baruzzi and Calcagno, “Academic Libraries and Graduate Students,” 401. 27. Fleming-May and Yuro, “From Student to Scholar,” 211–12. 28. Gibbs et al., “Assessing the Research Needs,” 270; Rempel, “A Longitudinal Assessment,” 541. 29. Jessica Kayongo and Clarence Helm, “Graduate Students and the Library: A Survey of Research Practices and Library Use at the University of Notre Dame,” Reference & User Services Quarterly 49, no. 4 (2009): 348, doi:10.5860/rusq.49n4.341. 30. Ismail, “Closing the Gap,” 167. 31. Andrew Johnson, Rebecca Kuglitsch, and Megan Bresnahan, “Using Participatory and Service Design to Identify Emerging Needs and Perceptions of Library Services among Science and Engineering Researchers Based at a Satellite Campus,” Issues in Science and Technology Li- brarianship 81 (2015), doi:10.5062/F4H99366. 32. Kirsten Kinsley et al., “Graduate Conversations: Assessing the Space Needs of Graduate Students,” College & Research Libraries 75 (2015): 761, doi:10.5860/crl.76.6.756. 33. See Kayongo and Helm, “Graduate Students and the Library”; Castro Gessner et al., “Supporting Humanities Doctoral Student Success”; Hannah Gascho Rempel, Uta Hussong- Christian, and Margaret Mellinger, “Graduate Student Space and Service Needs: A Recommen- dation for a Cross-Campus Solution,” Journal of Academic Librarianship 37 (2011), doi:10.1016/j. acalib.2011.07.004. 34. John W. Creswell et al., “Advanced Mixed Methods Research Design,” in Handbook of Mixed Methods in Social & Behavioral Research, eds. Abbas Tashakkori and Charles Teddlie (Thousand Oaks, Calif.: Sage Publications, 2003), 225–28. 35. Greg Guest, Kathleen M. MacQueen, and Emily E. Namey, Applied Thematic Analysis (Los Angeles, Calif.: Sage Publications, 2012), 10–13. 36. Valerie M. Sue and Lois A. Ritter, Conducting Online Surveys (Los Angeles, Calif.: SAGE, 2012), 35. 37. Linda J. Sax, Shannon K. Gilmartin, and Alyssa N. Bryant, “Assessing Response Rates and Nonresponse Bias in Web and Paper Surveys,” Research in Higher Education 44 (2003): 411, doi:10.1023/A:1024232915870. 38. The study population includes all of the students enrolled at the time that the data were received from the Registrar’s office in March 2014. Final enrollment for the Spring 2014 semester was 4,703, as reported by the university’s Office of Institutional Research. http://crl.acrl.org/content/early/2016/03/22/crl16-854.full.pdf http://www.ala.org/acrl/sites/ala.org.acrl/files/content/conferences/confsandpreconfs/2013/papers/DoucetteFyfe_DroDrown.pdf http://www.ala.org/acrl/sites/ala.org.acrl/files/content/conferences/confsandpreconfs/2013/papers/DoucetteFyfe_DroDrown.pdf http://10.1016/j.acalib https://doi.org/10.1108/00907321211277341 https://doi.org/10.1080/01930826.2010.489004 https://doi.org/10.1080/01930826.2010.489004 http://eprints.rclis.org/16961 http://eprints.rclis.org/16961 https://doi.org/10.11645/7.1.1722 http://10.1353/pla https://doi.org/10.5860/rusq.53n2.164 https://doi.org/10.5860/rusq.49n4.341 https://doi.org/10.5062/F4H99366 http://10.5860/crl http://10.1016/j.acalib http://10.1016/j.acalib https://doi.org/10.1023/A:1024232915870 _GoBack