ISSN: 2474-3542 Journal homepage: http://journal.calaijol.org More Than a Number: Measuring Value and ROI in a Library Microgrant Program Melody Condron, Wenli Gao, and Keith Komos Abstract: The objective of this study was to identify whether the internal microgrant program at the University of Houston Libraries provided a suitable return on investment. This study identified three criteria for assessing return on investment from library microgrants: engagement, continuation, and professional impact. The results indicated that compared to costs for other library outreach programs and events, the amount for the microgrant program is not excessive. However, it resulted in significant participation and impact, with many continuing positive outcomes. Internal microgrant programs offered a positive return on investment for the University of Houston Libraries during the period reviewed. This study will help with the decision-making process of other libraries looking to fund similar programs. It also has implications for assessment in libraries with similar existing programs. To cite this article: Condron, M., Gao, W., & Komos, K. (2018). More Than a Number: Measuring Value and ROI in a Library Microgrant Program. International Journal of Librarianship, 3(1), 94-103. doi: https://doi.org/10.23974/ijol.2018.vol3.1.57 To submit your article to this journal: Go to http://ojs.calaijol.org/index.php/ijol/about/submissions INTERNATIONAL JOURNAL OF LIBRARIANSHIP, 3(1), 94-103 ISSN:2474-3542 More Than a Number: Measuring Value and ROI in a Library Microgrant Program Melody Condron, Wenli Gao, and Keith Komos University of Houston, Houston, TX, USA ABSTRACT The objective of this study was to identify whether the internal microgrant program at the University of Houston Libraries provided a suitable return on investment. This study identified three criteria for assessing return on investment from library microgrants: engagement, continuation, and professional impact. The results indicated that compared to costs for other library outreach programs and events, the amount for the microgrant program is not excessive. However, it resulted in significant participation and impact, with many continuing positive outcomes. Internal microgrant programs offered a positive return on investment for the University of Houston Libraries during the period reviewed. This study will help with the decision-making process of other libraries looking to fund similar programs. It also has implications for assessment in libraries with similar existing programs. Keywords: microgrant, return on investment, library assessment, program impact INTRODUCTION The University of Houston Libraries first implemented the internal microgrant program in 2007 as a tool to highlight the libraries’ strategic directions. The microgrant program was designed to foster the creation of new and innovative initiatives in support of University of Houston Libraries’ Strategic Directions and the University’s Tier One initiatives. At the outset of the program, successful microgrant applications were awarded up to $2000, but most projects were able to succeed for less than that amount. Since that time the program has grown and changed, and requirements and guidelines have shifted as budgets and expectations have changed. So, too, has the library administration changed; a new Dean of Libraries was hired in August 2015. University of Houston Libraries also has a new 2017-2021 strategic plan. As the needs, leadership, and strategic plans for the Libraries shifted, it became important to identify whether the microgrant program should continue to receive support, both in terms of staffing resources and financing. Several questions emerge from this review: does the microgrant program still encourage new endeavors that fulfill the libraries’ goals and plans? Is the funding adequate, sub-par, or excessive based on the need? Can the impact of microgrant programs be quantified and, if so, how? Condron, Gao, & Komos / International Journal of Librarianship, 3(1) 95 LITERATURE REVIEW Return on investment (ROI) is a cost-benefit analysis that “seeks to estimate and compare costs and benefits of an undertaking” (Matthews, 2011, p.1). In a strict sense, ROI is a quantitative measure that calculates the ratio of value returned for money invested. However, when examining the value of library use, it also includes indirect benefits such as more productive faculty, more successful students, and more satisfied library users. The history of libraries using ROI analysis is long, with public libraries in particular as early adopters of using ROI to demonstrate value. Imholz and Arns (2007) reviewed 17 public library valuation studies, including cost benefit analysis, economic impact, and social returns on investment to show their value in their communities. Aabo (2009) conducted a meta-analysis of previous library ROI studies. Findings from his study showed 84.2% (32 of 38) studies were from public libraries. Besides public libraries, special libraries evaluated their impact with ROI. Jemison et al. (2009) reported a work group from Department of Veterans Affairs Library Network created tools to measure ROI. This group divided these tools into three components: an ROI analysis tool, a library scorecard, and a management support report. Later on, the Health Sciences and Human Services Library of the University of Maryland used an automated tool to calculate ROI and cost benefit figures for their annual book and journal expenditures (Bodycomb & Del Baglivo, 2012). While public and special libraries were early adopters of ROI studies, academic libraries also began to respond to the need to use ROI to measure the value of academic libraries. Some studies measured the contribution of library collections to university research and teaching. Pan, Wiersma, and Fong (2011) used citation analysis and faculty interviews to calculate ROI. Their study demonstrated that “the library collection development efforts directly support the research and teaching outcomes of academic faculty” (Pan, Wiersma, & Fong, 2011, p.464). Tenopir (2010) presented a case study at University of Illinois at Urbana-Champaign where researchers used ROI to measure the value of e-journals in relation to grants income. Faculty members reported that “e- journals have transformed the way they do research, including making them more productive and competitive” (Tenopir, 2010, p.39). Other studies used the contingent valuation method to measure users’ willingness to pay time and money for the service they received. Harless and Allen (1999) surveyed 382 students and faculty at Virginia Commonwealth University. The results indicated that students were willing to pay $5.59 per semester for reference desk service and instructional faculty were willing to pay $45.76 per year to maintain current library hours. The ratio of ROI in this study was 3.5 to 1. In Kingma and McClure (2015)’s study, the economic and environmental value of Syracuse University library showed an ROI of $4.19 returned to the university for every $1 spent each year. The investment of library expenditure and professional staff also had a positive effect on student retention. Mezick (2007) employed statistical measures to analyze the relationship of both library expenditures and number of professional library staff to student persistence. The result of this study revealed that each category of expenditure correlated significantly with student retention within every Carnegie Classification. The strongest relationships existed between total library expenditures and serial expenditures at baccalaureate colleges. Despite the fact that ROI studies at public, special, and academic libraries aim to measure the value of library work and services, a gap exists in research to measure ROI for library internal grant programs. Barry (2005) describes the elements of a library-funded grant program to support faculty projects, how to find money for such a program, when and how to call for proposals, and Condron, Gao, & Komos / International Journal of Librarianship, 3(1) 96 how to evaluate potential projects. However, he did not mention how to measure the return for such grant programs. Herring (2014) describes the Ohio State University Libraries course enhancement grant program and how it impacted the collaboration between librarians and instructors. The focus of the evaluation of the program was on students’ assessment of course objectives and outcomes, rather than on ROI for the grant program. Throughout the literature, there are many studies aiming to evaluate library programs and services, both in public and academic libraries. Rhoden and Crumbley (2013) published a study to examine the outreach department of Calvert Library in southern Maryland for its effort to reach and serve customers who do not have access to library services. The results indicated the outreach services were worth the cost. Miller (2014) surveyed the faculty and assessed the liaison program at Rollins College. The purpose of this study was not to share the results of liaison assessment, rather it was designed to share the assessment methods that can be useful for other libraries. In another study about library outreach events during exam weeks, survey results showed libraries use feedback from users in person and on social media, number of attendees at each event, questionnaires and surveys, and number of users in the library as ways to assess outreach events (Meyers-Martin & Borchard , 2015). None of these studies directly incorporate monetary investment into account in evaluating library programs and activities. The goal of this paper is to help fill the gap and measure ROI for library internal grant-funded projects. AIMS The aim of this study is to identify whether the internal microgrant program at the University of Houston Libraries provides a suitable ROI, based on funding provided. Despite the perceived success of individual microgrant projects, no significant assessment of the overall program was previously conducted. Without such data it is difficult to claim that the program is worthwhile. Externally, interested librarians and their administrators want to know the details of the program, including how successful the program has been. Was it, and is it still viable and worth the effort? Internally, administrators and librarians at University of Houston want to know whether the program should continue to be funded at the same level. As such, the purpose of assessing ROI for the library’s microgrant program is two-fold: first, to identify the relevance and sustainability of the program and its funding, and second, to offer a model for identifying whether similar programs are worthwhile for other libraries to pursue. This project aims to answer the following research questions: • What defines success in a library microgrant program? • How can an individual project be assessed in terms of contribution to students/patrons, library staff members, and the library profession? • How many funded library microgrants produced sustainable results past original time of project? • Do library microgrant-funded projects contribute to a library’s success and goals in the long term? METHOD Condron, Gao, & Komos / International Journal of Librarianship, 3(1) 97 This study reviewed available data from the 39 funded microgrant projects approved at the University of Houston Libraries between 2007 and 2015. ROI criteria related to microgrant funding is not clearly defined by previous research. As such, the project team identified three primary criteria for assessment: engagement, continuation, and professional impact. In looking at levels of engagement, this review identified how many individuals took part in each project at four different levels: initial planning team, collaborators, volunteers, and participants. Continuation tracked whether projects resulted in a continuing impact, including the repetition of the event or a permanent/semi-permanent change in workflow within the libraries. Finally, professional impact gauged the contributions of the microgrant program to the library and the library profession by identifying presentations, articles, and awards generated by the University of Houston Libraries microgrants. Engagement, continuation, and professional impact were deemed by the research team to be highly relevant to the expressed goals of the microgrant program. In addition, all three are directly related to the University of Houston Libraries’ Strategic Plans for this period. Missing data was an issue during this study, as ROI had not been part of early consideration for the microgrant program. Microgrant teams were required to submit a final report, but some final reports were not retained and could not be found. Where possible, microgrant collaborators and participants were asked for details or participation estimates when data was insufficient. Of the 39 approved projects, 8 were not included in the review because projects were not implemented fully. Of the remaining 31 projects, all had sufficient information available for assessment. Engagement was the primary means of assessing participation in each project. Engagement was considered at four levels: the initial planning team, collaborators, volunteers, and participants. The initial planning team consisted of those who wrote and submitted the microgrant. These individuals usually contributed most to each project, experiencing the highest level of engagement. Collaborators were those who participated in the planning or creation of the project, though they were not the initiators and may not have participated in the project’s entirety. Often these collaborators did not work in the library and common collaborators included members of the faculty or individuals from area organizations or community groups. Collaborators had a relatively high engagement level in the project. Volunteers were those asked to participate in some working aspect to the project, often at an event related to the project. These individuals had a moderate level of engagement, as they had a specific role to play. Participant engagement included all others who either participated in or attended events or related project engagement. When considering engagement level this study looked at both the broad, total engagement numbers as well as the breakdown of engagement by level; implications of higher levels of engagement may indicate higher impact, including the forging of faculty and community relationships and student interest. In addition, engagement was broken down by year to identify trends in participation. During this assessment, the study identified one project with particularly high participation. The project, Finals Mania, a large event that served free food to students in coordination with other finals week activities, has an extreme high number of participants. It is an outlier, thus is broken out in some of the per-project reporting for engagement only. Continuation was assessed by identifying continuing impact, including whether an event was repeated, or a process resulted in a permanent or semi-permanent change in library workflow. Projects were counted and separated into those that did and did not have any continuing aspect. Most information on continuation was collected by members of the microgrant project who were still working at the library, as final reports for the project would not contain continuation Condron, Gao, & Komos / International Journal of Librarianship, 3(1) 98 information. In some cases, librarians who had moved to new jobs, or those in the same project area were able to provide continuation information. Projects that continued beyond the initial scope were included even if they were no longer active at the time of the study. Professional impact was gauged by identifying “first level” professional outcomes: papers, awards, and presentations that resulted from or focused on the microgrant project or program. The impact of these end products resulted in a higher reputation for the University of Houston Libraries. Participants from previous microgrants were asked to self-report any awards, presentations and programs. In addition, supervisors and others who track professional involvement were contacted for information. Papers, awards, and presentations were all considered equally in terms of impact. RESULTS From the years 2007 to 2015, 31 microgrants were awarded, completed, and offered viable data that could be used for study. The number of microgrants awarded had dropped since 2009. As a result, there was a review and change of application process in 2012, which led to no microgrant for that year. Figure 1 shows the number of microgrants awarded by year. Figure 1. Number of microgrants awarded by year. Total cost for the microgrant program during this period was $43,011, with an average budget of $5,376 per year, and an average of $1,387 per project. Total engagement from 2007 to 2015 included 4,617 individuals. On average, microgrant projects experienced total engagement of 149 individuals per project. However, that number is more accurately reported as 100 individuals per project when eliminating Finals Mania, which had an unusually high total participation count of 1,612. Total participation was 4,161, with an average of 134.22, or 88.7 without the high attendance rate from Finals Mania. On average, projects included 4.45 team 0 1 2 3 4 5 6 7 8 9 2007 2008 2009 2010 2011 2012 2013 2014 2015 Condron, Gao, & Komos / International Journal of Librarianship, 3(1) 99 members, 3.74 collaborators, and 6.51 volunteers. Figure 2 is the number of planning team, collaborators and volunteers by year, and figure 3 is the number of participants by year. Figure 2. Number of planning team, collaborators and volunteers by year. Figure 3. Number of participants by year. When assessing continuation, in all, 42% of the 31 microgrant projects were continued in some manner: either through adoption of a new process or workflow or as a continuing event or 0 20 40 60 80 100 120 2007 2008 2009 2010 2011 2012 2013 2014 2015 Planning Team Collaborators Volunteers 79 2017 820 207 100 0 125 243 570 0 500 1000 1500 2000 2500 2007 2008 2009 2010 2011 2012 2013 2014 2015 Condron, Gao, & Komos / International Journal of Librarianship, 3(1) 100 program. This percentage is reasonably high, given that some projects cannot be gauged on continuation; for example, the Foley’s Scrapbook microgrant project focused on improving the condition and accessibility of specific Special Collections materials. Once completed, this project did have a lasting effect but was not “continued” as other projects were. Microgrant projects resulted in 16 presentations and articles. Projects coming from the Microgrant funding were presented locally and nationally at the Texas Library Association Conferences, the Internet Librarian Conference, Library Assessment Conference, and the Association of College and Research Libraries Conference. Articles were published in titles such as Journal of Academic Librarianship, Journal of Library Innovation, and Library Journal. Additionally, microgrant support resulted in three awards, two from the Texas Library Association: the Branding Iron award for Collateral Materials, and Best of Show: University of Houston Libraries for Broadcast Advertising. A recent project, Walk and Learn for Wellness, was recognized by University of Houston Staff Council for Cougar Cudos. DISCUSSION Results of this study imply a positive impact and ROI for microgrants projects at University of Houston Libraries. This positive impact comes from the defined success measures of engagement, continuation, and professional impact, as these criteria fit most closely with the outlined goals and strategic plans for University of Houston Libraries. This is in line with similar efforts to define ROI in other library-related research where benefit analysis, economic impact, social returns, and value to research and teaching were considered. While not relying on ROI in the strictest sense, these success measures nevertheless provide a clearer picture of program value within the library. Compared to non-microgrant events in the library funded by other means, microgrant projects had a high level of engagement to cost ratio. Event planners at University of Houston Libraries estimated that an event for 100 participants (the adjusted average engagement total for microgrant programs reviewed) commonly costs between $2,200-$4,500, or $22-$45 per person. The average cost of each microgrant program was $1,387, or $13.87 per person. While this comparison initially indicates value, this number obviously depends on the amenities and needs of each program, which are difficult to compare with other events out of context. In addition, many microgrant programs are not events but programs or initiatives for which there is no clear price point comparison. The cost comparison per project also does not include the cost of staff time. There would likely be value in continued research into the ROI of internal grant programs in relation to the staff time and volunteer hours spent on grant versus non-grant programs. Because this time is currently not tracked for either type of program at University of Houston Libraries it could not be assessed during this study. Further, the nature of the microgrant program in funding innovation and new ideas is difficult to quantify. Some projects could not be proven as worthwhile without testing or trial. They are unlikely to have moved forward without microgrant funding. It is difficult to quantify if projects funded through the microgrant program offer a significant return in the way of innovation within the library. As with the research by Harless and Allen (1999), which assessed ROI by whether people would spend time or money on the activity, the high level of engagement Condron, Gao, & Komos / International Journal of Librarianship, 3(1) 101 (time) may be a positive indication that engagement at all levels shows a willingness to participate in new, untested projects and programs. There are several limitations to this study. First, because of the time span of the program, the library has changed administrators and strategic plans, thus selection criteria have changed over time. Other libraries embarking on a similar internal grant program would benefit from clearly outlining the goals at the outset so that participants are informed and assessment measures are clear. Due to programmatic change, some projects awarded in the first few years would not fit into criteria under the new guidelines, thus projects of a similar nature were directed to seek funding elsewhere. At the beginning of the program, the inclusion of an assessment tool was not required, leading to incomplete or missing data. Many project team reports do not report information such as engagement numbers. Researchers for this project made an effort to contact those microgrant team members, but because of personnel changes or the length of time which has passed, some of the numbers were estimates. Another challenge was a lack of existing literature or research relating to internal library grants. Throughout the literature, there were no existing guidelines for this type of assessment, so the researchers established our own criteria. The adopted criteria mainly consists of quantitative information, without considering qualitative feedbacks, which could be essential for measuring participants’ perceptions about the program or indirect impact. Finally, one project was identified during the study to be an outlier due to high participation. The inclusion of that program and the related engagement numbers created challenges for reporting and comparison. CONCLUSION Compared to costs for other library outreach programs, the amount allocated for the microgrant program is not excessive, yet it resulted in significant participation and impact, with many continuing positive outcomes. Available cost comparison estimates additionally indicate a value of engagement per person compared to financial cost. Moreover, the program is a great way to encourage innovation and provides library staff chances to try out new ideas, with over 40% of projects deemed valuable enough to continue past the initial microgrant plan. Because most of the programs involved members from multiple library departments, it also fosters collaboration internally. Externally, professional contributions from papers, presentations, and awards related to microgrant programs enhance the University of Houston Libraries’ national reputation while improving the professional standing of those involved in each project. Finally, microgrants have established relationships between other campus units and the community. As such, the internal microgrant program appears to offer a positive ROI at the University of Houston Libraries based on defined criteria. The research team recommends that other interested libraries with the budget to offer this kind of internal grant programs proceed mindfully. When creating such a program, set clear guidelines, expectations, and priorities for microgrant funding based on goals for the program. Compare the library microgrant budget with other programming and project budgets in the library to create a budget that balances the library’s financial situation and program needs. It is also suggested that libraries establish a tracking system and build in assessment for each project to not only help the implementation of the projects but also facilitate future assessment of the program. Condron, Gao, & Komos / International Journal of Librarianship, 3(1) 102 When assessing outcomes, clearly define impact or value. Besides the criteria used in this study, identify other indicators that will show the success of a program. Finally, if professional engagement is a priority for the library, encourage project teams to produce scholarly outcomes from the projects. References Aabo, S. (2009). Libraries and return on investment (ROI): A meta-analysis. New Library World, 110(7/8), 311-324. Barry, J. (2005). Supporting faculty/librarian collaboration in the development of Internet resources. Internet Reference Services Quarterly, 10(2), 101-105. Bodycomb, A., & Del Baglivo, M. (2012). Using an automated tool to calculate return on investment and cost benefit figures for resources: The health sciences and human services library experience. Journal of the Medical Library Association: JMLA, 100(2), 127-130. Harless, D. W., & Allen, F. R. (1999). Using the contingent valuation method to measure patron benefits of reference desk service in an academic library. College & Research Libraries, 60(1), 56-69. Herring, D. N. (2014). A purposeful collaboration: Using a library course enhancement grant program to enrich ESL instruction. The Reference Librarian, 55(2), 128-143. Imholz, S., & Arns, J. W. (2007). Worth their weight: An assessment of the evolving field of library evaluation. Public Library Quarterly, 26(3-4), 31-48. Jemison, K., Poletti, E., Schneider, J., Clark, N., & Stone, R. D. (2009). Measuring return on investment in VA libraries. Journal of Hospital Librarianship, 9(4), 379-390. Kingma, B., & McClure, K. (2015). Lib-value: Values, outcomes, and return on investment of academic libraries, phase III: ROI of the Syracuse University Library. College & Research Libraries, 76(1), 63-80. Matthews, J. R. (2010). What’s the return on ROI? The benefits and challenges of calculating your library’s return on investment. Library Leadership & Management, 25(1). Meyers-Martin, C., & Borchard, L. (2015). The finals stretch: Exams week library outreach surveyed. Reference Services Review, 43(4), 510-532. Mezick, E. M. (2007). Return on investment: Libraries and student retention. The Journal of Academic Librarianship, 33(5), 561-566. Miller, J. (2014). A method for evaluating library liaison activities in small academic libraries. Journal of Library Administration, 54(6), 483-500. Pan, D., Wiersma, G., & Fong, Y. (2011). Towards demonstrating value: Measuring the contributions of library collections to university research and teaching goals. Proceedings of the 2011 Association of College and Research Libraries Conference, Philadelphia, PA. pp. 459-465. Retrieved from http://www.ala.org/acrl/sites/ala.org.acrl/files/content/conferences/confsandpreconfs/nati onal/2011/papers/towards_demonstratin.pdf Rhoden, M., & Crumbley, M. (2013). An evaluation of current outreach services at Calvert library and its future outlook. Qualitative and Quantitative Methods in Libraries, 4, 379- 386. Tenopir, C. (2010). Measuring the value of the academic library: Return on investment and other value measures. The Serials Librarian, 58(1-4), 39-48. Condron, Gao, & Komos / International Journal of Librarianship, 3(1) 103 About the authors Melody Condron is the resource description and management coordinator at University of Houston. Her research interests include information organization theory, advances in next-generation library systems, personal digital archiving, and cooperative metadata. Wenli Gao is the data services librarian at University of Houston. Her research interests include data visualization, digital scholarship, and data services. Keith Komos is the technical lead at University of Houston.