C&RL News November 2018 566 Academic library assessment has grown as a field over the last 20 years. The pres- sures of increased competition over scarce resources and rapid technological changes have put pressures on academic libraries as they have on their parent institutions. In the midst of all the pressures and transformations lies a strong desire to be user-focused and responsive to the changing needs of faculty and students. Academic library assessment, as it is called in the United States, or performance measurement, as it is typically called in the United Kingdom, has flourished as an area of research that informs practice or as an area where practice engages systematic research methods for continuous improvement. Aca- demic library assessment, therefore, is tightly linked to university and higher education transformations, performance, and effective- ness across the globe. Academic libraries advance the disciplines as well as serve the nexus of where disciplinary perspectives come together, where an engineer can read about policies and laws and an accountant can understand the political environment. How do we know that academic libraries are truly contributing to both economic de- velopment and informed citizenry outcomes? By focusing on the user.1 The development of library assessment is an indicator of the robustness of the academic library focus on the quality of their services delivered to the user and the user-centered approaches that have dominated in the recent years with the development of User Experi- ence and Assessment (UXA) or Assessment and User Experience (AUX) programs. How do we know what we know about academic library assessment? How well developed are libraries across the world? To be able to answer this accurately we would have needed to be knowledge- able about languages and cultures across the world, but like every researcher, I profess certain limitations here in that our analysis below has a certain bias reflecting the English-speaking world through the re- sources we are discussing below. OCLC is offering a website on global sta- tistics2 that is a good start for someone who would like to learn how many libraries, how many librarians, how many volumes, how much expended, and how many registered users we have in libraries across the globe. The site breaks down the information for academic libraries, national libraries, public libraries, school libraries, and special librar- ies. And the data are compiled from a variety of data sources available at the country level. Martha Kyrillidou Academic library assessment Barriers and enablers for global development and implementation Martha Kyrillidou is library consultant at QualityMetrics, research associate at the School of Information Sciences at the University of Illinois at Urbana-Champaign and chair of the Z39.7 Standard, email: martha@ qualitymetricsllc.com. Contact series editors Clara M. Chu, email: cmchu@illinois.edu, and Barbara J. Ford, email: bjford@illinois.edu, with article ideas. © Martha Kyrillidou international insights mailto:martha%40qualitymetricsllc.com?subject= mailto:martha%40qualitymetricsllc.com?subject= mailto:cmchu%40illinois.edu?subject= mailto:bjford%40illinois.edu?subject= November 2018 567 C&RL News We wanted to place academic library statis- tics in the context of other developments in libraries, so the table below presents select data for China, the United States, and South Africa for academic and total number of li- braries. The disparities among regions of the world as viewed from these representative countries are vast. For China and the United States, about a third of the library economy is comprised from the resources represented by academic libraries (see Table 1). Barriers to global benchmarking: The LibQUAL+ example We need to increase our efforts for opening opportunities across the globe to counteract the forces that are creating systemic increas- ing inequalities, a trend not easy to shift. Any comparison of academic libraries at the global level needs to be contextualized with the socio-economic forces that are shaping the performance of organizations, research and development, and well-being of the us- ers and citizens of each country. For one thing a wariness to participate in global benchmarking efforts of academic library service quality may arise once a local institution perceives that it may not be on a par with other more advanced countries and environments. I observed such behavior with the widely adopted international pro- tocol for measuring library services quality, LibQUAL+. Rooted in the tradition of the services marketing field, LibQUAL+ started as a grant-funded project3 formally in 2000. With widespread deployment in the United States over the first three years, our Canadian French-speaking colleagues and our U.K. colleagues worked with us to create French and British English versions. The protocol spread internationally, gradually with more language translations, including Hebrew, Arabic, Greek, and Swahili, among a total of more than 39 different language variations. LibQUAL+ did become a widely used way of measuring library service quality across the globe. However, not every library in the world that wants to deploy LibQUAL+ par- ticipates in the official service we created at the Association of Research Libraries (ARL). Many implementations have taken place on a research basis outside of ARL infrastructure, as can be seen from published articles. Fur- thermore, more systematic local implementa- tion and adaptations of LibQUAL+ often do appear through the peer review workflows. In conversations I have had with col- leagues in South America and in Greece, there are a couple of reasons why such implementations are not happening in the context of closer collaboration with the U.S.- based offering, managed by ARL. One reason is that what may be viewed as a modest participation fee ($3,200) in the U.S. is still a China Total Libraries COUNT USA COUNT South Africa COUNT Expenditures $152,000,440 Expenditures $21,759,280,324 Expenditures $484,705,816 Librarians 58,953 Librarians 157,685 Librarians 2,341 Libraries 109,673 Libraries 101,349 Libraries 11,406 Users 15,160,109 Users 231,262,659 Users N/A Volumes 1,063,356,687 Volumes 2,580,863,485 Volumes 52,756,234 Academic Librar- ies Expenditures $55,412,959 Academic Librar- ies Expenditures $7,008,113,939 Academic Librar- ies Expenditures $7,212,590 Librarians 30,894 Librarians 26,606 Librarians 648 Libraries 3,842 Libraries 3,793 Libraries 66 Users 4,272,000 Users 7,641,610 Users N/A Volumes 447,893,493 Volumes 1,099,951,212 Volumes 14,411,691 Table 1. Select data for China, United States, and South Africa from OCLC Global Statistics C&RL News November 2018 568 relatively expensive proposition for academic library assessment in many other countries like Peru, Chile, or even places like Greece. A second reason is the observation that scores in these other settings are lower and, therefore, create a hesitation for engaging at the same level as other institutions that are doing better. And, a third reason is a desire to customize the protocol and experiment and publish alternative and more useful versions of the protocol, locally. Though these reasons may not fully exhaust all the possibilities, they offer some insights on what some of the obstacles of widespread international col- laboration in academic library assessment are. Enablers of global benchmarking for academic library assessment The world of academic library assessment does come together with exchange of ideas and perspectives through a few venues that have consistently helped inform the debate around international developments in aca- demic library assessment over the last few years. For the U.S. environment, much in- sight can be gained from the proceedings of the biennial ACRL Conference, where the latest research is featured and, for the inter- national environment, the International Fed- eration of Library Associations and Institu- tions (IFLA) conference offers some insights on library assessment.4 More specifically, on library assessment issues at the international level, we have specialized events such as the following: • Library Assessment Conference (LAC) (biennial, started in 2006, United States)5 • International Conference on Perfor- mance Measurement in Libraries (biennial, started in 1995, United Kingdom, formerly known as Northumbria)6 • Evidence-Based Library and Informa- tion Practice Conference (EBLIP) (biennial, started in 2001, rotating among United States, United Kingdom, Australia, and Canada)7 • Qualitative and Quantitative Methods for Libraries (QQML) (annual, started in 2009, rotating among Greece, Ireland, Italy, Turkey, United Kingdom, and France, so far)8 These events have consistently produced research that is subsequently published in peer-reviewed journals in the profession. The themes of these conferences often have similarities focusing on issues of transforma- tion, user outcomes, user experience and institutional assessments, as well as some international comparisons. Some of these venues are more focused on methods (LAC) and others are more focused on context (QQML). Transformations related to library spaces and user experience have dominated the research trends presented. User experi- ence (UX) with its roots in usability studies also has its own separate conferences and publishing venues, as does digital library assessment. The latest call for papers from EBLIP10 makes evident the current marriage of UX and assessment.9 This trend translates into assessment programs that are multifaceted and expand beyond institutional assessment. UX is also expanding beyond usability into a more holistic interpretation of the user experience, often including the physical en- vironment and facilities aspects of a library’s operations. Thus, we are seeing academic libraries and librarians devoting more time on UX and assessment approaches and creating programs with professionals that specialize in approaches (quantitative or qualitative), tools (e.g., Tableau), development (Python), and often including project management services bundled with assessment, planning, market- ing, and outreach activities. But is this all happening in a similar way across the globe? Is there a need for more participation from certain parts of the world? These developments are uneven across the globe, and there is an increasing need for knowledge transfer across the globe. With libraries having multiple people with embed- ded assessment responsibilities or multiple people employed in UXA or AUX programs, the need to support people from less afflu- ent regions to attend some of these events is paramount. Among the four events highlighted, the representation of people from non-English November 2018 569 C&RL News speaking countries is more prevalent in the QQML conference, where one can interact with colleagues from countries in Africa, Asia, and Europe. The other three events tend to have a typical representation of 5% to 15% from regions outside the host countries. What can be improved? Academic library assessment can be im- proved at a personal, institutional, profes- sional, policy, and standards level. Exam- ples for improvements at each level include: Exchanges/Internships/Training: The per- sonal level Professionals hired in the new expanding roles of library assessment, digital develop- ments, project management, planning, mar- keting, and outreach are in high demand. It would be very useful if more opportuni- ties existed for these professionals to offer targeted internships (online or in-person, ,nationally and internationally) aimed at expanding the know-how about tools and methods. Beyond the conferences, targeted training in this area is frequently offered by organizations like the National Information Standards Organization (NISO).10 Cross-library type assessments: The institu- tional level UXA methods and tools are similar across types of libraries, and more dialogue is needed on how these approaches are im- plemented across different library types. Today’s high school student is tomorrow’s university student and next year’s faculty professor. Understanding our users and their information needs in the context of their life cycle is extremely important, much like development psychology tries to under- stand the development of human beings in the context of their life span. Integrated library services: The professional level Libraries have spent enormous amounts of energy developing integrated library systems but have failed in developing in- tegrated library services. Our licensing ap- proaches, which absorb larger and larger portions of our budget, are working against the concept of an integrated library service as each vendor tries to create a monopoly with specialized bells and whistles, includ- ing enhancements of the user interface that creates a unique and distinct identity of the product, but fails the user in being able to search seamlessly with a satisfactory user experience across products. Efforts like the development of FOLIO are trying to address some of these challenges, but further work is needed. Why do we keep licensing re- sources that are perpetuating the bubble ef- fect of privilege and exclusion for the price of a poor user experience? Privacy: The policy level As we deploy more systematic approaches to understanding our library users, we need to spend more time understanding the library privacy frameworks, policies, and proce- dures. We need to have in place in order for our organizations to deploy not only robust, but also ethical approaches to the develop- ment of new services and environments. We need to nurture thoughtful understanding of how we use data in libraries to improve ser- vices and protect privacy with nuanced ap- proaches for different contexts. Data dictionary: The standards level One of the longest-standing standards for li- braries and information services is the NISO Z39.7 standard entitled “Information Servic- es and Use: Metrics and Statistics for Librar- ies and Information Providers Data Diction- ary.” The work of this standard has given birth to multiple other standards, includ- ing COUNTER and SUSHI, and is currently undergoing another revision. It is one of a few venues where stakeholders from dif- ferent types of libraries and vendors come together to develop consensus on what is what. Though much more needs to happen beyond what’s what, this standard offers an important foundation for related interna- tional work at the ISO level. C&RL News November 2018 570 Conclusion In summary, in the future we would like to see integration of technology and library as- sessment solutions with deep understanding of the psychology of the user. Such environ- ments will allow us to address the needs for specialized populations, will be more invit- ing to everyone, and will allow us to bridge the increasing gap among the global divides in a sustainable way. Notes 1. Martha Kyrillidou, “From Input and Output Measures to Quality and Outcome Measures, or, from the User in the Life of the Library to the Library in the Life of the User,” Journal of Academic Librarianship 28 (1) (2002): 42-46. 2. OCLC Global Library Statistics, ac- cessed September 23, 2018, https://www. oclc.org/en/global-library-statistics.html. 3. LibQUAL+: Measuring Library Ser- vice Quality, accessed September 23, 2018, http://www.libqual.org/home. Support for LibQUAL+ was provided over the years through the following funding agencies: De- partment of Education Fund for the Improve- ment of Post-secondary Education, National Science Foundation, and Institute of Museum and Library Services. 4. For those who plan to attend IFLA in August 2019 in Athens, Greece, we recom- mend visiting the Stavros Niarchos Founda- tion Cultural Center, where the new head- quarters of the National Library of Greece are located, accessed September 23, 2018, https://www.snfcc.org/about/vision/the- national-library-of-greece/. If you’re planning to visit Thessaloniki, we recommend you visit the Municipal Public Library of Thessaloniki and the Central Library of the Aristotle Uni- versity. If you’re planning to visit Veria on your way to Vergina archeological site, we recommend you visit the award-winning Central Public Library of Veria. 5. Library Assessment Conference, accessed September 23, 2018, https://libraryassessment. org/. 6. International Conference on Performance Measurement in Libraries, accessed September 23, 2018, https://libraryperformance.org/. 7. Evidence-Based Library and Informa- tion Practice, accessed September 23, 2018, http://eblip9.org/ and https://library.usask. ca/ceblip/eblip/eblip-conferences1.php. 8. Qualitative and Quantitative Methods in Libraries, accessed September 23, 2018, http://qqml.org/. 9. Evidence-Based Library and Informa- tion Practice, accessed September 23, 2018, https://eblip10.org/CallforPapers/tabid/8101 /Default.aspx. 10. NISO Training Series: Assessment Practices and Metrics for the 21st Cen- tury, accessed September 23, 2018, https:// www.niso.org/events/2018/10/niso-training -series-assessment-practices-and-metrics -21st-century. NEW FROM ACRL PRESS Available in the ALA Store at http://www.alastore.ala.org Transforming Libraries to Serve Graduate Students edited by Crystal Renfro and Cheryl Stiles NEW! pubs ad_novnews18.indd 1 10/22/2018 4:48:39 PM https://www.oclc.org/en/global-library-statistics.html https://www.oclc.org/en/global-library-statistics.html http://www.libqual.org/home https://www.snfcc.org/about/vision/the-national-library-of-greece/ https://www.snfcc.org/about/vision/the-national-library-of-greece/ https://libraryassessment.org/ https://libraryassessment.org/ https://libraryperformance.org/ http://eblip9.org/ https://library.usask.ca/ceblip/eblip/eblip-conferences1.php https://library.usask.ca/ceblip/eblip/eblip-conferences1.php http://qqml.org/ https://eblip10.org/CallforPapers/tabid/8101/Default.aspx https://eblip10.org/CallforPapers/tabid/8101/Default.aspx https://www.niso.org/events/2018/10/niso-training-series-assessment-practices-and-metrics-21st-century https://www.niso.org/events/2018/10/niso-training-series-assessment-practices-and-metrics-21st-century https://www.niso.org/events/2018/10/niso-training-series-assessment-practices-and-metrics-21st-century https://www.niso.org/events/2018/10/niso-training-series-assessment-practices-and-metrics-21st-century