RESEARCH Hypothesis Vol. 33 No.1 Fall/Winter 2021 Piloting a diversity, equity, and inclusion needs assessment to explore patron perceptions at a university health science library Jane Morgan-Daniel, MLIS, MA, AHIP, Corresponding Author Community Engagement and Health Literacy Liaison Librarian Assistant University Librarian George A. Smathers Libraries University of Florida Health Science Center Gainesville, Florida https://orcid.org/0000-0003-4047-8669 morgandanie.jane@ufl.edu Lauren E. Adkins, MLIS, AHIP College of Pharmacy Liaison Librarian George A. Smathers Libraries University of Florida Health Science Center Gainesville, Florida https://orcid.org/0000-0002-0914-2086 lauren.adkins@ufl.edu Michele R. Tennant, PhD, MLIS, FMLA, AHIP Interim Senior Director, Academic Research Consulting and Services (ARCS) George A. Smathers Libraries University of Florida Health Science Center Gainesville, Florida https://orcid.org/0000-0002-8439-6772 tennantm@ufl.edu Hannah F. Norton, MSIS, AHIP Department Chair George A. Smathers Libraries University of Florida Health Science Center Gainesville, Florida http://orcid.org/0000-0001-8062-2763 nortonh@ufl.edu Chloe Hough, MLIS, Evening Circulation Supervisor George A. Smathers Libraries University of Florida Health Science Center Gainesville, Florida http://orcid.org/0000-0002-4865-3049 choughatuf@ufl.edu Mary E. Edwards, MLIS, EdD Liaison Librarian George A. Smathers Libraries University of Florida Health Science Center Gainesville, Florida meedwards@ufl.edu https://orcid.org/0000-0003-4047-8669 mailto:morgandanie.jane@ufl.edu https://orcid.org/0000-0002-0914-2086 mailto:lauren.adkins@ufl.edu https://orcid.org/0000-0002-8439-6772 mailto:tennantm@ufl.edu http://orcid.org/0000-0001-8062-2763 mailto:nortonh@ufl.edu http://orcid.org/0000-0002-4865-3049 mailto:choughatuf@ufl.edu mailto:meedwards@ufl.edu RESEARCH Hypothesis Vol. 33 No.1 Fall/Winter 2021 Matthew Daley, MLIS Webmaster and Designer George A. Smathers Libraries University of Florida Health Science Center Gainesville, Florida daleym@ufl.edu Melissa L. Rethlefsen*, MSLS, AHIP Executive Director and Professor Health Sciences Library and Informatics Center University of New Mexico Albuquerque, New Mexico https://orcid.org/0000-0001-5322-9368 mlrethlefsen@gmail.com *Ms. Rethlefsen is the former Fackler Director, Health Science Center Libraries, University of Florida ABSTRACT Objective: A diversity, equity, and inclusion (DEI) needs assessment was conducted at a health sciences library. The objectives were to (1) quantitatively assess patrons’ attitudes towards the existing DEI climate in the library and (2) qualitatively identify contextual factors influencing patrons’ perceptions. The purpose was to better understand patrons’ views of the DEI climate in order to identify potential gaps, strengths, and areas of improvement within the library. Population: The assessment was led by a DEI Team at the University of Florida’s Health Science Center Libraries (HSCL). HSCL serves the university’s six health science colleges from two sites, Gainesville and Jacksonville. HSCL Gainesville was selected as the study location, due to higher annual footfall within the physical library. Methods: Two “HappyOrNot” customer satisfaction machines were placed by library exits. Twelve DEI-related questions were displayed on the machines, one question per week, for the duration of 12 weeks total. Additionally, a box for open-ended comments was placed by each machine. The survey used convenience sampling and was anonymous. Findings: The total number of responses was 3,445, with 7 written comments. Overall, respondents felt happiest with the library’s welcoming and home-like environment. The library was perceived by many as a physically safe space. Most also felt that the library demonstrates a commitment to DEI and that they are treated fairly, equitably, and with respect by library staff. Areas of concern were a perceived lack of respect from other library visitors, as well as library services for patrons with disabilities and non-native English language speakers. mailto:daleym@ufl.edu https://orcid.org/0000-0001-5322-9368 mailto:mlrethlefsen@gmail.com RESEARCH Hypothesis Vol. 33 No.1 Fall/Winter 2021 Conclusions: The project enabled the HSCL to gather important information concerning patrons’ perceptions of the library’s DEI climate and their associated needs. The HappyOrNot machines provided a straightforward and convenient method for patron feedback. However, limitations of the pilot included the inability to obtain participants’ demographic information as well as the anecdotal evidence indicating that some respondents did not read the questions. Future plans therefore involve a follow-up online survey with detailed demographic questions and further use of the HappyOrNot machines to conduct additional micro-assessments. Keywords: diversity, equity, inclusion; climate assessment; needs assessment; medical libraries; health sciences libraries INTRODUCTION In October 2018, the Health Science Center Libraries (HSCL) at the University of Florida (UF) formed a new Diversity, Equity, and Inclusion Team. The Team was created by the HSCL’s director in alignment with a new strategic direction for diversity, equity, and inclusion identified by UF’s George A. Smathers Libraries, of which the HSCL is a part [1]. One of the Team’s earliest priorities—falling under one of its stated goals to “assess and identify diversity, equity, inclusion, and justice opportunities within the HSCL’s spaces”—was to understand patrons’ views of the diversity, equity, and inclusion (DEI) climate in HSCL and thus identify potential gaps, strengths, and areas of improvement. To do so, HSCL piloted the use of two “HappyOrNot” customer satisfaction machines to conduct a DEI needs assessment, with the purpose of exploring the DEI-related issues facing library patrons. It was anticipated that the results would inform the team’s future directions by opening a dialogue with patrons about their DEI-related needs. The HSCL has two physical library sites in Florida, one on the main UF campus in Gainesville and the other at a teaching hospital in Jacksonville, called the Borland Library. The DEI Team has members from both sites and is composed of library faculty, staff, and student assistants. HSCL Gainesville was the location selected for this study. The library serves the university’s six health science colleges in addition to a number of associated centers and institutes and as such experiences a significantly higher level of foot traffic in the physical library throughout the year (e.g., in 2019 over 200,000 walk-in patrons visited HSCL Gainesville). HSCL Gainesville’s patrons include, but are not limited to, the health science center colleges’ faculty, staff, and students, as well as hospital patients and members of the public. The HappyOrNot assessment launched on April 12, 2019, and ran until July 8, 2019. The study’s aim was to explore the DEI- RESEARCH Hypothesis Vol. 33 No.1 Fall/Winter 2021 related issues currently experienced by HSCL patrons in Gainesville. Its objectives were to (1) quantitatively assess patrons’ attitudes towards the existing DEI climate in the library and to (2) qualitatively identify contextual factors influencing patrons’ perceptions. LITERATURE REVIEW Assessing patron satisfaction in libraries The inspiration for the needs assessment was a study conducted by the University of Central Florida (UCF), in which HappyOrNot machines were used alongside several other micro-assessments to elicit patron feedback regarding library services, events, resources, and policies [2]. The goal of the micro-assessments was to decrease the time taken to obtain feedback in comparison to more common methods of assessments such as surveys and focus groups. According to UCF, the HappyOrNot machines were an effective method for gauging patrons’ opinions on library policies and resources. Overall, the micro-assessments enabled the librarians to quickly obtain feedback from patrons, allowing them to create solutions more efficiently. HappyOrNot machines are utilized globally to measure customer satisfaction in a variety of sectors such as retail, transportation, restaurants, education, healthcare, and human resources [3]. The machine terminals have four emoji buttons: a “very positive” green smiling emoji, a “positive” light green slightly smiling emoji, a “negative” light red slightly frowning emoji, and a “very negative” red frowning emoji. Users can press an emoji button to answer questions on the terminal screen, thereby recording their experiences and responses in real time. When considering selecting HappyOrNot as the data collection instrument for this study, the DEI Team had a number of initial concerns about the emojis and the color combinations of the buttons. These concerns related to whether the associated meaning of the buttons would be clearly conveyed to all patrons, including international students and people with color-blindness or autism. These concerns were allayed through a literature search pertaining to emojis and color as methods of communication. Communicating emotion through emojis and color associations Emojis consist of facial images that convey a variety of emotions in pictorial form and have been used by industries to gauge users’ satisfaction [4]. For example, Jaeger et. al’s study demonstrates how emoji surveys were successfully used to measure customers’ satisfaction concerning new food and beverage products [5]. Emojis have also been used for educational purposes; for example, a study by Voss discusses the RESEARCH Hypothesis Vol. 33 No.1 Fall/Winter 2021 use of emojis to teach children with autism how to better recognize emotions [6]. The DEI Team found this article encouraging, as it indicated that the meaning of HappyOrNot emojis would be inclusive for most patrons. Regarding color, many studies discuss the use of color to convey particular emotions or to assess participants' perceived emotions. A study of college students conducted by Kaya and Epps found that participants universally attributed the red to anger when assigning colors to emotional reactions. [7] In contrast, the color green was mainly associated with positive emotions due to reminding participants of nature. When seeking to obtain participant feedback, the literature illustrates that not all cultures and nationalities ascribe the same meaning to colors and emotions. A seminal study by Saito found that people from Japan and nearby Asian countries often have a preference for the color white due to religious connotations [8]. Study participants associated white with cleanliness, purity, harmony, beauty, and gentleness. Similarly, a study conducted by Hupka with participants from Germany, Mexico, Poland, Russia, and the United States found that the colors red and black were universally associated with anger [9]. Even though studies have shown that various cultures and nationalities can hold different emotional associations for color, the association of green with the emotion of positivity and red with negativity is common across several cultures and nationalities [7- 10]. After reviewing this literature, the DEI Team therefore felt comfortable moving forward with piloting the use of the HappyOrNot machines for the needs assessment. DEI climate assessments The development of the pilot questions stemmed from a search conducted for DEI assessment literature in February 2019. First, the team looked for existing DEI climate- related surveys within the librarianship and health literature. The majority of these search results focused on surveying employees not patrons. For example, the literature highlights the recurring use of ClimateQUAL, which is an instrument frequently used within libraries to assess organizational climate through staff perceptions [11]. The search was therefore expanded to encompass university-level DEI surveys, which are often accessible through Google as grey literature in the form of reports. The Google search yielded useful results. The first was Case Western Reserve University’s climate survey, which used Likert Scale questions to assess how students and faculty perceived the environment for inclusion and diversity [12]. These questions pertained to participants’ general comfort levels, their experiences of discrimination, and whether they felt that the environment encourages diverse groups of people to interact. Overall, the results illustrated “differences both among and within groups by gender, race/ethnicity, and international status” (p.1) [12]. Another source of inspiration for the RESEARCH Hypothesis Vol. 33 No.1 Fall/Winter 2021 HappyOrNot questions was the University of Michigan’s campus climate survey on diversity, equity and inclusion [13]. Participants were asked to indicate their level of agreement with a number of statements, including those focused on institutional commitment to DEI and on individuals being valued, treated with respect, and feeling they belong. The survey found that students with historically marginalized identities experienced the campus less positively [13]. At the time of planning the pilot, the DEI Team’s Lead was a member of the Medical Library Association’s (MLA) Diversity and Inclusion Task Force and, as such, helped with question formation for an online general membership survey that took place in 2019 [14]. The survey’s purpose was to facilitate deeper understanding of the membership’s demographics and their feelings in relation to MLA’s DEI efforts. Three questions from the survey were repurposed for the HappyOrNot assessment: “I have felt welcomed and included at MLA annual meetings”; “I feel MLA has a strong commitment to diversity, equity, and inclusion”; and “I feel I am treated with respect in MLA.” Since the original searches in early 2019, the librarianship literature has expanded to include three new sources of relevance. The first is Pionke’s article which concludes by emphasizing the importance of conducting recurring DEI surveys, ideally every three years [14]. Secondly, Jones and Murphy discuss how DEI climate surveys benefit library administration through enabling leaders to assess organizational readiness for “maintaining and supporting an environment that is safe, inclusive, and welcoming” (p.79) [15]. Finally, the Wisconsin Department of Public Instruction provides a reflection tool for libraries seeking to evaluate services on an ongoing basis through a DEI lens [16]. The guide is a checklist that encourages organizations to reflectively consider their communities’ demographics, to ensure that every patron feels “safe, welcomed, and respected” throughout all library experiences. METHODS Twelve closed-ended questions (Table 1) were developed through team discussion and consensus, based on the recurring DEI assessment themes discovered in the literature: welcome, respect, inclusivity, accessibility, physical and emotional safety, fair and equitable treatment, and organizational commitment to DEI. The overarching purpose of these questions was to facilitate assessment of the library’s DEI climate by evaluating patrons’ satisfaction with the library’s physical space, library services, and staff attitudes and behaviors. The questions were displayed on two HappyOrNot machines, one question per week, for the duration of 12 weeks total. A comment box was placed by RESEARCH Hypothesis Vol. 33 No.1 Fall/Winter 2021 each machine, so that patrons could provide open-ended written responses to the weekly questions if they chose to while still maintaining their anonymity. Week 1 Do you feel welcome at this library? Week 2 Do you feel you are treated with respect by this library’s staff? Week 3 Do you feel you are treated with respect by other visitors at this library? Week 4 Do you feel this library is an inclusive physical space? Week 5 Do you feel this library is a physically safe place for people of all backgrounds? Week 6 Do you feel this library is an emotionally safe space? Week 7 Are you satisfied with this library’s services for people whose native language is not English? Week 8 Do you feel this library reliably meets the needs of individuals with disabilities? Week 9 Do you feel this library is a welcoming environment for families? Week 10 Do you feel you are treated fairly and equitably at this library? Week 11 Do you feel that this library’s staff will take appropriate action in response to incidents of discrimination? Week 12 Do you feel this library demonstrates a strong commitment to diversity, equity, and inclusion? Table 1 HappyOrNot questions by week Two HappyOrNot machines were purchased by the HSCL and placed by high-traffic library exits on the first and second floors. Specifically, the Smiley Terminal, depicted in Figure 1, was selected as this model is light-weight and portable, runs on batteries so is wireless, and has a simple interface encompassing a sleeve for printed questions. An additional advantage was that the data analysis process was expected to be mostly automated as these terminals are linked to a web-based software that compiles respondents’ feedback as it is submitted. In terms of initial machine set-up, the online software provided access to a personalized HappyOrNot dashboard to which the 12 assessment questions were added. Each RESEARCH Hypothesis Vol. 33 No.1 Fall/Winter 2021 question had to be entered separately and the dates during which each question would be active had to be scheduled. This added a layer of complexity to the data collection process because, although the software was straightforward to use, the weekly question displayed on the actual HappyOrNot terminals needed to be printed. This meant that the DEI Team needed to physically change out the printed question on the correct dates to ensure that patrons’ responses matched the correct question on the web-based dashboard. For the design of the printed questions, the DEI Team chose dark text on a white background. An explanatory sentence was added to each printout stating “HSC Library staff want to better understand your experiences relating to diversity & inclusion in the library – please provide feedback to help us enhance services.” This design was replicated on the two comment boxes (Figure 1). Figure 1 Image of the HSCL’s HappyOrNot Smiley Terminal and comments box After the study was approved as IRB exempt (IRB201900781), the pilot launched on April 12, 2019, and ran until July 8, 2019. An earlier timeframe was initially selected to coincide with the spring semester, but machine-ordering delays meant that the study continued into finals week and the beginning of the summer semester. The assessment was promoted through the library’s Facebook and Twitter accounts, through the website, and in-person by the desk staff. FINDINGS RESEARCH Hypothesis Vol. 33 No.1 Fall/Winter 2021 The total number of responses received for all questions was 3,445 across both HappyOrNot terminals, along with 7 written comments. The response rate was approximately 4.3% as around 81,000 visitors entered the library during the assessment period. Patterns of gate count matched well with the number of responses over the study timeframe. The first two weeks of the assessment occurred during the last two weeks of the semester, and average gate count and response numbers were 13,979 and 724 respectively (19.3 visits per response). Weeks 4 through 12 reflect the lower gate count seen during intersession and summer sessions weeks, with average gate count as 4,631 and response numbers averaging 184 (25.2 visits per response). Finals week was the outlier, with 11,413 entrances and exits but only 342 responses (33.4 visits per response). Quantitative results The quantitative results were automatically collated through the HappyOrNot software, by number of responses per question and percentages of responses by question, for the four available button options on the machines. Microsoft Excel was used to filter the data by time period (weeks, days, hours) and by location within the library (first or second floor). Overall, the team was excited to see that the majority of the responses received for all 12 questions was “very positive.” RESEARCH Hypothesis Vol. 33 No.1 Fall/Winter 2021 Figure 2 Quantitative results by percentage of responses for weeks 1–12 The highest number of responses for the whole assessment were received for the first 2 questions posted, which were Do you feel welcome at this library? (828 responses) and Do you feel you are treated with respect by this library’s staff? (620 responses). Figure 2 shows the responses for all 12 questions, broken down by the percentage of individuals answering for “very positive,” “positive,” “negative,” and “very negative.” The following questions had the highest percentages of responses for “very positive” and “positive”: RESEARCH Hypothesis Vol. 33 No.1 Fall/Winter 2021 • Do you feel you are treated fairly and equitably at this library? (combined 98% for “very positive” and “positive” from 91 responses) • Do you feel this library demonstrates a strong commitment to diversity, equity, and inclusion? (combined 98% for “very positive” and “positive” from 106 responses) • Do you feel you are treated with respect by this library’s staff? (combined 97% for “very positive” and “positive” from 620 responses) • Do you feel this library is a physically safe place for people of all backgrounds? (combined 95% for “very positive” and “positive” from 259 responses) The questions with the largest percentage of “negative” and “very negative” responses were as follows: • Do you feel you are treated with respect by other visitors at this library? (combined 10% “very negative” and “negative” from 342 responses) • Are you satisfied with this library’s services for people whose native language is not English? (8% combined “negative” and “very negative” from 163 responses) • Do you feel this library reliably meets the needs of individuals with disabilities? (7% “negative” or “very negative” from 261 responses) The results were also analyzed by the overall number of positive (“positive” combined with “very positive”) and negative (“negative” combined with “very negative”) responses per hour over the whole 12-week period (Appendix A). The highest number of negative responses received coincided with the hours that had the most overall responses, which was around lunchtime between 12pm and 2pm. The same pattern held true for the highest number of positive responses received. An exception to this was the negative results often received between 7am and 9am, which we attribute to the unavoidable daily noise that the library cleaners make during this time period. In terms of the highest percentages of positive or negative responses per hour throughout the whole assessment period, participants responded most favorably at 2am, 3am, and 5am (100% “very positive” or “positive”) and most unfavorably at 4am and 6am (50-60%). However, due to the smaller number of overall responses at these times (2am, n=8; 3am, n=5; 4am, n=2; 5am, n=2; 6am, n=4), little weight can be placed on these findings. Finally, the results were analyzed by the location of the terminals (Appendix B). Unsurprisingly, the machine at the main entrance of the library, on the first floor by the information services desk, collected 72% (2,488) of the total responses. During the assessment period, the second floor entrance received approximately 40% as much RESEARCH Hypothesis Vol. 33 No.1 Fall/Winter 2021 traffic as the first floor; likewise, 28% (1,273) of the total responses were from the second floor. Responses from the second floor were slightly more negative (combined 8.16% “very negative” and “negative” from 104 responses) than those from the first floor (combined 2.74% “very negative” and “negative” from 88 responses). Qualitative results Only 7 written responses were received via the comment boxes over the entire 12-week period of the survey. These qualitative results were thematically analyzed, with written responses divided into three categories: positive, mixed, and negative. Three of the seven responses were extremely positive, suggesting that the library is perceived as a welcoming and homey place and that patrons appreciate the personal greetings they receive when they walk in from the desk staff: “Best. library. ever. I love how the staff always says ‘Hi’ when you walk in. I feel at home here.” (Week 1) “LOVE THIS LIBRARY” (Week 7) “Yaaaaaaaaaaaa :)” (Week 8) Two of the written responses were mixed with one response again implying that the library is perceived to be a welcoming and safe space and the other referring positively to DEI. At the same time, the mixed responses also commented on aspects of the library space that could be improved: “Diversity is good but please do not eat snacks in [the] library” (Week 1) “I really enjoy coming here, I feel welcome & safe. I feel sometimes it could be a little cleaner.” (Week 1) The final two responses were negative. Both were received during the week the HSCL was experiencing problems with overnight access to the library via swipe-card (see the Discussion section): “Notify students of all 6 Health Colleges that they have 24 hour access and to use their IDs when [the] library is closed.” (Week 3) “Larger Warning and STOP signs!” (Week 3) RESEARCH Hypothesis Vol. 33 No.1 Fall/Winter 2021 DISCUSSION Implications The quantitative and qualitative findings suggested that patrons are happiest with the library’s welcoming and home-like environment; the library is perceived by many as a physically safe space; patrons largely feel the library demonstrates a commitment to DEI; and patrons feel they are treated fairly, equitably, and with respect by library staff. Of concern, some patrons expressed that they did not receive enough respect from other library visitors. The HSCL has taken these responses seriously and has created a patron code of conduct, delineating expectations related to respect, language, and physical contact [17]. The code emphasizes to patrons that the library is a community space, welcoming and inclusive to all, and clearly communicates consequences for repeated violations. The team also created accompanying guidance documentation for staff regarding how to respond to specific instances of disruption and harassment as well as a three strikes policy to remove offenders. The pilot was administered during a time when the library experienced an ongoing malfunction in its overnight entrance to quiet study space, and it is possible that this contributed to patrons’ perceptions of lack of respect from other library users. During this timeframe, the library received numerous complaints about patrons causing noise and disruption through knocking on the doors and accidentally setting off the alarms. Subsequently, the issue with the doors has been resolved. Additionally, the library has been proactive in decreasing user error, by improving the visibility and readability of the door signage instructions for 24-hour library access and emailing all of the students from the six health science center colleges to remind them how to access the library outside of staffed hours. The library’s Marketing Team also produced a video tutorial on how to access the library after hours, which was distributed through the library’s social media pages [18]. Other concerns revealed by the assessment suggest that the library needs to improve its services for patrons with disabilities and non-native English language speakers. The result for disabilities is unfortunately not surprising. The HSCL encompasses three floors, with the main book collection and heavily used quiet study space located on the third floor and the information desk (check out, reserves, front-line information/reference) located on the first floor. There are no elevators in the library proper, and the building elevators and ADA door buttons were unreliable at times during the assessment period. Following the pilot, library administration advocated for RESEARCH Hypothesis Vol. 33 No.1 Fall/Winter 2021 replacement of the ADA door buttons as well as the creation of a wheel-chair accessible option to enter the HSCL’s newly refurbished computer lab. Both of these improvements have been accomplished. The DEI Team was initially surprised by the results of the language-related question. On reflection, UF’s Health Science Center does have a large number of international graduate and professional students (12.01% of students from 33 countries) [19]. A large number of post-doctoral associates at UF are also international (41.9% of them are not U.S. citizens or nationals) [20]. Although UF does not keep statistics on the number of languages spoken at the university, these figures on the international presence suggest that large numbers of the UF community do not speak English as their primary language. While the responses to the language question points to an area of concern, it is not possible, based on the lack of specificity in the question, to determine exactly what needs to be improved in this area. Although HSCL cannot make all resources and signage in all languages, it is certainly within the library’s purview to be knowledgeable about language resources and services on campus and to train staff to better and more patiently communicate with patrons whose native language is not English. While the overall feedback related to staff respect, comfort, equity and safety was positive, the Team understands that ongoing success in DEI requires ongoing effort. We have organized a number of trainings for staff (micro-aggression, bystander intervention, and many others) and will continue to do so. LIMITATIONS While the library did implement a number of changes based on the HappyOrNot results, there were multiple limitations that hindered the team’s ability to conclusively answer all of its questions. Due to the structure of the HappyOrNot machines, there was no collection of demographic data, which made it impossible to discern whether diverse population groups experience the library differently. As such, we do not know whether the negative responses came from specific population groups. As the machines were located in the physical library, this meant that individuals who did not visit the library in person were unable to participate (such as distance students or those who may feel uncomfortable coming to the library). Since the purpose of the pilot was to assess patrons’ feelings about the library’s DEI climate, patrons who do not feel welcome at the library being excluded could skew the results to the positive. Another issue with the format of the pilot was anecdotal evidence indicating that patrons did not always read the questions and answer accordingly. Desk staff noticed patrons not fully reading the prompts, or not reading them at all, before hitting the response RESEARCH Hypothesis Vol. 33 No.1 Fall/Winter 2021 buttons. For example, a team member working the information desk at the first floor entrance observed multiple instances of patrons hitting one of the buttons after incorrectly or incompletely reading the prompt and voicing their mistake afterwards, i.e., not realizing that the question was asking about DEI in the library. It was also observed that many patrons would repeatedly and sometimes excessively hit the “very positive” button without looking at the prompt when exiting the library. While this could indicate that the patrons are happy with the library in general, the unknown extent of repeat responses muddles the data regarding the specific questions posed. The disregard for the actual questions may have been a result of question fatigue or simply the fact that the HappyOrNot format is generally associated with customer satisfaction, causing patrons to not look at the specific wording presented each week. HappyOrNot’s website states that they are “the global leader in instant customer and employee satisfaction reporting.” If patrons are mainly familiar with seeing feedback machines in businesses and retail locations, it is possible that they would assume the question is asking about general satisfaction [3]. The timing of the assessment caused another barrier to participation. The pilot was launched toward the end of the spring semester, when the HappyOrNot machines arrived at the library after several ordering delays; based on the pilot start date, the study continued into finals week and the summer semester. Since many students opt not to take summer classes, a significant number leave Gainesville from May to August. As such, the library receives a significantly lower footfall during the summer in comparison to the fall and spring. In 2019 the number of visitors to the library dropped from 59,239 in April to 21,567 in May, a 64% decrease. This drop in overall library visitation during the assessment period is reflected in the lower participation rate in the later weeks of the pilot. As mentioned, most responses were received during the first two weeks of the study, the time that coincided with the end of the spring semester. During the final 6 weeks of the pilot, the number of responses was 1,017, accounting for 30% of the total responses over the 12 weeks. While questions posed later in the assessment received fewer responses, this timeframe did allow the DEI Team to learn from users quickly and act accordingly, more so than if the study had been delayed to the fall semester. The DEI Team feels that the HappyOrNot machines were useful for gauging patrons’ general opinions of the library and its services, thereby concurring with UCF who used this data collection instrument during their own micro-assessments [2]. However, HappyOrNot machines are less useful for DEI-specific needs assessments, due to the inability to collect data on participants’ demographics. It is the team’s opinion that when paired with demographic data, the pilot questions used in this study would likely reveal whether underrepresented groups experience the library’s DEI climate differently to their RESEARCH Hypothesis Vol. 33 No.1 Fall/Winter 2021 majority counterparts. The team therefore looks forward to gaining more detailed evidence through future iterations of this study. With those iterations, we also hope to compare our findings to Case Western Reserve University and the University of Michigan’s campus climate surveys [12, 13], in which participants with historically marginalized identities tended toward less positive experiences. FUTURE RESEARCH As described previously, our experiences suggest that HappyOrNot machines provide only a very cursory snapshot when considering specific issues within DEI. It is unclear whether respondents consistently read the questions or were aware that the questions changed every week—either of these issues had the potential to affect data collection. Of greater consequence was the inability to track who was answering and how, which made it impossible to determine whether underrepresented groups have different perceptions or experiences than their majority counterparts. In order to address some of the limitations of this pilot project, the HSCL has developed an online survey that includes detailed demographic questions (age, gender identity, race, sexual orientation, native language, international status, first generation status, and caregiving responsibilities). The survey includes the original HappyOrNot questions, augmented with new queries related to the inclusiveness of the library’s web presence and HSCL’s inclusiveness to families, diverse religions, and other identities. The DEI Team looks forward to analyzing the results of this assessment of inclusivity and to responding to any deficits discovered. The team also plans to complete a local library-based environmental scan, stemming from the priorities identified through the pilot HappyOrNot project and subsequent online survey. From the pilot’s results, we anticipate that the environmental scan will begin by focusing on current facilities and services for people with physical disabilities— identifying current resources and resource gaps and areas for improvement such as where policies need to be made, training conducted, or resources publicized. Because the HappyOrNot pilot results suggest services for non-native English language speakers and persons with disabilities are of concern, the DEI Team plans to conduct focus groups with such patrons to find out which specific improvements the library can make to its spaces and resources to be more inclusive. The HSCL’s DEI assessment efforts will be an ongoing process. Although the HappyOrNot system is not ideal for a detailed assessment, it is possible that making its intent (DEI) and framework (the questions change) more apparent to patrons could yield results that better reflect their experiences and perceptions. As such, the team plans to RESEARCH Hypothesis Vol. 33 No.1 Fall/Winter 2021 run a re-worked HappyOrNot assessment in 2022, post-Covid. In addition to an advertising campaign introducing the intent of the HappyOrNot assessment, it may be beneficial to switch questions every two weeks rather than every week, as this may give patrons who visit the library less frequently a better chance to respond to each question. CONCLUSION It is crucial for libraries to continuously work towards incorporating DEI and social justice initiatives into their services for their communities while also as striving to become safe and welcoming spaces that celebrate the diversity of their patrons and staff. As evidenced by the literature, the majority of DEI-related surveys conducted in libraries tend to assess DEI climate through evaluating staff perceptions. However, it is also essential to evaluate patron perceptions on an ongoing basis through a DEI lens [15- 16]. Overall, the HSCL DEI Team feels that this pilot project was fairly successful. The HappyOrNot machines enabled us to gather important information concerning patrons’ perceptions of the library’s DEI climate and their associated needs in relation to physical space, services, and staff attitudes and behaviors. The HappyOrNot machines provided a straightforward and convenient method for patron feedback, resulting in an impressive number of responses. However, for a more nuanced analysis of a library’s DEI climate, multiple data collection methods are recommended. In conclusion, it is essential to note that DEI work is an ongoing process that takes time, effort, innovation, and—most importantly—the ability to listen to and act on the needs of the communities being served. AUTHOR CONTRIBUTIONS Conceived and designed project: MLR, JMD, HFN, MD Data Collection: JMD, MD, CEH, MRT, HFN, MLR, MEE, LEA Data Analysis: Quantitative: HFN, MEE; Qualitative: JMD Figures: HFN, MEE, MD Written Draft: JMD, LEA, MRT, CEH, HFN Article Revision: HFN, MRT, MLR, JMD, MEE, CEH, LEA, MD RESEARCH Hypothesis Vol. 33 No.1 Fall/Winter 2021 Final Approval: JMD, CEH, HFN, LEA, MD, MEE, MLR, MRT REFERENCES 1. University of Florida George A. Smathers Libraries. Strategic directions [Internet]. Gainesville, FL: University of Florida; 2018 Dec [cited 1 Mar 2021]. . 2. Gillum S, Williams N, Gotschall T. A year of creative assessment: incorporating micro-assessments into the library [Internet]. Medical Library Association Conference: Chicago, IL; 2019 Mar 18 [cited 1 Mar 2021]. . 3. HappyOrNot [Internet]. [cited 1 Mar 2021]. . 4. Rodrigues D, Prada M, Gaspar R, Garrido M, Lopes D. Lisbon emoji and emoticon database (LEED): norms for emoji and emoticons in seven evaluative dimensions. Behav Res. 2017 March 31;50:392–405. DOI: https://doi.org/10.3758/s13428-017-0878-6. 5. Jaeger S, Lee S, Kim K, Chheang S, Jin D, Ares G. Measurement of product emotions using emoji surveys: case studies with tasted foods and beverages. Food Qual Prefer. 2017 December; 62: 46–59. DOI: https://doi.org/10.1016/j.foodqual.2017.05.016. 6. Voss C, Schwartz J, Daniels J, Kline A, Haber N, Washington P, Tariq Q, Robinson T, Desai M, Phillips J, Feinstein C, Winograd T, Wall D. Effect of wearable digital intervention for improving socialization in children with autism spectrum disorder: a randomized clinical trial. JAMA Pediatr. 2019 May 1;173(5):446–454. DOI: https://doi.org/10.1001/jamapediatrics.2019.0285. 7. Kaya N, Epps HH. Relationship between color and emotion: a study of college students. Coll Stud J. 2004 Sep;38(3):396–405. 8. Saito M. Comparative studies on color preference in Japan and other Asian regions, with special emphasis on the preference for white. Color Res Appl. 1996 Feb;21(1):35–49. DOI: https://doi.org/10.1002/(SICI)1520-6378(199602)21:1<35::AID- COL4>3.0.CO;2-6. 9. Hupka RB, Zaleski Z, Otto J, Reidl L, Tarabrina NV. The colors of anger, envy, fear, and jealousy: a cross-cultural study. J Cross Cult Psychol. 1997 Mar;28(2):156–71. DOI: https://doi.org/10.1177%2F0022022197282002. 10. Yu HC. A cross-cultural analysis of symbolic meanings of color. Chang Gung Journal of Humanities and Social Sciences. 2014 Apr 1;7(1):49–74. https://uflib.ufl.edu/wordpress/files/2021/04/Strategicdirections2018web.pdfhttps:/library.ufl.edu/pers/documents/Strategicdirections2018web.pdf https://uflib.ufl.edu/wordpress/files/2021/04/Strategicdirections2018web.pdfhttps:/library.ufl.edu/pers/documents/Strategicdirections2018web.pdf https://www.eventscribe.com/2019/MLA/assets/pdf/MLA19_Paper_Abstracts_March_18.pdf https://www.eventscribe.com/2019/MLA/assets/pdf/MLA19_Paper_Abstracts_March_18.pdf https://www.happy-or-not.com/en/smiley-terminal/%3e. https://www.happy-or-not.com/en/smiley-terminal/%3e. https://doi.org/10.3758/s13428-017-0878-6 https://doi.org/10.1016/j.foodqual.2017.05.016 https://doi.org/10.1001/jamapediatrics.2019.0285 https://doi.org/10.1002/(SICI)1520-6378(199602)21:1%3c35::AID-COL4%3e3.0.CO;2-6 https://doi.org/10.1002/(SICI)1520-6378(199602)21:1%3c35::AID-COL4%3e3.0.CO;2-6 https://doi.org/10.1177%2F0022022197282002 RESEARCH Hypothesis Vol. 33 No.1 Fall/Winter 2021 11. Association of Research Libraries. ClimateQUAL [Internet]. Washington, DC; [cited 10 Mar 2021]. . 12. Case Western Reserve University. 2010 - 2011 Climate survey: results on common questions [Internet]. Cleveland, OH: Case Western Reserve University; 2011 [cited 10 on Mar 2021]. . 13. University of Michigan. Results of the 2016 University of Michigan student campus climate survey on diversity, equity and inclusion [Internet]. Anne Arbor, MI: University of Michigan; 2016 [cited 10 on Mar 2021]. . 14. Pionke, J. Medical library association diversity and inclusion task force 2019 survey report. JMLA. 2020 Jul;108(3):503–512. DOI: https://doi.org/10.5195/jmla.2020.948. 15. Jones SD, Murphy B. Diversity and inclusion in libraries: a call to action and strategies for success. Lanham, Maryland, 2019. Chapter 8: Recruiting and retaining a diverse workforce; p. 75–88. 16. Wisconsin Department of Public Instruction. The inclusive services assessment and guide for Wisconsin public libraries [Internet]. Madison, WI; 2019 Jun [cited on 10 Mar 2021]. . 17. University of Florida Health Science Center Libraries. Code of conduct [Internet]. Gainesville, FL: University of Florida; 2021 Apr [cited 1 Apr 2021]. . 18. University of Florida Health Science Center Libraries. In today's #HowToLibrary post, we're reminding HSC students of their 24/7 access to our 2nd floor! We've been having some swipe card system issues, so if you're an HSC student and your card isn't working, drop by the 1st floor desk and we'll help get you sorted. Happy studying! [Internet]. 2021 Jan 7 [cited 1 Apr 2021]. . 19. University of Florida Graduate School Data Management Unit. Re: Question about grad students and professional students. Email message to: Michele Tennant. 2021 Mar 8, 11:15 a.m. [12 Lines]. 20. University of Florida. UF facts workforce [Internet]. Gainesville, FL: University of Florida; 2021 Mar [cited on 1 Apr 2021]. . https://www.climatequal.org/home https://case.edu/diversity/about/diversity-climate-survey https://diversity.umich.edu/wp-content/uploads/2017/11/DEI-STUDENT-REPORT-FINAL.pdf https://diversity.umich.edu/wp-content/uploads/2017/11/DEI-STUDENT-REPORT-FINAL.pdf https://doi.org/10.5195/jmla.2020.948 https://files.eric.ed.gov/fulltext/ED599993.pdf https://library.health.ufl.edu/about-us/library-user-policies/code-of-conduct/ https://www.facebook.com/UFHSCL/posts/5027652200608520 https://public.tableau.com/shared/D49WW6BX7?:display_count=y&:origin=viz_share_link&:embed=y&:showVizHome=no https://public.tableau.com/shared/D49WW6BX7?:display_count=y&:origin=viz_share_link&:embed=y&:showVizHome=no RESEARCH Hypothesis Vol. 33 No.1 Fall/Winter 2021 APPENDIXES Appendix A: Responses by Hour RESEARCH Hypothesis Vol. 33 No.1 Fall/Winter 2021 Appendix B: Percentage of Responses by Library Floor