Evidence Based Library and Information Practice Evidence Based Library and Information Practice 2010, 5.4 105 Evidence Based Library and Information Practice Evidence Summary Adherence to RUSA’s Guidelines for Virtual Reference Services is Below Expected in Academic Libraries A Review of: Platt, J. & Benson, P. (2010) . Improving the virtual reference experience: How closely do academic libraries adhere to RUSA guidelines? Journal of Library and Information Services in Distance Learning, 4(1-2), 30-42. Reviewed by: Annie M. Hughes Reference Librarian, Herman Ostrow School of Dentistry University of Southern California Los Angeles, CA, United States of America Email: Received: 14 Sept. 2010 Accepted: 20 Nov. 2010 amhughes@usc.edu 2010 Hughes. This is an Open Access article distributed under the terms of the Creative Commons- Attribution-Noncommercial-Share Alike License 2.5 Canada (http://creativecommons.org/licenses/by-nc- sa/2.5/ca/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly attributed, not used for commercial purposes, and, if transformed, the resulting work is redistributed under the same or similar license to this one. Abstract Objective – To evaluate the quality of academic libraries’ virtual reference services and measure compliance to the Reference & User Services Association’s (RUSA’s) Guidelines for Virtual Reference & User Services. Design – Qualitative research study evaluating virtual reference chat sessions using RUSA’s Guidelines for Virtual Reference & User Services. Setting – Virtual reference environments in public academic libraries in the United States. Subjects – Twenty virtual reference providers from public academic libraries. Methods – Initially researchers selected 1 academic library out of each of the 50 states to evaluate for quality virtual chat reference services, however because of factors including time and availability of virtual chat services to unaffiliated institutions; the sample included only 20 academic libraries. After selecting the 20 academic libraries for evaluation, researchers posed as virtual chat reference patrons using emails and aliases that had no affiliation to any particular institution. Researchers then asked the librarian or library mailto:amhughes@usc.edu� Evidence Based Library and Information Practice 2010, 5.4 106 staff a two-part question making sure to leave out any library jargon or anything that would lead the virtual chat reference operator to recognize that they are also affiliated with a library or library school. Using the RUSA Guidelines for Virtual Reference & User Services, researchers then evaluated their virtual chat reference experience for the following: Approachability; Interest; Listening/Inquiring; Searching; Follow-Up; Suggests patron call or visit the library. Main Results – When evaluated for jargon- free websites and overall usability in finding all types of reference services, 80% of the library’s websites were easy to use and jargon free, reflecting overall high usability. Evaluation of library staff’s ability to maintain “word contact” by writing prompts to convey interest in the patron’s question left some room for improvement. Sixty percent of researchers coding their virtual reference experience thought the level of contact was below expected. Information regarding question and answering procedures, question scope, types of answers provided and expected turnaround time for questions was only available in 30% of examined websites. Thirty-five percent of researchers felt that library staff members gathered enough information to answer the question without compromising privacy, however, 25% thought that staff members gathered a very small amount of information on the patron’s need, although privacy never felt compromised. When researchers evaluated the library staff member on their ability to explain how to utilize resources properly, 50% thought the instruction provided was below average. Although 15% believed they received “superior instruction.” Seventy-five percent of the researchers were not asked by a library staff member if the question received an adequate answer, 50% of reference transactions library staff did not consult a librarian or expert, and in 55% of transactions the staff member did not suggest that the patron visit or call the library. Conclusion – While the researchers received some valuable information about the need to improve virtual reference services in academic libraries, there were some flaws in their research. The question they developed was almost too clear and made it difficult for the individual answering the chat reference to adequately perform a reference interview or ask probing questions. It is possible that because researchers carefully planned out their question they set themselves up to create an interaction that would not normally occur in a virtual chat reference environment. Also, because researchers were unable to evaluate what was occurring in the environment surrounding the virtual chat reference providers it was impossible to make a judgment on the speed or length of the interaction. The researchers did come away from the study with results that point to a need to utilize the RUSA guidelines in order to conduct effective reference interviews, maintain appropriate contact with the user when engaging in chat reference, provide instruction and point patrons to quality resources as well as consult an expert on the topic if needed. They surmised that if libraries utilized these guidelines, virtual chat reference services would be improved. Commentary The researchers identify limitations of the study and give those who would like to explore virtual reference quality further a few ideas to consider. The two-part reference question posed by the users to the library staff did not prompt an in-depth reference interview because the question was very clear. Evidence Based Library and Information Practice 2010, 5.4 107 If the question posed was ambiguous it is possible that the library staff would have asked more questions of the user. By selecting only one question to ask of the virtual chat reference providers, the data available was limited. If the researchers had utilized several questions of various levels of difficulty, perhaps quality of service would be easier to clearly evaluate. Researchers also evaluated their virtual chat reference experience by looking at whether or not library staff or librarians consulted experts to effectively answer the question. Experts need not be consulted for “manageable” or simple reference questions, hence another reason to use several questions of varying levels of difficulty. Another important aspect missing from this study is that it is not known how much training library staff had on providing virtual reference services. Also, because the user had to appear to be a regular patron who is unaware of library jargon, the level of expertise of the respondent could not be recorded. One of RUSA’s Guidelines discusses the importance of maintaining “word contact,” but what researchers did not consider is the type of chat client used. Chat clients that do not provide sound alerts or other prompts could affect responsiveness especially if another patron interrupts the service. Usability of the chat client and the library staff’s comfort level and knowledge of how to use the client could affect maintenance of contact. While there are identified limitations provided by the researchers and the sample size was not particularly representative of the academic libraries in the United States, the study does reflect on what aspects of virtual reference need improvement and provides some evidence of what users expect from virtual reference service. Unfortunately, it would be difficult to point other researchers to this paper as an example since there are so many flaws with the research. There are other higher quality studies available that evaluate virtual chat reference services against the RUSA Guidelines. / Evidence Based Library and Information Practice Evidence Summary