blecic.p65 Using Transaction Log Analysis 39 Using Transaction Log Analysis to Improve OPAC Retrieval Results Deborah D. Blecic, Nirmala S. Bangalore, Josephine L. Dorsch, Cynthia L. Henderson, Melissa H. Koenig, and Ann C. Weller This OPAC transaction log analysis study compared data derived from two sets of logs within a six-month period. Analysis of the first set of data revealed that users experienced difficulty with basic searching techniques. The OPAC introductory screens were simplified and clarified to help users improve search success rates. The second set of data, analyzed after screen changes had been made, showed statistically significant differ ences in search results. Regular monitoring of OPACs through transac tion log analysis can lead to improved retrieval when changes are made in response to an analysis of user search patterns. PAC users leave behind them a trail of searches and results that provides evidence of how well they understand and use a sys tem. Can analysis of this trail lead to im proved OPAC use if changes are made in response to observed problems? The in formation an OPAC contains is as crucial as the way the information is presented to users. Introductory screens can affect the way an OPAC is perceived and there fore affect the success of its users. This study is an analysis and comparison of transaction logs within a six-month pe riod at a large public university. The Transaction Logs Task Force, composed of public and technical services librarians, was charged with reviewing OPAC trans action logs to identify librarywide prob lems and issues, monitoring and analyz ing online catalog use, and reporting any suggestions for improvement to the Inte grated Library Systems Advisory Com mittee. Members of the task force were asked to consider potential changes to screens, system features, and instructional programs. The task force analyzed trans action logs of the university’s OPAC alongside existing online user aids, evalu ated screen displays, and recommended changes to enhance the system’s user- friendliness. Deborah D. Blecic is the Bibliographer for Life and Health Sciences and an Assistant Professor at the University of Illinois at Chicago; e-mail: dblecic@uic.edu. Nirmala S. Bangalore is an Assistant Catalog Librarian and a Clinical Assistant Professor at the University of Illinois at Chicago; e-mail: nbangalo@uic.edu. Josephine L. Dorsch is an Assistant Health Sciences Librarian and an Assistant Pro fessor in the Library of the Health Sciences (Peoria) at the University of Illinois at Chicago; e-mail: jod@uic.edu. Cynthia L. Henderson is Director of the John A. Graziano Memorial Library and an Assis tant Professor at Samuel Merritt College; e-mail: clhndrsn@msn.com. Melissa H. Koenig is a Visiting Assistant Reference Librarian at the University of Illinois at Chicago; e-mail: mkoenig@uic.edu. Ann C. Weller is Deputy Director of the Library of the Health Sciences and an Associate Professor at the Univer sity of Illinois at Chicago; e-mail: acw@uic.edu. 39 mailto:acw@uic.edu mailto:mkoenig@uic.edu mailto:clhndrsn@msn.com mailto:jod@uic.edu mailto:nbangalo@uic.edu mailto:dblecic@uic.edu 40 College & Research Libraries Literature Review Transaction log analysis has been used as a tool to study user interaction with online catalogs since the late 1960s.1 A transac tion consists of a question or query by the user followed by an answer or response from the system. This paper defines trans action log analysis as the detailed and sys tematic examination of each search com mand or query by a user and the following database result or output by the OPAC. This paper defines transaction log analysis as the detailed and system atic examination of each search command or query by a user and the following database result or output by the OPAC. One traditional use of transaction logs has been the analysis of failure rates—in particular, the probable causes of patron failure. A 1989 transaction log analysis study at the University of Missouri-Kan sas City had three goals: “to determine failure rates, to study usage patterns and to investigate probable causes of patron failure when using the OPAC.”2 In this study, Thomas A. Peters reported high failure rates averaging around 40 percent. Misspellings and typographical errors ac counted for 20.8 percent of probable causes of patron failure; another 39.1 per cent was attributed to searches for items not held in the database. Rhonda N. Hunter ’s 1991 study of the success rate of patrons searching the online catalog at North Carolina State University showed a 54.2 percent overall failure rate. She defined failure as a zero postings search and concluded that patrons did have trouble searching the OPAC.3 Transaction log analysis, however, only views the transaction trail; it does not provide an overall picture of patron behavior. Martin Kurth suggested that bibliographic database additions and withdrawals make it all but impossible January 1998 for searches to be replicated over time, leading to problems with verification of transaction log studies.4 He pointed out that transaction logs cannot characterize individual users or user perceptions of the searches they perform. Moreover, he cau tioned that transaction logs need to be viewed as snapshots in time that permit comparisons of the same system as long as variables can be minimized and con trolled. Finally, he recommended that national and international standards for transaction log data be created to help understand OPAC use. Another traditional use for transaction log analysis has been as a management tool. Beth Sandore suggested using it to determine resource allocation, collection development, search patterns, effect of systems changes, and the addition of lo cal cross-references to database authority files.5 Peters also suggested that transac tion log analysis can be used as a man agement tool, noting that “the results of transaction log analysis often challenge management’s mental model of how the automated library system does or should work.”6 The performance of the online system and the success of its users can be communicated through transaction log analysis to those in a position to make li- brary-wide management decisions. Bib liographic instruction modification is yet another application for transaction log analysis. Transaction log analysis can help to pinpoint what concepts should be em phasized in bibliographic instruction classes as well as online tutorials.7 In 1995, Jane Scott, Jeffrey A. Trimble, and L. Fleming Fallon studied the ben efit of making changes in the OPAC and the screens to affect the success rate of searches performed.8 Library of Congress Subject Heading (LCSH) authority records were loaded into the OPAC, key word searching was promoted by plac ing the keyword directions at the begin ning of the introductory screen, and the default operator was changed from “near” to “and” to expand retrieval. These U sin g T ran saction L og A n alysis 41 an d th at th is h as n o t im p ro v ed w ith tim e. 10 Sh e stated th at alth ou g h q u eries m ay b e easier to in p u t th an th ey w ere a few y ears ag o, th ere ap p eared to b e v ery little ev id en ce th at stu d ies of search in g b eh av ior h ad b een ap p lied to b u ild b et ter on lin e catalog s. M eth o d o lo g y T h e task force stu d ied th e su ccess rate of O PA C u sers an d tested w h eth er th e su c- FIGURE 1 Transaction Log Sample Time View Search Postings Resulting Screen of query Database mode* modet received screen type numbert Character string input by user 08:16:04 UI BR FIN K 0 NEF1K 111 K=ROSIE THE RIVETER 08:16:12 UI BR INVLD CMND NEF1K 111 ENOLA GAY 08:18:06 UI BR FIN A 0 NEF1A 111 A=MARK HANDLEY 08:40:10 UI BR EXP T E1T 112 EXP T 08:41:53 UI BR FIN T 0 NEF1T 111 T=THE COUNSELING PSYCHOLOGIST 08:42:49 UI BR FIN A 2 A1I A=HELMS JANET 08:44:31 UI BR FIN T 1 BR1V 111 T=JOURNAL OF COUNSELING PSYCHOLOGY 08:53:27 UI BR FIN S 0 NEF1S 111 S=LATINO IDENTITY 08:53:40 UI BR FIN K 4 K1I K=LATINO IDENTITY 08:54:01 UI BR DIS # BR1V 111 1 09:07:25 UI LO FOR LO1V 212 09:08:18 UI LO INVLD CNTX LO1V 212 GUIDE 09:08:26 UI LO BAC LO1V 112 09:30:29 UI BR MSG SRCH ARG LBC1INT 111 K * Catalog view mode: BR (brief view) in NOnIS provides the user the following information: author (if main entry), title,name of publisher, date of publication, location, call number, and circulation status. In addition to all the data in the brief view, LO (long view) in NOnIS provides the user all information contained in the bibliographic record with appropriatelabels (e.g., added entries for authors have the following label: For other items by author(s) type A = � �)t Search mode (command issued): LUIS users can query the database by title (t=), author (a=), LC subject (s=), medical subject (sm=), call number (c=), or keyword (k=).t In the above log, 111 represents screen 1 of a one-screen display. ch an g es resu lted in a 4.6 p ercen t in crease in th e k ey w ord search su ccess rate. In a 1 9 9 5 stu d y th a t ex a m in ed k ey w o rd search in g , Joy T illotson fou n d th at k ey w ord search in g w as lik ely to y ield m ore resu lts, alth ou g h th e stu d y d id n ot in d i cate w h eth er u sers w ere m ore satisfied w ith th e resu lts. 9 A 1996 stu d y b y C h ristin e L . B org m an rep orted th at p atron s h av e con sid erab le d ifficu lty w h en search in g on lin e catalog s 42 College & Research Libraries January 1998 cess rate could be improved by making changes in the introductory screens. It analyzed NOTIS transaction logs from the eight busiest ports during a four-day pe riod in the middle of the academic term in the fall of 1995. Midterm was chosen as a representative period of use because by this point students had had time to become familiar with the OPAC. The task force was interested in examining “nor mal” use. Each public service computer in all eight university libraries is hardwired to the NOTIS system. Because the first available port is engaged at log on, it was impossible to track use of a par ticular computer or activity within a par ticular library. Thus, the resulting transaction logs represented use of the busiest ports. Each line of the NOTIS transaction log represents one transaction, as shown in figure 1, providing a wide range of infor mation about each transaction, including time of query, database name, catalog view mode, search mode, validity of commands, number of postings received, screen type, number of postings dis played, and the character string input by the user. The NOTIS system requires OPAC users to select a basic search strat egy: author, title, keyword, call number, or subject. If users select a subject search, they must further decide if they need an LCSH or an MeSH (medical subject head ing). Analysis of the transaction logs revealed that many users experi enced difficulty with basic searching techniques. Both correct and incorrect OPAC use was measured for each set of transaction logs. Counts were made of the following: number of transactions; number of search statements with correct syntax; number of correct search statements re sulting in zero postings; number of cor rect search statements resulting in one to ninety-nine postings; number of correct search statements resulting in more than a hundred postings; number of uses of ex plain commands; number of invalid com mands; number of invalid context; num ber of search statements that were missing the search argument; number of keyword searches; and number of “redirects” for author, subject, and title. The search statements resulting in zero postings were further analyzed. The to tal number of correct search statements resulting in zero postings was subdivided by: incorrect spelling; subject wrong or not in the catalog; title wrong or not in the cata log; initial article from the title included in the search strategy; author wrong or not in catalog; author in incorrect order; retry of the exact same search strategy; and un successful keyword search. The following types of transactions were not recorded or counted: time of search, use of the full OPAC or a subset of it, number of retrieved items displayed, or selection of brief or long view display. Analysis of the transaction logs re vealed that many users experienced dif ficulty with basic searching techniques. The introductory screens were evaluated with careful attention given to the types of problems identified by the transaction log analysis. Members of the task force concluded that changes to the introduc tory screens might help users conduct more successful searches. Introductory screens from other universities using the NOTIS system were reviewed. The task force made a series of recommendations, which included simplifying and clarify ing wording on introductory screens, having fewer words on each introductory screen, using the same publication for all examples, using a publication that con tained both LCSH and MeSH, and mov ing more specific explanations to a sec ond screen. The new OPAC introductory screens were designed to promote key word searching by moving the instruc tions for the keyword command to the top of the list. Using Transaction Log Analysis 43 FIGURE 2 Original Introductory Screen Welcome to the LUIS UICCAT, The Library Catalog of the University of Illinois at Chicago! You can use this catalog to identify books, journals, and other materials in the UIC collections. You may begin a search from any screen. TO SEARCH BY: TYPE: EXAMPLE: Author A= a=king stephen Title T= t=catcher in the rye LC Subject Heading S= s=labor supply Medical Subject Heading SM= sm=nursing diagnosis Call Number C= c=qa76.6 Keyword K= k=walker and purple To find items in particular buildings, use the SET CAT command and look for index line numbers marked with >. Type CHOose at any time to return to the Navigator Menu. For library news, type NEWS. ------------------------------------------------------------------ Page 1 of 1 ----------------- Prior to screen redesign, the examples on the introductory screen were drawn from different bibliographic records. Feel ing that search examples from the same bibliographic record might be more meaningful, the examples were drawn from a popular title on OCLC’s list of top one hundred monographs owned by member institutions. The title selected, The Hot Zone by Richard Preston, was con sidered a good example because it begins with an article and has both LCSH and MeSH. In addition, specific examples on the introductory screens were designed for specialized libraries within the uni versity, namely the Architecture and Art, Mathematics, and Science libraries. All recommendations were approved by the Integrated Library Systems Advi sory Committee. After notification of the public services staff, the changes were made to the introductory screens. Figure 2 shows the original introductory screen used by the Main and Health Sciences li braries, and figure 3 shows the changes made as a result of the transaction log analysis. Three weeks after the introductory screens were changed, a second set of transaction logs was run, using the same parameters as the first set, to determine if these changes resulted in greater search success. This second set also was run dur ing the middle of an academic term (spring 1996) to achieve comparable re sults. It was thought that any improve ment in searching techniques might then be attributed to the redesign of the screens. The second set of transaction logs was analyzed using the same methodol ogy as the first set. Results and Analysis The results of the two periods of transac tion logs sampling are detailed in table 1. The table includes the total number of transactions, the total number of search statements, and the items that were ex pected to be influenced by the screen changes. The total number of transactions was a figure provided at the end of each transaction log which included all patron inputs at the terminal: searches, invalid commands, navigational instructions (for ward and back), screen view changes (brief and long views), and exit com mands. The total number of search commands was derived from the number of searches with correct syntax plus those searches -- 44 College & Research Libraries January 1998 FIGURE 3 Revised Introductory Screens Welcome to UICCAT UICCAT is the computerized catalog of materials held by the UIC Libraries. You may begin a search from any screen. TO SEARCH BY: FOR EXAMPLE TYPE: TO GET HELP TYPE: Keyword k virus and animals exp k <ENTER> Author (last name first) a preston richard exp a <ENTER> Title (skip initial article) t hot zone exp t <ENTER> LC Subject Heading s ebola virus disease exp s <ENTER> Medical Subject Heading sm ebola virus exp sm <ENTER> Call Number c rc140.5 exp c <ENTER> For more information on UICCAT, press <ENTER> Press <F3> to exit. --------------------------------------------------------------------- Page 1 of 2 -------------- Welcome to UICCAT (cont.) CHO command If you entered from the Navigator Menu, type CHOose at any time to return to that menu. To select another CIC Library, type CHO and the name of the library catalog. EXAMPLE: CHO NUCAT (This would take you to Northwestern's catalog) SET CAT Command To find items in particular buildings, use the SET CAT command and look for index line numbers marked with >. NEWS Command For library news, type NEWS You may begin a search from any screen. Press <F3> to exit UICCAT. that failed because the search arguments were missing. This number was the most accurate indication of the total number of attempted searches and did not require second-guessing the patron’s intention based solely on the transaction log record. It is possible that some of the other error messages that patrons received, such as “invalid command” and “invalid con text,” resulted from a failed search at tempt, but analysis of these searches proved difficult without benefit of patron interviews. From the available data, it was ex pected that the changes to the screens might influence the number of times each of the following occurred: search state ments with correct syntax; search state ments missing the search argument; use of the explain command; searches result ing in zero postings; title searches that included an initial article; author searches in proper order of last name first; and use of keyword searching. To test whether the changes between the two sets of data were significant, a z-test Using Transaction Log Analysis 45 for the equality between two proportions (binomial distribu tion) was used. All the changes listed below were statistically significant to the .0005 level us i n g a o n e - t a i l e d t e s t . T h e change in correct syntax and missing search argument was calculated as a percentage of the total number of searches attempted. Use of the explain command was determined as a percentage of the total of all t r a n s a c t i o n s r e c o r d e d . A l l other categories reflect a per centage of the total number of correct syntax searches. The percentage of search statements with correct syntax increased from 96.45 to 97.68 per cent between the first and second sets of transaction logs. The dif ference was statistically signifi cant (z = -3.76; p < .0005). As the percentage of search statements with correct syntax increased, the percentage of those that failed because they were missing the search statement decreased from 3.55 to 2.32 percent, also a statistically significant difference (z = 3.78; p < .0005). The percentage of time the ex plain command was used, al though still a relatively small number, increased fivefold be tween the two sets of transaction logs. Information on using the explain command had been added to the introductory screens when the screen changes were made. Prior to its inclusion, this command would not have TABLE 1 Transactions Analyzed First Second Set Set Total number of transactions Total number of explain commands Percentage of transactions that were explain commands (%) Total number of search statements Total number of correct syntax searches Percentage of search statements that were correct syntax searches (%) Total number of commands missing the search argument Percentage of search statements missing the search argument (%) Total number of correct syntax searches resulting in zero postings Percentage of correct syntax searches resulting in zero postings (%) Total number of correct syntax searches with initial article included Percentage of correct syntax searches with initial article included (%) Total number of correct syntax searches with author in incorrect order Percentage of correct syntax searches with author in incorrect order (%) Total number of keyword searches Percentage of correct syntax searches that were keyword searches (%) 39421 20585 15 41 0.038 0.199 7204 4605 6948 4498 96.45 97.68 256 107 3.55 2.32 2435 1410 35.05 31.35 100 27 1.44 0.60 71 17 1.02 0.38 924 712 13.30 15.83 been easily found by OPAC users need- The number of correct syntax title ing online assistance. Use of this com- searches that failed because the initial ar mand increased from .038 to .199 percent ticle (a, an, the) was included decreased of all transactions after the introductory from 1.44 to .60 percent after the introduc screens were changed. This difference tory screens were changed, a statistically was statistically significant (z = -6.14; p < significant difference (z = 4.19; p <.0005). .0005). This decrease may have resulted from the 46 College & Research Libraries January 1998 TABLE 2 Reasons for Zero Postings First Set Second Set Total % Total % Z Values Incorrect spelling 259 10.64 129 9.15 na Subject wrong or not in catalog 560 23.00 217 15.39 5.66 Title wrong or not in catalog 847 34.78 597 42.34 na Initial article included in title search 100 4.11 27 1.91 3.68 Author wrong or not in catalog 204 8.38 211 14.96 na Author search done in incorrect order 71 2.91 17 1.21 3.40 Exact retry of previous search 182 7.47 64 4.54 na Unsuccessful keyword search 212 8.71 148 10.50 -1.84 Total number of zero postings searches 2,435 1,410 inclusion of clearer examples and instruc tions on the introductory screen. Author searches in the NOTIS system require that the last name be entered first. The percentage of correct syntax author searches that failed because the author ’s name was not entered in reverse order decreased from 1.02 to .38 percent after the introductory screen was changed to include clearer examples and instruc tions. The percentage decrease was a sta tistically significant difference (z = 3.83; p < .0005). The keyword example was moved to the top of the list of searching options in the revised introductory screens. Use of keyword searching increased from 13.30 to 15.83 percent of correct syntax searches, a statistically significant difference between the two sets of data (z = -3.78; p < .0005). The per centage of correct syntax searches that resulted in zero postings decreased from 35.05 to 31.35 percent. The decrease between the two sets of data was found to be statistically significant (z = 4.09; p< .0005) and might be attributed to a number of screen changes: inclusion of explain commands on the introductory screen, movement of keyword to the top of the list of commands, and clarification of directions for the exclusion of initial articles in title searches or the order of en try for author searches. Searches that resulted in zero postings were analyzed in detail (see table 2 and figure 4). The decreases in the percentages of zero postings between the two sets of data due to subject wrong or not in cata log, initial article included, and author in incorrect order were statistically signifi cant differences to the .0005 level, one- tailed. These decreases might be attrib uted to the introductory screen changes. The increase in the percentage of zero postings due to unsuccessful keyword searches also was a statistically significant difference to the .05 level, one-tailed, which might be attributed to the new emphasis on keyword searching. Though the percentages of zero postings due to incorrect spelling, incorrect titles or au thors, and retries of searches changed, Using Transaction Log Analysis 47 F IG U R E 4 R ea so ns fo r Z er o P os ti ng s 05 1 0 1 5 2 0 2 5 3 0 3 5 4 0 4 5 In co rr e ct sp el lin g S ub je ct w ro ng or n ot in ca ta lo g T itl e w ro ng o r no t i n ca ta lo g In iti a l a rt ic le in cl ud e d in ti tle se ar ch A ut ho r w ro ng or n ot in ca ta lo g A u th or s ea rc h do ne in in co rr ec t or d er E xa ct r e tr y o f p re vi ou s se ar ch U ns u cc es sf ul ke yw or d se a rc h R ea so n s P ercentage 1s t se t p er ce nt ag es 2n d se t p er ce nt ag es these changes could not be attributed to the introductory screen revisions except as an artifact of the changes in other cat egories and thus were not analyzed for statistical significance. Discussion The Transaction Logs Task Force studied transaction logs alongside existing online user aids and implemented changes to achieve simplified and more user-friendly introductory screens within the space limi tations of NOTIS-based OPACS. The task force followed Wallace’s advice: “Success ful screen redesigns and search engines should focus first and foremost on meet ing the quick-searching needs of the ma jority of users. By simplifying the routine and technical aspects of searching, de signers can eliminate or at least reduce 48 College & Research Libraries January 1998 many of the more common searching er rors.”11 The data show that simply chang ing the introductory screens significantly impacted user search success. Prior to screen redesign, the introduc tory screen featured search command keys separated by several spaces from the actual examples. The analysis of transac tion logs gave the impression that users mistakenly believed that choosing a search key would automatically put them into a certain index. The error message “Missing search argument” occurred as a result of users inputting search keys with no following search strings. After the screen changes, incidence of this error decreased from 3.55 to 2.32 percent of all search statements. The changes had a positive effect on searches with correct syntax. In the first set of data, 96.45 per cent of searches had correct syntax; in the second set, the percentage rose to 97.68 percent. After keyword became the first instead Searching is a complex process, and OPAC users are heterogenous in their information needs, online behavior, and searching skills. of the last search option on the introduc tory screens, keyword searching in creased from 13.30 to 15.83 percent of all search statements. Keyword searching can be useful to OPAC searchers, particu larly when attempts to match controlled vocabulary result in zero postings. A 1983 survey by Joseph R. Matthews, Gary S. Lawrence, and Douglas K. Ferguson found that although OPAC users have problems formulating subject searches, 59 percent of all searches were for subject information.12 As OPACs matured and transaction log analysis became more common, Ray R. Larson found a steady decline of about 2.2 percent a year in sub ject searching, being supplanted by title keyword searching.13 Such changes may be due in part to screen redesigns similar to those at this library. Transaction log analysis revealed that two common errors were incorrect order for an author search and use of initial ar ticles for a title search. Hints on avoiding both these errors were placed in promi nent view on the introductory screens. Such visual reminders on avoiding zero postings may have helped reduce the number of incorrect author and initial article searches. The plaguing problem of searches re sulting in zero postings is a recurring theme in literature on transaction log analysis. Pauline A. Cochrane and Karen Markey reported that OPAC users re trieve nothing twice as often (30%) as they retrieve too much (15%).14 In studying patron transaction logs from the Univer sity of Nevada’s WolfPac, Steven A. Zink found 27.81 percent zero postings out of a total of 6,118 searches.15 Xirong Shi’s thesis examined the design of the University of Toronto’s OPAC, FELIX, finding that ap proximately 29 percent of searches resulted in the message “There were no matches found for your choice.”16 In comparison, the present study found that 35.05 percent of correct syntax searches resulted in zero postings in the first set of data, whereas the second set contained 31.35 percent zero postings searches. These findings suggest that users became more efficient OPAC searchers with the help of screen redesign. Today’s OPAC is a do-it-yourself un mediated tool. Precise, easy-to-under stand online help is a necessity for both on-site and remote users. Searching is a complex process, and OPAC users are het- erogenous in their information needs, online behavior, and searching skills. Nevertheless, judging from transaction logs in this study, the effort to improve online assistance to OPAC users appeared to be beneficial. The efforts of the public services departments in publicizing screen redesign and promoting use of keyword searching in bibliographic in struction sessions also contributed to http:searches.15 http:searching.13 http:information.12 Using Transaction Log Analysis 49 more effective OPAC use. Conclusion Although transaction log analysis has come a long way since first-generation OPACs, researchers have shown that this popular tool has limitations. In 1994, Brendan J. Wyly investigated OPAC searchers’ judgments about the relevance of search results. In weighing user success, he cautioned: “Online catalogs are commu nication devices that allow searchers to in teract with a database. We must analyze them as communications facilitators.”17 The task force found that evaluating the introductory screens and implementing changes in response to errors observed in the transaction logs resulted in changed searching behaviors and improved user success. The simplification and clarification of wording on the introductory screens, avoidance of jargon, and rearrangement of the order of search options appeared to be the changes responsible for improving re trieval. A process of transaction log analysis was established and a methodology set in place to review the transaction logs regularly and to monitor the effectiveness of any subsequent screen changes. This model proved to be a valuable tool that could be adapted by other institutions. Although individual institutions will have different OPACs, clienteles, organi zational settings, institutional missions, and a host of other variables, this struc tured approach to monitoring OPAC use and responding to users may serve as a model for other libraries. The transaction logs will be monitored continuously and changes made as needed. The analysis of the second set of logs already has resulted in a series of changes to the explain screens. The rec ommendations included reducing the number of screens, using standard lan guage, and simplifying wording. Because these changes were substantial, there were a number of discussions with pub lic services staff prior to presenting the recommendations to the Integrated Li brary Systems Advisory Committee for approval. The committee again approved the revisions, and changes to the explain screens were implemented. Further re search will center on the impact of the new explain screens, the instructional as pects of OPAC use, collection develop ment opportunities arising from transac tion log analysis, exploration of natural language versus controlled vocabularies, and examination of data yielded from the emerging Web environment. NOTES 1. Thomas A. Peters, “The History and Development of Transaction Log Analysis,” Library Hi Tech 11, no. 2 (1993): 41–66. 2. ———, “When Smart People Fail: An Analysis of the Transaction Log of an Online Public Access Catalog,” Journal of Academic Librarianship 15, no. 5 (1989): 267–73. 3. Rhonda N. Hunter, “Successes and Failures of Patrons Searching the Online Catalog at a Large Academic Library: A Transaction Log Analysis,” RQ 30 (spring 1991): 395–402. 4. Martin Kurth, “The Limits and Limitations of Transaction Log Analysis,” Library Hi Tech 11, no. 2 (1993): 98–103. 5. Beth Sandore, “Applying the Results of Transaction Log Analysis,” Library Hi Tech 11, no. 2 (1993): 87–97. 6. Thomas A. Peters, “Using Transaction Log Analysis for Library Management Informa tion,” Library Administration and Management 10 (winter 1996): 20–25. 7. Patricia M. Wallace, “How Do Patrons Search the Online Catalog When No One’s Look ing? Transaction Log Analysis and Implications for Bibliographic Instruction and Design,” RQ 33 (winter 1993): 239–52. 8. Jane Scott, Jeffrey A. Trimble, and L. Fleming Fallon, “@*&#@ This Computer and the Horse It Rode In On: Patron Frustration and Failure at the OPAC,” in Continuity and Transforma tion: The Promise of Confluence: Proceedings of the ACRL 7th National Conference, (Chicago: ACRL, 1995): 247–56. 50 College & Research Libraries January 1998 9. Joy Tillotson, “Is Keyword Searching the Answer?” College & Research Libraries 56 (May 1995): 199–206. 10. Christine L. Borgman, “Why Are Online Catalogs Still Hard to Use?” Journal of the Ameri can Society for Information Science 47 (July 1996): 493–503. 11. Wallace, “How Do Patrons Search the Online Catalog When No One’s Looking?” 249. 12. Joseph R. Matthews, Gary S. Lawrence, and Douglas K. Ferguson, Using Online Catalogs: A Nationwide Survey (New York: Neal-Schuman, 1983). 13. Ray R. Larson, “The Decline of Subject Searching: Long-Term Trends and Patterns of In dex Use in an Online Catalog,” Journal of the American Society for Information Science 42 ( Apr. 1991): 197–215. 14. Pauline A. Cochrane and Karen Markey, “Catalog Use Studies Since the Introduction of Online Interactive Catalogs: Impact on Design for Subject Access,” in Redesign of Catalogs and Indexes for Improved Online Subject Access: Selected Papers of Pauline A. Cochrane (Phoenix, Ariz.: Oryx Pr., 1985), 159–84. 15. Steven A. Zink, “Monitoring User Success through Transaction Log Analysis: The WolfPAC Example,” Reference Services Review 19 (spring 1991): 49–56. 16. Xirong Shi, “Usage of the Online Library Catalogue at the University of Toronto: An Analy sis of Computer Monitoring Data” (master ’s thesis, University of Toronto, 1990), 58. 17. Brendan J. Wyly, “From Access Points to Materials: A Transaction Log Analysis of Access Point Value for Online Catalog Users,” Library Resources and Technical Services 40 (July 1996): 211– 36.