Microsoft Word - ES_1791 Evidence Based Library and Information Practice 2008, 3:3 64 Evidence Based Library and Information Practice Evidence Summary The Utilization of Wireless Handheld Computers with MEDLINE is an Effective Mechanism for Answering Clinical Questions at the Point of Care A Review of: Hauser, Susan E., Dina Demner-Fushman, Joshua L. Jacobs, Susanne M. Humphrey, Glenn Ford, and George R. Thoma. “Using Wireless Handheld Computers to Seek Information at the Point of Care: An Evaluation by Clinicians.” Journal of the American Medical Informatics Association 14.6 (Nov./Dec. 2007): 807-15. Reviewed by: Martha Ingrid Preddie Doctoral Student, School of Information and Library Science University of North Carolina at Chapel Hill Chapel Hill, North Carolina, United States of America Email: mipreddie@gmail.com Received: 02 June 2008 Accepted: 31 July 2008 © 2008 Preddie. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Abstract Objective – To assess the effectiveness of wireless handheld computers (HHCs) for information retrieval in clinical environments and the role of MEDLINE in answering clinical questions at the point of care. Design – A prospective single-cohort study. Setting – Teaching rounds in the intensive care units and general medicine wards in two hospitals associated with a university’s school of medicine in the United States. Subjects – Five internal medicine residents with training in evidence-based practice. Methods – While accompanying medical teams on teaching rounds for approximately four consecutive weeks, each resident used MD on Tap (an application for handheld computers) on a TreoTM 650 PDA/cell phone to find answers in real time, to questions that were raised by members of the medical teams. Using a special version of MD on Tap, each resident initialized a UserID. Serving as evaluators, the residents described and categorized clinical scenarios and recognized questions. They also Evidence Based Library and Information Practice 2008, 3:3 65 formulated search terms, searched MEDLINE and identified citations determined to be useful for answering the questions. An intermediate server collected details of all MEDLINE search query transactions, including system response time, the user (based on UserIDs), citations selected for viewing, the saving of citations to HHC memory, as well as use of the Linkout and Notes features. In addition evaluators submitted daily summaries. These summaries included information on the scenarios, clinical questions, evidence- based methodology (EBM) category, the team member who was the source of the question, the PubMed Identifiers (PMIDs) of relevant citations, and comments. At the end of the data collection period, each evaluator submitted a summary report consisting of a qualitative and quantitative evaluation of his experience using MEDLINE via the handheld device to find relevant evidence based information at the point of care. The report also focused on the usefulness of MD on Tap features, along with suggestions for additional features. Data analysis encompassed matching the text of daily summaries to transaction records in order to identify sessions (containing a scenario, clinical question, one or more search queries, citation fetches and selected PMIDs). A senior medical librarian/expert indexer reviewed all the citations selected by evaluators and graded each citation as A (useful for answering the question), B (provided a partial answer) or C (not useful for answering the question). Only those graded A were regarded as “relevant.” For the purpose of analysis a session was deemed to be successful “if at least one of the citations selected by the evaluator as relevant was also classified as Relevant” (810) by the expert indexer. Similarly, an individual query was successful “if at least one of the citations among the results of the query was Relevant, that citation was viewed by the evaluator during rounds, and it addressed the clinical question as recorded in the daily summary” (810). Various types of relationships were analyzed including the characteristics of clinical questions vis-a-vis successful sessions, search strategies in relation to successful queries, and the association between MD on Tap features and successful queries. SAS/SUDAAN version 9.1 was used for statistical analysis. Main Results – Evaluators answered 68% (246 of 363) clinical questions during rounding sessions. They identified 478 “relevant” citations, an average of 1.9 per successful session and 1.3 for each successful question. Session lengths averaged 3 minutes and 41 seconds. Characteristics of the evaluator (training, interest, experience and expertise) were a significant predictor of a session’s success. The significant determinants of query success were “the number of search terms that could be mapped to Medical Subject Headings (MeSH)” (812), the number of citations that were found for a query, and the use of MD on Tap’s auto-spellcheck feature. Narrative comments from the evaluators indicated that using MEDLINE on a HHC at the point of care contributed positively to the practice of evidence -based medicine. Conclusion – Wireless handheld computers are useful for retrieving information in clinical environments. The application of several MeSH terms in a query facilitates the retrieval of MEDLINE citations that provide answers to clinical questions. The MD on Tap program is a valuable interface to MEDLINE at the point of care. Commentary This study investigates the usefulness of wireless handheld computers and the value of MEDLINE as a resource for finding Evidence Based Library and Information Practice 2008, 3:3 66 answers to point of care clinical questions. The methodology employed was a cohort study of five participants. Despite the small number of participants, the use of a prospective cohort study was suitable as the data collection method of choice. Additionally, the application of the research methodology and the data analysis processes were clearly detailed, thus allowing for replication of the study. Nevertheless, a major shortcoming of this research is that the study population was not totally representative of all eligible users of handheld computers. Inclusion and exclusion criteria were not stated explicitly. However, the selected participants were affiliated with a clinical elective in medical informatics and were all residents who had been trained in evidence-based practice. It is likely that this training predisposed them to being more adept at constructing and executing EBM-type searches of databases. This can be surmised from the fact that their MD on Tap Medline training averaged only 25 minutes. This prior training in evidence- based practice would have facilitated favourable results pertinent to the objectives of the study. This occurrence introduced some amount of bias into the study, and the possibility exists that the study outcomes could have been different if clinicians who were not residents, or physicians with longer years of practice who had not been trained in evidence-based practice, had also been included as evaluators. The researchers acknowledged that the small number of participants limited the strength of the study, since they were unable to statistically confirm the value of features such as the date limit and the clinical query hedge. They also admitted that the process of checking the primary literature to answer all questions (while on the rounds), produced an artificial situation that would have impacted the overall success rate. With regard to opportunities for further research, the researchers suggested an examination of the relationship between clinicians’ background and training and search success. Also indicated is a comparative research study assessing the searching of identical questions using the same information databases on a handheld computer versus a desktop. Given the inadequacies of the study emanating from the homogeneity of the study population and the small number of participants, both of which affect the external validity of this cohort study, a larger study with a more heterogeneous population is also warranted. This would render a higher level of generalizability. Despite its shortcomings this study is significant because it “is the first reported analysis of online information searching by clinicians using wireless HCCs in which search actions are automatically recorded and associated with both the questions being addressed and the perceived value of the search results” (814). The quantitative measures of search strategy and search success add tangible evidence to the literature on the use and value of wireless handheld computers, for information retrieval in clinical settings. The findings revealed that the evaluators’ “individual characteristics of training, experience, expertise or interest” (811) was the sole “significant predictor of a successful Session” (811). These results suggest that medical librarians need to continue to engage in user training, to build competencies among clinicians in the formulation of effective search strategies for the retrieval of evidence-based information. Evidence Based Library and Information Practice 2008, 3:3 67 With regard to the MEDLINE database, such training should highlight the use of multiple MeSH terms in the construction of search queries.