Mobile Access to Clinical Information at the Point of Care 1 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 8(3):e197, 2016 OJPHI Mobile Access to Clinical Information at the Point of Care Fatima M. Mncube-Barnes1, EDD, MPH, MSIS, Ben Lee2, BS, Olumuyiwa Esuruoso2, M.D., F.A.C.P., Phil N. Gona3, PhD, MPH, MSc, Stephane Daphnis2, MBA 1. Louis Stokes Health Sciences Library, Howard University 2. Meharry Medical College, Nashville, TN 3. University of Massachusetts, Boston Abstract Objectives: Using library subscriptions and accessible on handheld devices, this study sought to promote authoritative health information apps, and evidence-based point-of-care resources. Methods: Three cohorts of internal medicine residents were issued iPads at the beginning of their second year, and were trained to skillfully access resources from the digital library. Pre- and post- intervention surveys were respectively administered at the beginning of the second year and end of the third year of training. The residents’ computer experience and computer knowledge was assessed. Additionally, before and after formal introduction to iPads, perceptions on the use of computers to access clinical information were assessed. Survey responses were compared using two sample methods and summarized through descriptive statistics. Results: Sixty-eight residents completed the pre-survey questionnaires and 45 completed the post- surveys. There were significant improvements in the residents’ level of computer experience, and familiarity with medical apps. Furthermore, there was increased knowledge obtained in accessing clinical information through electronic medical records. Residents positively perceived the potential effects of computers and electronic medical records in medicine. Conclusion: Study findings suggested that health science libraries can be instrumental in providing search skills to health professionals, especially residents in training. Participants showed appreciation of iPads and library support that facilitated successful completion of their related tasks. Replicating this study with a larger sample derived from multiple sites is recommended for future studies. Participation of mid-level healthcare professionals, such as Physician Assistants and Nurse Practitioners is suggested. Keywords: Active Learning; Diagnostic Reasoning; Technology Integration; point-of-care resources http://ojphi.org/ Mobile Access to Clinical Information at the Point of Care 2 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 8(3):e197, 2016 OJPHI Correspondence: fatima.barnes@howard.edu DOI: 10.5210/ojphi.v8i3.7099 Copyright ©2016 the author(s) This is an Open Access article. Authors own copyright of their articles appearing in the Online Journal of Public Health Informatics. Readers may copy articles without permission of the copyright owner(s), as long as the author and OJPHI are acknowledged in the copy and the copy is used for educational, not-for-profit purposes. Introduction With the tremendous ongoing improvements in both portable hardware and information connectivity, mobile technology has become an essential and ubiquitous component of modern healthcare [1]. Mobile devices coupled with advanced health information technology, either by direct or wireless download, can make vital medical information accessible to medical practitioners at the point of care for medical education and delivery [2]. This makes mobile devices powerful in the clinical settings. Mobile access to medical information facilitates “just-in-time” learning at the point of care, where learning “…is time- and place-independent and results in the functional use of information.” [3] Mobile access to medical information allows for a novel method of experiential and self-directed learning. Furthermore, knowing precisely where specific information is located digitally is likely to save lives at the point-of-care through well-informed decision-making. Mobile devices have shown great promise not only for medical students and residents, but also they have the potential to empower medical educators [4-6]. By supplementing the medical information base of students, mobile devices can allow educators to more effectively train and “…assess [student] performance and competence at the highest levels of Miller’s Pyramid of Clinical Competence, thereby reflecting real-world practice.” [3] The four sequential stages of Miller’s Pyramid, from base to apex, include: “Information”; “knows how”; “Shows how”; and “Does.” Although mobile access to clinical information is a relatively new field, there exists a wealth of professional medical applications (apps) for mobile devices of varying content, purposes, and delivery, accessible across all mobile device platforms, both freely and commercially available. It can be expected that the medical apps ecosystem will continue to grow and develop substantially [1]. Furthermore, the future of healthcare computing is expected to become rooted in mobile technologies [7-9]. The objective of this study was to better understand how the introduction of mobile access to clinical information at the point of care is beneficial to medical resident physicians. Methods Medical librarians and the Internal Medicine residency coordinators conducted this study for three successive years (2012-2014). To raise awareness of point of care resources, free iPads were issued http://ojphi.org/ Mobile Access to Clinical Information at the Point of Care 3 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 8(3):e197, 2016 OJPHI as an incentive to promote the use of mobile devices in accessing specific authoritative resources pertinent to medical education. The residents’ computer experience and computer knowledge was assessed, along with computer-related attitudes regarding mobile access to clinical information, before and after formal introduction to iPads and point-of-care resources for clinical use. Three cohorts of second year internal medicine residents volunteered to participate and signed a participation contract. They were surveyed prior to intervention and were surveyed again at the end of the third year of training. The intervention comprised of equipping the residents with iPads and also providing them training and continued user support in the basic utilization of the iPad and point-of-care apps and resources. The Institutional Review Board approved the study. To formally introduce mobile access to clinical information, study coordinators gave each participant an iPad with point-of-care resources and apps without any cost to the participants. The medical librarians demonstrated the iPad’s clinical utility for accessing clinical information in the form of authoritative point-of-care resources available through the digital library and medical information apps on the iPad. Specifically, medical information apps introduced consisted of point-of-care resources from institutional subscriptions, authoritative public health apps from the National Library of Medicine, and general healthcare apps that were evaluated and approved by physicians for clinicians through imedicalapps.com. Comprehensive technical and resource support was provided to participants throughout the 18 months of the study. Manning and Gadd’s (2001) survey for evaluating handheld computing in a residency program was modified and customized for this study [10]. The survey questions were created with REDCap Survey™ software (Research Electronic Data Capture), (http://project-redcap.org) [11]. The REDCap software provided an intuitive interface using validated data entry; collecting, manipulation and export procedures; automated export procedures for seamless downloads to common statistical packages. Using the Likert-scale responses, residents were assessed on their familiarity with current computer technologies, opinions regarding the introduction of mobile technology within a clinical setting, and familiarity with medical information apps. Composite scores were computed by summing up individual Likert-scale item scores in each of the following domains: “Computer experience”, “Computer knowledge”, “Perceived necessary capabilities of computer systems in medicine”, “Familiarity with medical information apps”, “Potential effects of computers”, and “Appreciation for electronic medical record (EMR) on medicine.” To assess the effectiveness of the intervention, changes in composite scores were calculated as the post-intervention composite score minus the baseline composite score. A high composite score indicates high information for a domain while a large positive pre-post difference is suggestive of improved information attributable to the intervention. Tabulated summary data were stratified by phase, pre-intervention, and post-intervention. For continuous variables the mean difference of composite scores between the two phases was computed. Data was summarized using descriptive statistics (i.e., mean and standard deviation for continuous variables and percentages for categorical variables). Histograms were also constructed comparing pre- and post- intervention categorical responses. Percentage change in composite scores was calculated according to this formula: [(post/pre) -1]*100%. Since there were no unique ID numbers linking pre- and post-intervention responses, paired statistical methods could not be used to assess changes in scores pre- and post-intervention. Instead two-sample t-test and Chi-squared test were used to compare continuous and categorical variables, respectively. Statistical Applications Software http://ojphi.org/ Mobile Access to Clinical Information at the Point of Care 4 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 8(3):e197, 2016 OJPHI (SAS version 9.4) was used to analyze the data. A P-value of 0.10 was considered statistically significant. Results Sixty-eight participants, (38% women, mean age = 35.1, SD = 5.1 years) completed the baseline survey. Forty-five completed the post-intervention survey. Post-intervention, there were statistically significant positive increases in composite scores for participant computer experience (percentage change = 9.52%; P = .09) and familiarity with medical information apps for mobile devices (percentage change=50.59%; P < .0001) (Table 1). Participant appreciation for Electronic Medical Record (EMR), (mean percentage change = 4.10%; P = .10) (Table 1) increased after intervention. Specifically, there were significant increases in the belief that an electronic medical record would be beneficial for time required for documentation (percentage change = 14.29%; P = .04), time required to enter orders (percentage change = 12.90%; P = .03), and patient privacy (percentage change = 25.00%; P = .02). Furthermore, in the assessment of the perceived necessary capabilities of computers in medicine, it appeared that the residents value mobility (Figure 1-2) and access (Figure 3, 4, 5) associated with mobile access to clinical information at the point-of-care both before and after intervention. Discussion Using the intervention, we observed varying magnitudes of increases of pre-versus post- intervention composite scores (Table 1). Composite score differences for the domains of computer experience, familiarity with medical information apps, and appreciation for EMR attained statistical significance. Such increases suggest that formal institutional intervention involving access to mobile technology and its utilization in the clinical setting could benefit residents providing patient care. The value of formal institutional intervention became apparent with a closer examination of the large increase of 51% in familiarity with medical information apps among residents. In addition to demonstrating that residents were unfamiliar with medical information apps, it also suggested that valuable digital resources for mobile technology in the clinical setting were possibly underutilized. This study provided evidence that an intervention such as ours would potentially benefit residents in facilitating the process of becoming familiar with digital resources provided by their library. Additionally, residents should be made aware of evidence-based subscriptions with mobile apps that require log-ins. Familiarity and pre-registration to these resources would allow them quick access to point-of-care information at the bedside. It is unclear if the improvement observed in the domains of resident computer experience, familiarity with medical information apps, and appreciation for EMR would translate into enhanced graduate medical education or improved quality of care for patients. Such questions would be best addressed in a prospective design and a control group combined with meticulous assessment of patient outcomes before and after implementation. http://ojphi.org/ Mobile Access to Clinical Information at the Point of Care 5 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 8(3):e197, 2016 OJPHI Table 1: Composite Scores Composite Scores Pre Post Percentage Change P- value* N=68 N=45 mean(SD) mean(SD) Computer Experience 21 (6.3) 23 (6.0) 9.52% .09 Computer Knowledge 17.3 (5.7) 17.9 (5.7) 3.47% .58 Perceived Necessary Capabilities of Computer Systems in Medicine 64.6 (15.8) 67.6 (8.8) 4.64% .25 Familiarity with Medical Information Apps 8.5 (4.5) 12.8 (3.5) 50.59% <.0001 Potential Effects of Computers 51.9 (9.8) 51.2 (13.9) -1.35% .77 Appreciation for EMR 70.8 (12.3) 73.7 (15.4) 4.10% .10 *P-value obtained from paired t-test; **Percentage Change = (mean-post / mean-pre - 1)*100 Computer Experience was derived by summing up 17 items (possible range of values = 0 – 40) Computer Knowledge was derived by summing up 14 items (possible range of values = 0 – 28) Perceived Necessary Capabilities of Computer Systems in Medicine was derived by summing up 17 items (possible range of values = 0 – 68) Familiarity with Medical Information apps was derived by summing up 13 items (possible range of values = 0 – 26) Potential Effects of Computers was derived by summing up 17 items (possible range of values = 0 – 68) Appreciation for EMR was derived by summing up 22 items (possible range of values = 0 – 88) Further analysis of Table 1 revealed the relatively high baseline perception for some composite scores both before and after formal intervention, implied that residents had already deemed, prior to intervention, that a computer system must be highly capable for clinical use, and positively perceived both the potential effects of computers and EMR on medicine. With such relatively high baseline perceptions, no room for additional information improvement could be derived from this study. This apparent “ceiling effect” meant that no further intervention effect on information level was possible as a result of a prevalent high baseline information. This finding, in addition to the positive effects of intervention, suggested that the residents generally saw great value and potential in integrating reliable computer systems and EMR into the clinical setting. As has been previously reported, such a ceiling effect was often the result of constraints on data-gathering instruments such as the one used in this study. When a ceiling effect occurs in data-gathering, there is a bunching of scores at the upper level reported by an instrument [12]. Further analysis of the perceived necessary capabilities of computers in medicine, revealed that residents valued mobility and access afforded by smart phones and iPads/Tablets in a clinical setting. For mobility, residents appeared to value, both before and after intervention, the ability to access the computer system at any place (Figure 1). This apparent consensus seemingly conflicted http://ojphi.org/ Mobile Access to Clinical Information at the Point of Care 6 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 8(3):e197, 2016 OJPHI with the varied opinions toward interacting with a computer without the need of a keyboard as reported (Figure 2). Assessing both sets of data simultaneously, suggested that mobility did not necessarily coincide with “keyboardless” interaction with computers in a clinical setting. For access, residents greatly seemed to favor, both before and after intervention, a system that always responded to queries in less than five seconds (Figure 3), always displayed X-rays and other images in less than 30 seconds (Figure 4), and always functioned without any “down time” (Figure 5). Such data underscored the idea that speed and reliability were important to mobile access to clinical information. Figure 1: Figure 2: 70.6 11.8 8.8 1.5 7.4 57.8 33.3 8.9 0.0 0.0 0.0 20.0 40.0 60.0 80.0 Vitally Necessary Generally Necessary Somewhat Necessary Not Necessary Unable to Respond Pe rc en ta ge "I can access the system at any place in the clinical setting" Pre (N=68) Post (N=45) 10.3 23.5 30.9 27.9 7.4 15.6 24.4 28.9 26.7 4.4 0.0 5.0 10.0 15.0 20.0 25.0 30.0 35.0 Vitally Necessary Generally Necessary Somewhat Necessary Not Necessary Unable to Respond Pe rc en ta ge "I can interact with the computer without using a keyboard" Pre (N=68) Post (N=45) http://ojphi.org/ Mobile Access to Clinical Information at the Point of Care 7 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 8(3):e197, 2016 OJPHI Figure 3: Figure 4: 39.7 30.9 14.7 7.4 7.4 40.0 35.6 24.4 0.0 0.0 0.0 5.0 10.0 15.0 20.0 25.0 30.0 35.0 40.0 45.0 Vitally Necessary Generally Necessary Somewhat Necessary Not Necessary Unable to Respond Pe rc en ta ge "The system always responds to my queries in less than five seconds" Pre (N=68) Post (N=45) 42.6 30.9 10.3 5.9 10.3 46.7 44.4 6.7 0.0 0.0 0.0 5.0 10.0 15.0 20.0 25.0 30.0 35.0 40.0 45.0 50.0 Vitally Necessary Generally Necessary Somewhat Necessary Not Necessary Unable to Respond Pe rc en ta ge "The system always displays X-rays and other images in less than 30 seconds" Pre (N=68) Post (N=45) http://ojphi.org/ Mobile Access to Clinical Information at the Point of Care 8 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 8(3):e197, 2016 OJPHI Figure 5: Limitations One significant limitation, despite a modest sample size which caused the authors to use a liberal significance level of p<0.10 instead of p<0.05, is that the two-sample statistical analysis used was less optimal for pre- and post-intervention studies. A paired-sample analysis in which each participant serves as own control between the two waves would have been more ideal. However, due to the anonymous nature in which the study was administered, a unique identifier could not be used to link participant responses in the two waves. With no ID variable to link the wave responses for each participant, the characteristics of completers versus the non-completers could not be compared. Another limitation was the smaller number of participants who completed both phases. There was no means to determine whether completers represented a high level of enthusiasm relative to non-completers. Larger multi-institutional studies are recommended in which participant responses are linked in each wave to allow a paired analysis. Furthermore, our small sample size did not allow for multivariable adjustment for potential confounding due to age and gender. It is therefore possible that findings for this study could be attributed to residual confounding that could not be adjusted for multivariable regression analysis. Conclusion Regardless of the above weaknesses, this 18-month pilot study offered an encouraging glimpse into merits of formal introduction of mobile access to clinical information and how it impacted internal medicine residents’ computer-related attitudes and selected preferences regarding accessing clinical information using iPads. It was apparent that the introduction of mobile access to clinical information was well received by internal medicine residents at our institution because 45.6 22.1 19.1 4.4 8.8 35.6 40.0 15.6 6.7 2.2 0.0 5.0 10.0 15.0 20.0 25.0 30.0 35.0 40.0 45.0 50.0 Vitally Necessary Generally Necessary Somewhat Necessary Not Necessary Unable to Respond Pe rc en ta ge The system is always functioning. There is never any 'down time' Pre (N=68) Post (N=45) http://ojphi.org/ Mobile Access to Clinical Information at the Point of Care 9 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 8(3):e197, 2016 OJPHI it facilitated medical education and direct healthcare delivery. Even though participants were technology-literate, our findings suggest that institutions play a critical role by providing technology training to their resident physicians, especially with regard to accessing authoritative clinical information from library subscriptions. Depending on the size of institutions and associated allied health programs, health science libraries pay millions of dollars in subscriptions for resources. Skillfully searching and mastering different aspects of these resources can be challenging. It is recommended that future studies be conducted at multiple sites and participants should include mid-level healthcare professionals such as Physician Assistants and Advanced Nurse Practitioners. Conflicts of Interest The authors have no financial relationships relevant to this article. Disclosure The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of Meharry Medical College. This project was funded in part by Title III’s technology enhancement grant for the library. References: 1. Boulos MNK, Wheeler S, Tavares C, Jones R. 2011. How smartphones are changing the face of mobile and participatory healthcare: An overview, with example from eCAALYX. Biomed Eng Online. 10(1), 24-24. doi:http://dx.doi.org/10.1186/1475-925X-10-24. PubMed 2. Osheroff JA, Teich JM, Middleton B, Steen EB, Wright A, et al. 2007. A roadmap for national action on clinical decision support. Journal of the American Medical Informatics Association: JAMIA. 14(2), 141-45. doi:http://dx.doi.org/10.1197/jamia.M2334. PubMed 3. Lumsden CJ, Byrne-Davis LMT, Mooney JS, Sandars J. Using mobile devices for teaching and learning in clinical medicine. Archives of disease in childhood-Education & practice edition. 2015:edpract-2014-306620. 4. Chatterley T, Chojecki D. 2010. Personal digital assistant usage among undergraduate medical students: Exploring trends, barriers, and the advent of smartphones. J Med Libr Assoc. 98(2), 157-60. doi:http://dx.doi.org/10.3163/1536-5050.98.2.008. PubMed 5. Tews M, Brennan K, Begaz T, Treat R. 2011. Medical student case presentation performance and perception when using mobile learning technology in the emergency department. Med Educ Online. 16, 1-7. doi:http://dx.doi.org/10.3402/meo.v16i0.7327. PubMed 6. Kho A, Henderson LE, Dressler DD, Kripalani S. 2006. Use of handheld computers in medical education. J Gen Intern Med. 21(5), 531. doi:http://dx.doi.org/10.1111/j.1525- 1497.2006.00444.x. PubMed http://ojphi.org/ http://dx.doi.org/10.1186/1475-925X-10-24 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=21466669&dopt=Abstract http://dx.doi.org/10.1197/jamia.M2334 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=17213487&dopt=Abstract http://dx.doi.org/10.3163/1536-5050.98.2.008 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=20428281&dopt=Abstract http://dx.doi.org/10.3402/meo.v16i0.7327 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=22013378&dopt=Abstract http://dx.doi.org/10.1111/j.1525-1497.2006.00444.x http://dx.doi.org/10.1111/j.1525-1497.2006.00444.x http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=16704405&dopt=Abstract Mobile Access to Clinical Information at the Point of Care 10 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 8(3):e197, 2016 OJPHI 7. Howard DJ, Coovert SA, Coovert MD, Nelson RM. 2015. Tablet computing in clinical training of pediatric residents [doi]. Telemed J E Health. 21(7), 588-92. doi:http://dx.doi.org/10.1089/tmj.2014.0130. PubMed 8. Lehnbom EC, Adams K, Day RO, Westbrook JI, Baysari MT. 2014. iPad use during ward rounds: An observational study. Stud Health Technol Inform. 204, 67-73. PubMed 9. Bullock A, Dimond R, Webb K, Lovatt J, Hardyman W, et al. 2015. How a mobile app supports the learning and practice of newly qualified doctors in the UK: An intervention study. BMC Med Educ. 15(1), 71. doi:http://dx.doi.org/10.1186/s12909-015-0356-8. PubMed 10. Manning B, Gadd CS. Introducing handheld computing into a residency program: Preliminary results from qualitative and quantitative inquiry. AMIA Annual Symposium Proceedings. 2001:428-432. 11. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, et al. 2009. Research electronic data capture (REDCap)—A metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 42(2), 377-81. doi:http://dx.doi.org/10.1016/j.jbi.2008.08.010. PubMed 12. Cramer D, Howitt D. The SAGE dictionary of statistics: A practical resource for students in the social sciences. GB: Sage Publications Ltd; 2004. http://ojphi.org/ http://dx.doi.org/10.1089/tmj.2014.0130 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=25748271&dopt=Abstract http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=25087529&dopt=Abstract http://dx.doi.org/10.1186/s12909-015-0356-8 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=25889996&dopt=Abstract http://dx.doi.org/10.1016/j.jbi.2008.08.010 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=18929686&dopt=Abstract Mobile Access to Clinical Information at the Point of Care Introduction Methods Results Discussion Limitations Conclusion Conflicts of Interest Disclosure References: