SHORT PAPER THE USE OF SMARTPHONES AND MOBILE CLINICAL DECISION SUPPORT SYSTEMS IN CLINICAL CLERKSHIPS The Use of Smartphones and Mobile Clinical Decision Support Systems in Clinical Clerkships A Pilot Study http://dx.doi.org/10.3991/ijim.v7i2.2446 AMR Schols, HHLM Donkers, M Voorend, DML Verstegen, H Hoogland, and PL Kubben Maastricht University Medical Center, The Netherlands Abstract—Smartphones and Mobile Clinical Decision Support Systems (MCDSS) could help to overcome informa- tion overload and support physicians in providing up-to- date medical care. However, would smartphones and MCDSS also be useful for medical students during their clinical clerkships? In this prospective pilot study eight students were asked to use a smartphone during their daily clerkship activities and specifically to evaluate a basic MCDSS. Students were asked to fill out a short survey and exit interviews were conducted. Most students found the smartphone, especially access to the internet, a useful tool during their clerkship. Some students thought that the internet was an easier accessible information source com- pared to many applications. Several students were not aware of any high-quality applications and most students were not willing to pay more than ten Euros for such an application. In the opinion of most students smartphones should not be used in the presence of a patient. Students expressed a desire for more basic content in the tested MCDSS. In conclusion, smartphones can be a useful tool for medical students during their clerkship. MCDSS for medi- cal students should be designed to their needs. Further research is needed to guide MCDSS development specifi- cally targeted at medical students. Index Terms—Decision support, E-learning, Mobile, PDA, smartphone I. INTRODUCTION Medicine has changed over the past century. There has been an exponential growth of research and knowledge [1- 2]. The volume of medical papers published almost doubles every decade [3]. Nowadays, PubMed cites more than 18 million papers [4]. The “gap of knowledge”, the gap between what we can learn and what is known, is increasing constantly. Keeping up with the literature has already become an impossible task [5]. Literature alert services, good evidence-based resources and real-time decision support systems can help to overcome this information overload [6]. The use of computer systems that can aid clinical deci- sion making is growing. Clinical Decision Support Sys- tems (CDSS) could generally be described as computer applications designed to aid physicians in making diag- nostic and therapeutic decisions [7]. However, CDSS could also be defined as: “providing clinicians or patients with computer-generated clinical knowledge and patient- related information, intelligently filtered or presented at appropriated times, to enhance patient care [8]. According to Payne [7] there is substantial evidence from trials in a wide range of clinical settings that CDSS may help physicians to provide better clinical care to their patients and according to Garg et al.[9] many CDSS improve practitioners performances. With the integration of cellular phone technology and additional hardware into Personal Digital Assistants (PDAs), the PDA has evolved into the smartphone [10- 11]. The medical community could also benefit from this evolution [11]. Smartphones have the potential to become an important tool in providing up-to-date medical care: from having access to the latest medical research at the point of care [12] to decision support systems that can help to overcome the information overload [6]. In 2009, 64% of US physicians owned a smartphone, and this number is predicted to rise to 81% by the end of 2012 [13]. An increasing number of Mobile Clinical Decision Support Systems (MCDSS) is available to support physi- cians in providing up-to-date medical care, mostly as applications or so-called “apps” for modern smartphones. These apps are usually developed for physicians. Would they also be useful for medical students who are at the point of becoming a medical professional? Medical students also face the problem of information overload: they are expected to have basic up-to-date knowledge in all fields of medicine. However, they often lack the medical knowledge that is necessary to fully understand how to use a MCDSS made for physicians with far more background in a certain medical specialty. Therefore, medical students may have different preferences for a MCDSS than physicians. The study described in this paper was executed in the context of the clerkship neurol- ogy. This lead to the following questions: how do medical students in their clerkship neurology experience the use of a smartphone with a basic MCDSS made for physicians, and what other use has a smartphone for medical students in their clerkship neurology? To our knowledge this is the first prospective study that specifically addresses the needs of the medical student in MCDSS. Furthermore, the results of this pilot study will be used to improve the tested MCDSS. II. MATERIALS AND METHODS In this prospective pilot study eight students were equipped with an iPhone during their clerkship neurology. During this period the students were allowed to use the iPhone for work, study and private activities. All iPhones were equipped with unlimited internet use, a limited 80 http://www.i-jim.org SHORT PAPER THE USE OF SMARTPHONES AND MOBILE CLINICAL DECISION SUPPORT SYSTEMS IN CLINICAL CLERKSHIPS amount of talk time and text messages, and a selection of pre-installed applications (Table 1). One of the pre- installed applications about neurology and neurosurgery, was the application NeuroMind (version 1.3), developed by the last author. NeuroMind is a simple MCDSS that offers quick access to frequently used scores (Figure 1) that can help in decision making. Additionally it contains anatomical images for explanation to patients and students (Figure 2). Upon distribution of the iPhones students were asked to use the smartphone during their daily clerkship activities and specifically to evaluate the application NeuroMind. Furthermore, students received a short manual in which some functions of the iPhone were explained. At the end of their clerkship students were asked to fill out a short survey. The survey consisted of 20 items (17 multiple choice questions and 3 open questions); (1) about the content, sort of use, and ease of use of NeuroMind, (2) open questions focusing on which elements students would like to keep or have changed in future versions of NeuroMind, (3) about acceptability of apps in the clinical setting, how much students would like to pay for applica- tions, and which functions students used for their educa- tion. After filling in the survey the students were invited for an exit-interview of about 40 minutes, which was conducted with two students per interview and consisted of open questions and some in-depth questions about answers given in the survey. The exit-interviews were recorded on tape after informed consent from the partici- pants, and used for re-listening the participants’ answers if this was considered to be of added value. All students participated voluntarily and received a gift card of 30 Euros upon completion of the exit-interview. At the end of the pilot study a quantitative and qualitative data analysis was performed by analyzing the survey results and inter- view tapes. III. RESULTS Eight medical students, four women and four men, par- ticipated in this pilot study. All students were fifth year medical students during their clerkship neurology. Five students were six weeks in possession of an iPhone and three students four weeks. Seven participants possessed a smartphone, although none of them owned an iPhone at the start of the pilot study. One student was not in posses- sion of a smartphone, but started thinking about purchas- ing one after the project. A. Use of the smartphone and applications Most students found the smartphone a convenient and useful tool during their clerkship, although most students stated that they used their own smartphones for most functions (except for the application NeuroMind) and not the smartphone they received at the start of the study. The majority of students reported that they used the internet on the smartphone frequently, mostly for quick access to medical information. Some students thought that internet (in particular using Google Mobile) was an easier accessi- ble information source compared to many apps. Even though most students stated that they would like apps on their smartphone to support their education in becoming a doctor (Table 2), several students were not aware of any high-quality apps. Furthermore, none of the students were willing to pay more than 25 Euros for a high-quality app. Two students were not willing to pay any amount of money for a high-quality app, four students were willing to pay between 0 en 10 euro and two students were willing to pay between 10 en 25 euro for a high-quality app. TABLE I. PRE-INSTALLED APPLICATIONS ON THE IPHONE Neurology NeuroMind http://DigitalNeurosurgeon.com specific 3D Brain http://www.g2conline.org/ applications Brain Tutor http://brainvoyager.com Nerve Whiz http://www.med.umich.edu/neurol ogy/nerve-whiz.htm General Medische zakkaartjes http://www.anno73.nl/ Evidence based medicine PubMed On Tap http://www.referencesontap.com/ Drug therapy Epocrates http://www.epocrates.com/ Calculators MedCalc http://medcalc.medserver.be/ PDF, Word etc. Stanza http://getsatisfaction.com/stanza/pr oducts/stanza_stanza PDF Reader http://www.kdanmobile.com/en/pd f-reader/ Figure 1. Scores section in NeuroMind 1.3. Figure 2. Example of anatomical image in NeuroMind. iJIM – Volume 7, Issue 2, April 2013 81 SHORT PAPER THE USE OF SMARTPHONES AND MOBILE CLINICAL DECISION SUPPORT SYSTEMS IN CLINICAL CLERKSHIPS B. Acceptance of the smartphone in the clinic In the opinion of most students smartphones should not be used in the presence of a patient for several reasons. Students believed that the use of a smartphone is not accepted by patients, and because the patient does not know what the physician is doing with his phone. One student explained: “I think that patients might not under- stand. They might think you are texting. That is not professional”. Another student said: “If I want to look up something, then I will step out of the room”. Most stu- dents agreed that if a physician or a medical student would like to use a smartphone in the presence of a patient, then the physician or student should explain what he or she is doing with the smartphone. Futhermore, the majority of the students felt that the use of applications by physicians and medical students is acceptable (Table 2). TABLE II. SUMMARY OF SURVEY RESULTS Survey questions * Median Range How many times have you used NeuroMind during your clerkship? 2-3 1-4 How many times did you use NeuroMind in the presence of a patient? 1 1-1 How many times did you handle a situation differently because you used NeuroMind? 1-2 1-3 Survey statements ** Median Range I found it useful to use NeuroMind during my clerkship Neurology 3-4 1-5 I would advise NeuroMind to other medical students during their clerkship Neurology 4 2-5 I find the use of apps by specialists/ residents/interns/medical students accept- able 4-5 1-5 I think that patients accept the use of apps by specialists/residents/interns/medical students 2-3 1-4 I would like to use apps on my smartphone that could support me in my education of becoming a physician 4 3-5 * Questions scale: (1) never, (2) seldom, (3) occasionally, (4) regularly, (5) often. ** Statements scale: (1) completely disagree, (2) disagree, (3) neutral, (4) agree, (5) completely agree. C. NeuroMind Four students have used the application NeuroMind fairly actively. The other four students almost never used NeuroMind and were therefore unable to give any feed- back about the app. The opinion of multiple students was that NeuroMind is a useful app, but the information in the app is too limited. The majority of students that used NeuroMind would advise other medical students during their clerkship neurology to use the app. Furthermore, most students liked the navigational structure of NeuroM- ind. None of the students used NeuroMind in presence of a patient. The students also reported that they rarely changed their actions because of using NeuroMind in the clinic. One example of a student that did change his actions after consulting the MCDSS is a case in which the student had a patient with paraesthesias of the upper extremity. He did not know which dermatome or reflex was involved in this particular situation. After consulting NeuroMind he realized that he tested the wrong reflex, so he went back and elaborated his physical exam. This student explained that the use of NeuroMind had changed his actions and conclusions three or four times during his clerkship. A quantitative summary of the most important survey results is provided in Table 2. D. NeuroMind strengths and weaknesses Most students evaluated the anatomical images and the differential diagnosis as the most useful elements of NeuroMind. Almost all students agreed that the differen- tial diagnosis should be extended. A number of students also reported that they would like more anatomical and radiographic images in NeuroMind. Moreover, the stu- dents would like diseases described in NeuroMind (no full texts, but preferably in a bullet-point style). Furthermore, all students agreed that providing a search module would be a great improvement. IV. DISCUSSION In this pilot study we investigated the use of a smart- phone for medical students during their clerkship neurol- ogy. Most students in this study found the smartphone a useful tool and frequently used the internet on the smart- phone for quick access to medical information. A system- atic review by Kho et al showed that PDAs have become a valuable resource for both medical students and residents [14]. An overview by Lindquist et al also showed a positive attitude towards the PDA, which was regarded as a valuable tool for personnel and students in health care [15]. These findings about the use of PDAs among medi- cal students correspond with the results of this study, in which students regarded the smartphone as a convenient and useful tool. Although the reviews by Lindquist et al en Kho et al included merely studies about PDAs and not smartphones as such, since smartphones were not yet developed then, it is reasonable to think that the results about the value of the PDA also applies to the contempo- rary smartphones. The students stated that the internet was an easier and faster solution for finding information than the use of apps. However, several students were not aware of any high quality apps. Furthermore, most of them were not willing to pay more than ten Euros for a high-quality application, and none of them more than 25 Euros. This might be in contrast with physicians. In this context, there are two important differences between physicians and students. Physicians have a larger financial capacity compared to students. They are able to buy more expen- sive applications without exerting their financial means. However, no evidence was found in the literature that physicians are willing to pay more for a high-quality application. Another difference is that physicians are specialists in one field of medicine, and they only have to consider buying high-quality applications in their own field of medicine. This is in contrast to students, who are still exploring different fields of medicine and therefore may want to buy apps in several medical disciplines. Both selection of useful applications by the medical faculty, and possibly financial support, might help to facilitate the adoption of such high-quality electronic resources. Since this will allow students to try out more apps than they would be able or willing to pay for. This makes students more acquainted with these apps and may form a better foundation for their future use. With regard to the acceptance of the smartphone in daily practice, the students in this study mainly stated that 82 http://www.i-jim.org SHORT PAPER THE USE OF SMARTPHONES AND MOBILE CLINICAL DECISION SUPPORT SYSTEMS IN CLINICAL CLERKSHIPS in their opinion smartphones should not be used in the presence of a patient. One of the reasons the students gave was that they believed that the use of a smartphone is not accepted by patients. This is in contrast with existing literature. Rudkin et. al. concluded that patients’ percep- tions of physicians who use PDAs are neutral or favorable [16]. One might argue that modern smartphones equipped with phone and internet connection might be interpreted differently by patients than traditional PDAs. For exam- ple, a patient may think that the student is text messaging with friends instead of taking care of the patient. We did not find any articles that supported this hypothesis, and we did not measure patients’ perceptions in this pilot study. In this pilot study we also focused on the question whether smartphone applications (in particular MCDSS) for medical students should meet different requirements compared to applications made for physicians. NeuroMind was used as an example of such a MCDSS. As demon- strated in the introduction, the definition of (M)CDSS varies in the literature. For this study we applied the definition by Payne [7], which is less restrictive. The results of this study showed that the students liked NeuroMind, a program originally designed for physicians, but that they preferred more basic information like ana- tomical images, extended differential diagnosis, and more disease descriptions in the application. This is in accor- dance with our hypothesis, that medical students have different preferences for a MCDSS than physicians, for example the need for more fundamental information in a MCDSS. The needs from the perspective of a medical student have hardly been reported in the available litera- ture. Further research is needed to guide MCDSS devel- opment specifically targeted at medical students. Another aim of the study was to improve the tested MCDSS. As a result of the outcomes of this study, the decision was made to focus NeuroMind primarily on the original target audience, which consists of physicians. From version 1.5 onwards the “Scores” category has more than doubled in number, and additional anatomical images have been added for explanation to patients and medical students. In contrast, the category “Differential diagnosis” has been discontinued, as it was known from personal communication that it was hardly ever used by physicians. To increase the reliability of the information presented in the MCDSS, references to peer-reviewed literature were added for all scores from version 1.5 onwards. The latest addition is an interactive form of decision support in version 2.0. A separate student version of NeuroMind is under consideration. A. Study strengths and weaknesses A particular strength of this study is the target group: medical students in their clinical clerkships. There is limited literature available on how to implement a mobile learning strategy for medical students. Another strength of the study is its prospective character and qualitative information collection and analysis. First the students were asked to fill out a survey, afterwards they were invited to comment on some of their answers and to provide general feedback about the test period with the smartphone. A weakness of this pilot study is the small sample size, which is aggravated by the fact that the majority of stu- dents preferred to use their own smartphone instead of the iPhone with pre-installed applications that was handed over to them. Only half of the students used NeuroMind and were therefore able to evaluate this application. We learned from this pilot study that an effective mobile learning strategy needs to incorporate students’ own devices. From a maintenance perspective this can be a nightmare, as many different software platforms (in particular operating platforms) have to be supported. This could be overcome by offering mobile websites instead of “apps”, but in general apps offer a much better and more interactive user experience compared to mobile websites. Furthermore, apps can be developed in such a manner that no constant internet access is necessary. On the other hand, the current trend towards “cloud computing” (online information available through web access) may be pre- ferred for rapidly changing information that does not require advanced visualization, and needs to be available on a wide range of devices. Of course, constant internet access is mandatory in that case. Although we did not collect as much information on the effectiveness of implementing MCDSS for medical students in their clerkships as we intended, we did learn that using pre-installed devices for such a purpose is a recipe for failure, as almost all students are already in possession of such a device. This definitely needs to be considered by other departments who are training medical students and wish to enhance learning by adopting a mobile strategy. V. CONCLUSION In conclusion, smartphones can be a useful tool for medical students during their clerkship, but MCDSS made for physicians are not automatically suitable for medical students. Therefore, MCDSS for medical students should be designed to their needs. Although half of the students reported to use the MCDSS as a quick reference, they expressed a desire for more basic content. When develop- ing a MCDSS for medical students one needs to take the willingness of students to pay for applications and their knowledge about high-quality applications into account. Both selection of useful applications by the medical faculty, and possibly financial support, might help to facilitate the adoption of such electronic resources. It seems mandatory to support students using their own smartphone devices instead of equipping them with preinstalled devices. Further research is needed to guide MCDSS development specifically targeted at medical students. REFERENCES [1] Arndt KA. Information excess in medicine. Overview, relevance to dermatology, and strategies for coping. Arch Dermatol 1992;128:1249-1256. http://dx.doi.org/10.1001/archderm.1992. 01680190105014 [2] Humphreys BL, McCutcheon DE. Growth patterns in the National Library of Medicine’s serials collection and in Index Medicus journal 1966-1985. Bull Med Libr Assoc 1994;82:18-24. [3] Hook O. Scientific communications. History, electronic journals and impact factors. Scand J Rehabil Med 1999;31:3-7. http://dx.doi.org/10.1080/003655099444669 [4] U.S. National Library of Medicine: National Institutes of Health. Detailed Indexing Statistics: 1965-2010. Available from: http://www.nlm.nih.gov/bsd/index_stats_comp.html [Last ac- cessed on January 10, 2012]. [5] Fraser AG, Dunstan FD. On the impossibility of being expert. BMJ 2010;341:c6815.  http://dx.doi.org/10.1136/bmj.c6815 iJIM – Volume 7, Issue 2, April 2013 83 SHORT PAPER THE USE OF SMARTPHONES AND MOBILE CLINICAL DECISION SUPPORT SYSTEMS IN CLINICAL CLERKSHIPS [6] Glasziou PP. Information overload: what’s behind it, what’s beyond it? MJA 2008;189(2):84-85. [7] Payne TH. Computer Decision Support Systems. CHEST 2000;118:47S-52S.   http://dx.doi.org/10.1378/chest.118.2_suppl. 47S [8] Osheroff JA, Pifer EA, Teich, JM, Sitting DF, Jenders RA. Improving Outcomes with Clinical Decision Support: An Imple- menter’s Guide. Chicago: Healthcare Information and Manage- ment Systems Society, 2005. [9] Garg AX, Adhikari NKJ, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J. Effects of Computerized Clinical Deci- sion Support Systems on Practitioner Performance and Patient Outcomes: A Systematic Review. JAMA 2005;293:1223-1238. http://dx.doi.org/10.1001/jama.293.10.1223 [10] Baumgart DC. Personal digital assistants in health care: experi- enced clinicians in the palm of your hand. Lancet 2005;366:1210- 1222. http://dx.doi.org/10.1016/S0140-6736(05)67484-3 [11] Baumgart DC. Smartphones in Clinical Practice, Medical Educa- tion, and Research. Arch Intern Med 2011;171:1294-1296. http://dx.doi.org/10.1001/archinternmed.2011.320 [12] Burdette SD, Herchline TH, Richardson WS. Killing bugs at the bedside: a prospective hospital survey of how frequently personal digital assistants provide expert recommendations in the treatment of infectious diseases. An Clin Microbiol Antimicrob 2004;3:22. http://dx.doi.org/10.1186/1476-0711-3-22 [13] Manhattan Research. Physician smartphone rate to reach 81% in 2012. Available from: http://manhattanresearch.com/News-and- Events/Press-Releases/physician-smartphones-2012. [Last ac- cessed on September 18, 2011]. [14] Kho A, Henderson LE, Dressler DD, Kripalani S. Use of Hand- held Computers in Medical Education: A Systematic Review. J Gen Intern Med 2006;21:531-537. http://dx.doi.org/10.1111/ j.1525-1497.2006.00444.x [15] Lindquist AM, Johansson PE, Petersson GI, Saveman B, Nilsson GC. The Use of the Personal Digital Assistant (PDA) Among Per- sonnel and Students in Health Care: A Review. J Med Internet Res 2008;10(4):e31. http://dx.doi.org/10.2196/jmir.1038 [16] Rudkin SE, Langdorf MI, Macias D, Oman JA, Kazzi AA. Personal digital assistants change management more often than paper texts and foster patient confidence. European Journal of Emergency Medicine 2006;13:92-96.   http://dx.doi.org/10.1097/ 01.mej.0000192049.04729.0c AUTHORS A.M.R. Schols, BSc, is a medical student at Faculty for Health, Medicine and Life Sciences, Maastricht Univer- sity Medical Center, The Netherlands (e-mail: a.schols@student.maastrichtuniversity.nl). H.H.L.M. Donkers, PhD, is with Institute for Health, Medicine and Life Sciences Education, Faculty for Health, Medicine and Life Sciences, Maastricht University Medi- cal Center, The Netherlands (e-mail: jeroen.donkers@maastrichtuniversity.nl). M. Voorend, MD, PhD, is a neurologist at the De- partment of Neurology, Maastricht University Medical Center, The Netherlands (manuela.voorend@mumc.nl). D.M.L. Verstegen, PhD, is with the Institute for Health, Medicine and Life Sciences Education, Faculty for Health, Medicine and Life Sciences, Maastricht Univer- sity Medical Center, The Netherlands (e-mail: d.verstegen@maastrichtuniversity.nl ). H. Hoogland, MD, PhD, is with the Institute for Health, Medicine and Life Sciences Education, Faculty for Health, Medicine and Life Sciences, Maastricht Univer- sity Medical Center, The Netherlands (e-mail: henk.hoogland@maastrichtuniversity.nl). P.L. Kubben, MD, is a resident at the Department of Neurosurgery, Maastricht University Medical Center, The Netherlands. Besides he is with the Faculty of Health, Medicine and Life Sciences, Maastricht University Medi- cal Center, The Netherlands, for development of mobile computing and clinical decision support systems. (e-mail: pieter@kubben.nl). For more information, visit http://www.kubben.nl . This study was financially supported by an educational grant from Maastricht University, The Netherlands. Maastricht University has not been involved in the study design, data collection or writing the paper. Maastricht University had no veto right. Received 15 December 2012. Published as resubmitted by the authors 20 March 2013. 84 http://www.i-jim.org iJIM – Vol. 7, No. 2, April 2013 The Use of Smartphones and Mobile ClinicalDecision Support Systems in Clinical Clerkships