Mobile medical and health apps: state of the art, concerns, regulatory control and certification 1 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014 OJPHI Mobile medical and health apps: state of the art, concerns, regulatory control and certification Maged N. Kamel Boulos 1 , Ann C. Brewer 2 , Chante Karimkhani 3 , David B. Buller 4 , Robert P. Dellavalle 5 1 Faculty of Health & Human Sciences, University of Plymouth, Drake Circus, Plymouth, Devon PL4 8AA, UK 2 Dermatology Residency Program, Mayo Clinic in Arizona, Scottsdale, AZ 85259, USA 3 Columbia University College of Physicians and Surgeons, New York, NY 10032, USA 4 Klein Buendel, Inc, Golden, CO 80401, USA 5 Dermatology Service, Denver VA Medical Center, Denver, CO 80220, USA Abstract This paper examines the state of the art in mobile clinical and health-related apps. A 2012 estimate puts the number of health-related apps at no fewer than 40,000, as healthcare professionals and consumers continue to express concerns about the quality of many apps, calling for some form of app regulatory control or certification to be put in place. We describe the range of apps on offer as of 2013, and then present a brief survey of evaluation studies of medical and health-related apps that have been conducted to date, covering a range of clinical disciplines and topics. Our survey includes studies that highlighted risks, negative issues and worrying deficiencies in existing apps. We discuss the concept of ‘apps as a medical device’ and the relevant regulatory controls that apply in USA and Europe, offering examples of apps that have been formally approved using these mechanisms. We describe the online Health Apps Library run by the National Health Service in England and the calls for a vetted medical and health app store. We discuss the ingredients for successful apps beyond the rather narrow definition of ‘apps as a medical device’. These ingredients cover app content quality, usability, the need to match apps to consumers’ general and health literacy levels, device connectivity standards (for apps that connect to glucometers, blood pressure monitors, etc.), as well as app security and user privacy. ‘Happtique Health App Certification Program’ (HACP), a voluntary app certification scheme, successfully captures most of these desiderata, but is solely focused on apps targeting the US market. HACP, while very welcome, is in ways reminiscent of the early days of the Web, when many “similar” quality benchmarking tools and codes of conduct for information publishers were proposed to appraise and rate online medical and health information. It is probably impossible to rate and police every app on offer today, much like in those early days of the Web, when people quickly realised the same regarding informational Web pages. The best first line of defence was, is, and will always be to educate consumers regarding the potentially harmful content of (some) apps. Keywords: mobile apps, text messaging, smartphones, mobile tablet computers, mobile health (mHealth), telemedicine, healthcare, evaluation, regulation and certification, quality Correspondence: mnkamelboulos@plymouth.ac.uk DOI: 10.5210/ojphi.v5i3.4814 Copyright ©2014 the author(s) http://ojphi.org/ Mobile medical and health apps: state of the art, concerns, regulatory control and certification 2 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014 OJPHI Background Smartphones, the most common “personal computer” today, have revolutionised the communication landscape. Almost ‘always on’ and highly portable (carried by their users everywhere they go), smartphones provide real-time, on-demand communication, while their rich multimedia touch-displays operate with increasing speeds, delivering data services and computing power to document and improve the networked lives of their owners [1,2]. Communication via smartphones is personalised: smartphones store and exchange large amounts of personal information and users are able to customise their phones to suit their personal preferences and needs. A smartphone can record a large number of details about its user’s current status and whereabouts. It can relay appropriate social support and enable real- time and asynchronous exchanges with other users via social networks and other forms of mobile communications. The latter include text messaging (Short Message Service—SMS), photography (still and video), location and other sensors (global positioning system [GPS], accelerometers, ambient light sensor, etc.), built-in applications or apps (e-mail, contacts, calendar, document readers and video players, etc.) and wireless data service [2]. (‘App’, short for ‘application (program)’, refers to a self-contained piece of software coded for a specific purpose and usually optimised to run on a mobile device.) Smartphones out-shipped feature phones worldwide for the first time in Q1 2013 [3,4]. One of the main differences between smartphones and feature phones is that the latter, besides being less expensive than the former, offer very limited or no support for third-party, full-fledged apps. According to the ‘Mobile Health 2012’ report published by Pew Research Centre’s Internet & American Life Project, 85% of US adults own a cell phone; of them, 53% own smartphones. Half of smartphone owners use their devices to get health information. One- fifth of smartphone owners have health apps on their devices [5]. The mobile revolution is offering an unprecedented opportunity to provide medical support when and where people need it. Large numbers and varieties of medical and health-related apps exist on the market today. A 2012 estimate puts the number of health-related apps at no fewer than 40,000 [6]. From basic apps composed of text message reminders to apply sunscreen, to sophisticated apps that coordinate the management of diabetes, apps play a multitude of functions in health and healthcare. Mobile technology has several potential advantages for providing actionable medical advice, but also has its own limitations and potential problems associated with it. These aspects of mobile technology will be the focus of the rest of this paper. Range of mobile applications Apps for medical providers Many apps are developed for a target audience of healthcare workers, including physicians, nurses and assistants. These apps are generally more sophisticated, with medical terminology and functions, and not easily navigable by non-health professionals. In a study published in This is an Open Access article. Authors own copyright of their articles appearing in the Online Journal of Public Health Informatics. Readers may copy articles without permission of the copyright owner(s), as long as the author and OJPHI are acknowledged in t he copy and the copy is used for educational, not-for-profit purposes. http://ojphi.org/ Mobile medical and health apps: state of the art, concerns, regulatory control and certification 3 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014 OJPHI 2012, a group of surveyed healthcare workers indicated the most popular categories of mobile applications functions include drug-referencing tools, clinical decision-support tools, communication, electronic health-record system access and medical education materials [7]. The top apps were drug-reference guides such as ‘Epocrates’ (Epocrates, Inc.) [8] and ‘Lexicomp’ (Wolters Kluwer) [9], as well as clinical decision-support reference tools such as ‘UpToDate’ (Wolters Kluwer) [10] and ‘Medscape’ (WebMD LLC) [11]. Specialty or disease-specific apps Though some apps cover a broad spectrum of general medical knowledge, others may be tailored to specific specialties such as colorectal disease-themed apps in gastroenterology or goniometer apps in orthopaedics [12,13]. The ‘Eye Handbook’ (developed at the University of Missouri Kansas City, USA) is a free ophthalmology-themed app offering mobile diagnostic visual tests, a user directory of eye-care professionals, ophthalmology specific calculations, ICD-9 codes and an atlas of common ophthalmic conditions [14,15]. The field of infectious disease may particularly benefit from the rapid updates of apps which provide news and updates in real time. ‘Outbreaks Near Me’ (developed at Boston Children’s Hospital, USA) utilises news media reports, medical e-mail list services and alerts from official national and international organisations to monitor global infectious diseases via the ‘HealthMap’ database [16,17]. Medical education and teaching The younger generation of technologically capable medical professionals in training, such as students and residents, harness the power of innovative apps to improve learning. The mobility of a smartphone or tablet allows students to carry a plethora of clinical resources in a convenient and searchable package. The flexibility of the mobile platform provides interactivity in teaching and a more personalised education. Given the high rate of mobile technology adoption among young clinicians, mobile applications display great potential to augment traditional training. A survey published in 2012 of ACGME (US Accreditation Council for Graduate Medical Education) programmes demonstrated over 85% of respondents used a smartphone and over half of respondents used mobile applications on a daily basis, most commonly drug guides, medical calculators, coding/billing apps and pregnancy wheels [18]. ‘Bump’ (Bump Technologies, Inc.), a mobile app that transfers information between two mobile devices when put in very close proximity together, demonstrated enhancement of pharmacy student learning and communication in patient simulation scenarios [19]. For new trainee doctors, a mobile app, which functioned as a portable electronic library, provided a wealth of information when senior or attending physicians were not available and thus enhanced patient care [20]. German physicians adapted the German College of General Practitioners and Family Physicians practice guidelines into an easily accessible app for physicians and medical students [21]. Flashcard and subject review applications, such as ‘Microbiology and Immunology Wiz’ (Current Clinical Strategies Publishing) [22] or ‘Lange Microbiology & Infectious Diseases Flash Cards’ (Modality Inc.) [23], allow for portable and customisable education for medical students [17]. When provided with access to smartphones, resident physicians in Botswana effectively utilised several point-of-care mobile applications in delivering healthcare in a resource-limited setting [24]. http://ojphi.org/ Mobile medical and health apps: state of the art, concerns, regulatory control and certification 4 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014 OJPHI Apps for patients and the general public (including health and fitness apps) On the other end of the spectrum, there are patient-centred apps capable of performing an equally wide array of functions. Current apps aid patients in managing chronic disease, lifestyle management, smoking cessation and even self-diagnosis. The diabetes mellitus epidemic is reflected in the number of apps geared towards diabetic patients. On the Android platform alone, over 80 diabetes apps offer a variety of functions, including self-monitoring blood glucose recording, medication or insulin logs, and prandial insulin dose calculators [25]. Another diabetes intervention app integrated communication between patients and a healthcare provider. The patient would log fasting blood sugars, daily eating behaviours, medication compliance, physical activity and emotions into a mobile online diary. A remote therapist with access to these diaries would then formulate personalised feedback to the patient [26]. The most number of apps belong to the exercise and weight loss category. The built-in camera, standard in smartphones today, allows users to record a photo diary of daily food and drink. These photos may be transferred to a server, which identifies and quantifies the food portion [27]. A weight loss trial utilised a mobile app to monitor dietary intake, body weight and objectively-measured physical activity (obtained from a Bluetooth-enabled accelerometer) of its participants [28]. Kamel Boulos and Yang surveyed dozens of mobile, location-based (outdoor) exergaming apps that harness the power of sharing through online social networks and gamification principles on GPS-enabled smartphones. In these apps, the real world becomes the ‘game map’ or playground, and players can even discover and learn about new places and their geographies while burning calories and keeping fit [29]. Apps are also developed for smoking cessation and alcohol addiction. At least 47 iPhone apps for smoking cessation are available [30]. ‘A-CHESS’ (Alcohol Comprehensive Health Enhancement Support System), a smartphone-based intervention for preventing relapse in alcoholic dependency harnesses mobile technology to improve treatment and motivation [31]. In patients with chronic disease characterised by life-threatening flares, apps may allow them to track and even report symptoms. The app ‘m.Carat’ was developed at Faculdade de Medicina da Universidade do Porto, Portugal, for asthmatic and allergic rhinitis patients to record their exacerbations, triggers, symptoms, medications, lung function tests and visits to the doctor or the hospital [32]. Users can also receive disease education, medication information, task notifications, and synchronise records with an online database to better control their symptoms [33]. Psychiatric patients benefit from ambulatory monitoring through an app that randomly prompts the patient to self-report psychotic symptoms multiple times throughout the day [34]. Another app for sickle cell disease patients allows them to access an online diary to record pain and other symptoms [35]. Monitoring symptoms in patients with COPD (chronic obstructive pulmonary disease) through a mobile app alerts patients and providers to suspected disease exacerbations, thereby facilitating prompt intervention [36]. An app developed for patients with dementia, ‘iWander’, assists patients with daily living by providing audible prompts offering to direct the patient home, sending notifications and GPS coordinates to caretakers, or by calling local 911 (US emergency) services [37]. http://ojphi.org/ Mobile medical and health apps: state of the art, concerns, regulatory control and certification 5 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014 OJPHI Patients may even use apps to attempt self-diagnosis without a medical visit. Patients with a camera-enabled smartphone can use apps to take photographs of skin lesions and send these to a remote server for computer analysis and/or review by a board certified dermatologist [38,39]. Such apps are not without their pitfalls and this will be discussed in detail later in this paper. Apps may empower non-medical professionals to provide basic triage at the scene of trauma such as on the sidelines of a sporting event. An app development team with a neurosurgeon among its members created ‘Concussion Test’, which follows the standardised and validated Sport Concussion Assessment Tool 2 (SCAT2). Five similar concussion apps exist for purchase as well [40]. Mobile app technology has far-reaching potential in the public health domain as well. Apps may be used to contribute to the care and prevention of sexually transmitted disease (STD). A study of available apps demonstrated 55 unique mobile apps for HIV (human immunodeficiency virus) and STD education, prevention, testing and resources [41]. Text messaging Text messaging, or Short Message Service (SMS), dates back to the early days of mobile phones, and most applications worked equally well on the simplest mobile phones, the more advanced feature phones, as well as full-fledged smartphones. Text messaging reminder applications are easy to implement with tools such as InSTEDD’s ‘Remindem’ (free and open source) [42]. Due to the pervasive nature of the mobile phone, text-messaging applications have the unique opportunity to alert patients directly regardless of location (or availability to take a voice call). Evaluation of text messaging shows promising results for assisting with clinical monitoring and counselling, keeping medical appointments, smoking cessation, weight loss, chronic disease management (e.g., diabetes care monitoring), or reminding people to use sun protection to prevent sunburns in the near term and skin cancer in the long term [43-46]. Adherence to therapy and disease control may be improved in psoriasis patients who receive daily educational and motivational text messages [47]. Text message reminders to patients may also improve antibiotic compliance [48] or even prevent recurrent cardiovascular events [49]. On the one hand, text messaging provides nearly universal user access, particularly in low- resource settings, with decreasing costs, simple interventions to develop and use, customisable content and schedule, and a push-mode delivery that prompts users to read and possibly respond. On the other hand, text messaging offers limited interaction, with often only passive engagement, and does not leverage the latest smartphone computing power. Electronic health records Mobile apps may also provide access to electronic health records and patient information. Kharrazi et al. mention 19 apps that allow patients to store personal health records on their mobile devices [50]. ‘HealthVault’ (Microsoft) acts as a platform for personal health data, including data from personal health monitoring and fitness devices, putting consumers in control of their health information, with the ability to securely share it with clinicians, caregivers, family members, or others, as needed [51]. ‘DocbookMD’, an app meeting HIPAA (US Health Insurance Portability and Accountability Act) encryption and security http://ojphi.org/ Mobile medical and health apps: state of the art, concerns, regulatory control and certification 6 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014 OJPHI requirements [52], allows physicians to easily transmit text messages and images to one another [53]. Telemedicine and telehealthcare The implementation of telemedicine and telehealthcare (including clinical telemonitoring of patients) through the application of mobile devices is clearly a practical and potentially low- cost choice in the delivery of healthcare, as seen, for example, in the mobile component of the CAALYX/eCAALYX (Complete Ambient Assisted Living Experiment/Enhanced Complete Ambient Assisted Living Experiment) prototype systems [2,54]. Several applications have been described that harness mobile devices and apps to increase efficiency and access to care, particularly in emergency situations [55]. When time is of the essence, apps can increase speed and accessibility to critical specialist care in real time, e.g., in stroke or acute trauma. Acute stroke care is made portable and accessible to non-urban centres via real-time video on smartphones [56]. The ‘i-Stroke’ system was developed to transfer clinical data, computed tomography (CT), magnetic resonance imaging (MRI), angiographic and intraoperative images, as well as expert opinion, all in real time [57]. ‘ResolutionMD’ (Calgary Scientific) [58] is an FDA (US Food and Drug Administration)-cleared teleradiology app incorporated into a telestroke network, which provides remote vascular neurologists with radiographic images [59,60]. Acute trauma patients also benefit from timely and efficient management. An iPhone-based teleradiology program was used for the diagnosis of acute cervical trauma, examining CT scans to evaluate for the presence of fractures or displacements [61]. Resource-limited settings and remote locations (e.g., distant rural areas and desert settlements) may benefit from access to specialist care and teleconsultations through mobile technology, particularly in disciplines with no locally-residing specialists, such as ophthalmology or dermatology. In one study, the iPhone was used to send fundoscopic images to board certified ophthalmologists for review to detect diabetic retinopathy [62]. Mobile phone multimedia messaging allowed general practitioners to send teledermatology referrals in the form of photos and relevant clinical information to specialist dermatologists for consultation [63]. In some instances, mobile apps may allow telemedicine to replace time-consuming office visits altogether. This modality may benefit specialties that require frequent follow-up care or monitoring, such as rehabilitation or post-operative care of patients. A physical therapy app provided virtual-reality-based balance exercises through a mobile device. Remote physiotherapists with access to the results could adjust the level of exercises accordingly [64]. Surgeons utilised remote real-time monitoring of free flaps via smartphone photography to replace in-person examination [65]. Smartphone attachments Several applications work in conjunction with some specialised phone attachment piece or wireless sensors in order to perform specialised activities that are not part of a phone’s standard functions, e.g., the CAALYX/eCAALYX system described in [2,54]. http://ojphi.org/ Mobile medical and health apps: state of the art, concerns, regulatory control and certification 7 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014 OJPHI For example, an adaptor with electrocardiogram (ECG) electrodes may transmit electrical data to detect abnormal heart rhythms in a non-hospital setting [66]. Patients with diabetes may synchronize a glucometer attachment to their mobile device to track blood glucose and share the data through an internet connection [67,68]. The next generation of smartphone app technology may even enable users to perform routine blood testing [69]. Using a Windows Phone with a micro lens mounted over the camera, the ‘Lifelens’ app captures high resolution images of cells in a drop of blood and then analyses them to detect to existence of malaria [70]. Evaluation studies of mobile medical and health-related apps Ever since the first cellular phone call was placed on 3 April 1973, the mobile age has grown and continues to do so at an exponential rate, particularly during the 21st century [71]. The vast array of smartphones, mobile tablets and mobile medical and health-related apps on offer today (see the app examples presented earlier in this article) provides consumers with an unprecedented opportunity to achieve their health and healthcare goals, and overcome many obstacles along the way. However, with such a booming industry also come concerns, risk and potential dangers. It is perhaps surprising that relatively very little research has been undertaken so far (as of 2013) to investigate the validity and efficacy of these devices and apps in the contexts of health and healthcare. Here, we will briefly present several evaluation and validation studies exploring the diversity of health and healthcare-related apps. The majority of these studies have been conducted on small patient populations to compare the efficacy of a smartphone/tablet app versus conventional resources. Diabetes management One of the most well studied areas is that of diabetes management. A review conducted by Demidowich et al. [25] investigated 42 Android apps for diabetes self-management. The mean composite usability score which evaluated six standard features per app was 11.3 out of a possible 30, and no apps offered direct data input from glucometers. The study concluded that surprisingly few apps provided a comprehensive method of diabetes self-management, but did mention ‘Glucool Diabetes’ (3qubits) [72], ‘OnTrack Diabetes’ (GExperts Inc.) [73], ‘Dbees’ (Freshware.pl) [74] and ‘Track3 Diabetes Planner’ (Coheso, Inc.) [75] as recommended apps [25]. Medical imaging Another fascinating area of investigation is that of tablet (iPad [Apple, Inc.]) clinical imaging apps used to view medical and diagnostic images. Such mobile apps pose particular advantages in emergency settings. Several studies have been conducted to compare the efficacy of the iPad to diagnose pulmonary embolism and intracranial haemorrhage versus conventional Picture Archiving and Communications System (PACS) or liquid-crystal display (LCD) monitor systems. The studies have found the iPad to be equivalent to conventional methods, but express the need for conducting further research to examine minor discrepancies [76-78]. In addition, the iPad is being explored as an aid in laparoscopic training for residents and for percutaneous kidney access [79,80]. Much of this exploration is described as ‘case report’ experimental studies rather than full evaluation reports. http://ojphi.org/ Mobile medical and health apps: state of the art, concerns, regulatory control and certification 8 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014 OJPHI Global health and infectious diseases The need for affordable, reliable and prompt diagnostic and therapeutic measures is especially evident in the global health infectious disease arena. The iPad has been shown to be comparable to conventional PACS LCD viewing in the diagnosis of tuberculosis [81]. A study performed in rural Bangladesh demonstrated that basic mobile phone technology is both efficient and effective in improving case detection and management of malaria [82]. Mobile phone messaging has been proposed and investigated as a method to improve medication adherence and communication in HIV management, but a systematic review published in 2013 by van Veltoven et al. examined 21 studies and determined that there is limited evidence that mobile phones are efficacious in HIV care [83]. Further studies are needed to adequately assess this topic. Pain management using smartphone-based diaries An innovative and novel use of smartphone technology is in the realm of pain management. As quoted in a 2011 New York Times interview, Dr Sean Mackey, chief of pain management at Stanford School of Medicine, USA, explains that “before we did not have good data on what is the burden of pain in our society…the number of people is more than diabetes, heart disease, and cancer combined” [84]. Pain is an incredibly diverse and prevalent state that it is often hard for patients to describe, which makes it even harder for caregivers to diagnose and treat. Smartphone technology has the potential to revolutionise real-time pain reporting. In a usability testing study published in 2012, a smartphone-based e-diary was successfully used by children and adolescents with sickle cell disease to report pain symptoms [34]. A randomised clinical trial has shown women with chronic widespread pain experience fewer catastrophising events (rumination, expecting the worst, and feeling helpless) when using smartphone-based diaries with immediate therapist feedback [85]. Similar studies have shown successful usability of smartphone pain assessment in wheelchair users and adolescents with cancer [86,87]. Dermatology The field of dermatology is also taking advantage of the technological smartphone revolution. One randomised community trial provided text message reminders to use sunscreen daily as the intervention (for sun protection, to prevent sunburns in the near term and skin cancer in the long term), and monitored dispensed sunscreen. Text messages increased sunscreen use, with greater daily adherence [46]. Text messaging has also been investigated as a tool for improving motivation and treatment adherence in patients with psoriasis. Following 12 weeks of daily text message reminders/educational tools, patients demonstrated significantly better improvement of disease severity and quality of life, with superior adherence to therapy and optimised patient- physician communication compared to a control group with no text message intervention [47]. Multimedia (text and photos) messaging (Multimedia Messaging Service[MMS]) has proven to be a promising tool in teledermatology, where it has been used to send digital photographs of skin conditions to specialist dermatologists for diagnosis. One study compared MMS photographs sent to dermatologists at a university hospital versus separate face-to-face visits http://ojphi.org/ Mobile medical and health apps: state of the art, concerns, regulatory control and certification 9 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014 OJPHI in 40 patients. The diagnosis based on multimedia message referral was correct for 78% and provided management recommendations for 98% of patients. This study employed two dermatologists for the multimedia message diagnosis and two separate dermatologists for the face-to-face visits. Interestingly, there was a lower concordance (68%) between the two dermatologists on the MMS arm of the study compared to 88% between the face-to-face dermatologists [63]. Another potential use of smartphones in dermatology involves apps targeted for patients. In particular, apps have been designed to aid patients with suspicious skin lesions to determine if their lesion is benign or malignant. A study published in 2013 in JAMA Dermatology analysed four of these apps and discovered that the ability of apps to assess melanoma risk is highly variable. In fact, three of the four studied apps incorrectly classified 30% or more of melanomas as ‘unconcerning’. A major conclusion of this study was that extreme caution should be exercised when consumers use apps to assess their medical risks, since many apps are subjected to very little or absolutely no regulatory oversight [39]. Such apps might still be useful, as long as there is an appropriate in-app disclaimer warning users in a clear and simple language about the app’s diagnostic limitations, the possibility and implications of ‘false negatives’, and that the app should not be taken as a substitute for proper professional clinician’s evaluation and advice. Regulatory control and certification of medical and health apps Studies expressing concerns about existing apps for colorectal diseases, microbiology, dermatology, asthma, diabetes and opioid converters A number of recent articles and studies have investigated the potential dangers and safety of some clinical and health apps aimed at healthcare professionals (but available to all) or aimed at the general public, and whether (and most importantly how) they should be assessed and controlled by the US Food and Drug Administration (FDA) and/or other relevant and corresponding entities in other countries such as the Medicines and Healthcare Products Regulatory Agency (MHRA) in England [88-91]. For example, O'Neill and Brady [12] recently questioned the reliability of unregulated medical apps specifically applied towards colorectal diseases. Of a total of 68 individual colorectal themed apps they surveyed in their study (amongst which five were duplicates), only 29% had had user satisfaction ratings and 32% had named medical professional involvement in their development or content. With such a little medical professional involvement in the design of the majority of these apps, increased regulation of some kind is definitely required if we were to improve accountability for app content [12]. Similarly, in another study by Visvanathan et al. [92], the accuracy and reliability of the content of apps used in diagnosis and patient management were called into question. Of 94 microbiology-themed apps they surveyed, only 34% had stated medical professional involvement. The lack of medical professional involvement in the design of the majority of these apps again undermines users' ability to be informed regarding app content quality. Visvanathan et al. conclude by proposing that increased regulatory measures be introduced to safeguard patient welfare [92]. Ferrero and colleagues [38] investigated the potential danger of clinical dermatology apps targeting patients, calling for FDA regulation of such apps. They tested ‘Skin Scan’, an app created to help with the identification and management of skin cancer, against 93 clinical http://ojphi.org/ Mobile medical and health apps: state of the art, concerns, regulatory control and certification 10 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014 OJPHI images from the National Cancer Institute and Fitzpatrick’s Dermatology in General Medicine, and found only 10.8% (10/93) were rated as high risk melanomas. ‘Skin Scan’ has since then been renamed to ‘SkinVision’ (SkinVision BV, Netherlands) [93] by its developers, who have also announced they are planning a clinical trial in Europe to compare the effectiveness of their app against traditional diagnostic tools [94]. Likewise, Robson et al. [95] and Wolf et al. [39] urged caution when using melanoma risk analysis and detection apps due to their diagnostic inaccuracy. Huckvale et al. [96] and McKinstry [97] expressed their concerns about worrying deficiencies in existing asthma self- management apps, concluding that none combine reliable information and appropriate supportive tools, and that some are even unsafe. Demidowich et al. [25] described deficiencies and other issues hindering usability in their reviewed sample of diabetes self- management apps for Android smartphones. O'Neill and Brady [98] pointed to apps where inaccuracy or inconsistency could potentially cost lives, citing their experience with opioid conversion calculators. They examined 23 different opioid conversion medical apps and found alarming inconsistencies in their outputs. Apps as a ‘medical device’: regulation in USA and Europe Through guidance first released in draft form in July 2011, the US FDA defined “a small subset of mobile medical apps that may impact on the performance or functionality of currently regulated medical devices and as such, will require FDA oversight” [99,100]. Such medical apps could present a real risk to patients if the apps do not work as intended. The FDA has already cleared a handful of mobile medical apps [101] that are either used as an accessory to an FDA-regulated medical device or transform a mobile platform into a regulated medical device (e.g., an app that turns a smartphone into an ECG machine). FDA-approved apps include ‘Mobile MIM’ for iPhone and iPad, a diagnostic radiology app by Cleveland-based MIM Software Inc. that enables a healthcare professional to view medical images on an iPad and make a diagnosis [102]. More recently, Intuitive Medical Technologies in Shreveport, Louisiana, received 510(k) FDA clearance for their ‘iExaminer’ adapter and companion app for the iPhone. The ‘iExaminer’ attachment connects ‘Welch Allyn’s PanOptic Ophthalmoscope’ to an iPhone, aligning the eye piece of the ophthalmoscope to the phone’s camera. The combined system and app allows clinicians to image the eye (fundus exams) and save the images for later review or sharing with colleagues [103]. In a must-read IEEE Spectrum article published in September 2012, Strickland presented a list of top-five requests to the FDA that came out of a survey of key industry figures on what they would like to see in FDA rules for medical apps [104]. The requests deal essentially with the need to crisply define the boundaries between apps that have to be regulated by the FDA and apps that do not have to go through the certification process. The surveyed industry experts called for clarifying the difference between a medical app and a wellness app, as well as the difference between diagnosing and monitoring; establishing the risk-level threshold for FDA enforcement; defining the limits of the FDA’s rule on apps that serve as device accessories; and making a plan for how to handle “modular” apps, i.e., multiple apps designed to work together (if one of those apps is regulated by the FDA, must all the others be as well?) [104]. In Europe, ‘ONCOassist’ [105], an Irish app for the iPhone and iPad that contains prognostic tools and useful calculators for oncologists at the point-of-care (and as such falls within the http://ojphi.org/ Mobile medical and health apps: state of the art, concerns, regulatory control and certification 11 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014 OJPHI definition of a medical device), has received CE certification or Conformité Européenne, a key mark of a product’s compliance with relevant EU [European Union] legislation [106] in 2013, and displays the ‘CE mark’ on its welcome screen [107]. Indeed, the European Medical Device Directive MDD 93/42/EEC explicitly mentions ‘software’ in its definition of ‘medical device’ [108]. But many apps with dosage calculator functions currently marketed in the UK still do not carry the CE mark to show that they have been registered as class I medical devices with the MHRA in England or one of the corresponding regulatory bodies in other EU countries [109]. The likelihood of an app being treated as a medical device by the MHRA depends on what the app does and the corresponding level of patient risk associated with it. High risk apps, e.g., those performing complex calculations using patient data to aid diagnosis or treatment decisions, can be safely classified as ‘medical devices’. According to MHRA, if an app is purely a record archiving and retrieval system (electronic health records), it is unlikely to be considered a medical device; however, if it includes a module that interprets data or performs some calculation, then it is likely that this particular component may be considered a medical device. Decision-support apps are also generally not considered a medical device by MHRA if they just provide existing (i.e., reference/evidence- based) information to enable a healthcare professional to make a clinical decision. However, if the apps perform a calculation or interpret or interpolate data and the clinician does not review the raw data, then such apps may be considered medical devices for MHRA purposes. Nevertheless, an app performing simple and straightforward calculations such as BMI (body mass index) should not be treated as a medical device, but a dosage calculator that recommends a dose based on individual patient details should be [109,110]. Developers and publishers wanting to have the ‘CE mark’ on their apps need to notify the MHRA in the UK (or a corresponding agency in other European countries), producing a ‘declaration of conformity’ that includes detailed technical documentation of how their app design conforms to the Medical Device Directive MDD 93/42/EEC. As part of the technical documentation, app developers will also need to have undertaken a controlled test and risk assessment to demonstrate that their app supports and improves upon any existing process used to present the same information or function. Once all the registration documentation is ready, developers and/or publishers should submit it to the MHRA with the appropriate registration fee (£70.00 GBP at time of writing) [109]. NHS (England) online Health Apps Library: a full-fledged ‘vetted app store’ is still a far distance away The National Health Service (NHS) in England runs an online Health Apps Library [111], where it lists and recommends some carefully selected apps such as ‘iBreastCheck’ (Breakthrough Breast Cancer) [112] and the NHS’ own ‘Health Choices’ app [113] (not all of the listed apps can be considered as medical devices). The NHS Health Apps Library only provides links to third-party stores hosting the actual apps. Developers can submit their apps for review and possible listing in the Library. Visitors can also review and rate apps in the NHS Health Apps Library. Many other app directory sites (and the app stores run by Amazon, Apple, Google, Microsoft and Research In Motion [RIM/BlackBerry]) allow users, including healthcare professionals, to review and rate apps, e.g., the ‘Medical App Journal’ directory [22]. http://ojphi.org/ Mobile medical and health apps: state of the art, concerns, regulatory control and certification 12 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014 OJPHI It has been proposed that the NHS should provide its own full-fledged ‘vetted app store’ [114,115]. A proper app store (cf. Apple iTunes Store or Google Play) would accept app submissions from developers and also deliver those apps (if approved) to users’ devices, while handling any payments that might be involved (in the case of paid apps). For the NHS, there would be the additional task of assessing and ensuring the quality of these apps from a medical/health point of view, e.g., ‘is the medical content or advice offered by an app sound, safe and up-to-date?’ This is in addition to the technical assessment of submitted apps, to determine if they behave as intended, without crashing the devices running them, and where applicable, if they are secure and protect user’s privacy. These are not trivial tasks, and could prove very demanding and well beyond the remits of the NHS, when one considers the thousands of medical and health apps that a specialised health apps store would have to deal with. Beyond ‘apps as a medical device’: desiderata for successful clinical and health-related apps Going outside the rather rigid and narrow limits of the definitions by the FDA and MHRA of ‘software (apps) as a medical device’ and the associated risks to patient safety, there are other aspects of clinical and health-related apps that deserve much attention. These aspects or factors are also equally important in the case of health-related and medical apps that are excluded from conventional (software as a) medical device regulations. These factors affect an app’s overall fitness for purpose, effectiveness and value-for-money (for paid apps), and can thus be seen as ‘desiderata or ingredients for app success’. Medical and health app reviewers, such as the curators of the above mentioned NHS Health Apps Library, should cover (and perhaps rate in some way) these aspects in any app review or assessment they undertake (where applicable), to allow end users to make informed comparisons and decisions about which apps to download and use. Furthermore, we recommend that these factors should be routinely considered by app developers and publishers, perhaps in the form of a checklist to be added to their existing quality assurance (QA) procedures (see also our discussion of Happtique Health App Certification Standards below), as a kind of industry self-regulation and/or voluntary certification. These app aspects cover content quality (for health education and reference apps), usability issues (including ‘cognitive accessibility’ and associated consumers’ general and health literacy issues), device connectivity standards, as well as a number of other issues that are fully covered in Happtique Health App Certification Standards [116,117], such as app security and user privacy. Regarding content quality, apps need to: (1) provide authorship information, including detailed information about authors’ affiliations and credentials and about any medical professional involvement in content preparation; (2) list all references or sources of content (attribution); (3) fully disclose any app sponsorship or other commercial funding arrangements, and any potential conflicts of interest; and (4) ensure a balanced, non-biased coverage of facts and information currency (up-to-datedness). These are the same essential criteria governing the quality benchmarking of online medical/health-related information resources and Web sites in general (prior to the apps era) [118,119]. For apps serving medical images, evaluators additionally need to establish that all the relevant ethical issues, such as obtaining informed consent from patients to publish their images in a smartphone app, were duly considered in the design of these apps [120]. http://ojphi.org/ Mobile medical and health apps: state of the art, concerns, regulatory control and certification 13 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014 OJPHI But high-quality, evidence-based content alone is of limited value, if presented in a way that does not adequately match and address the usability, accessibility, readability (reading with understanding) and health literacy needs of target audiences. An app that is perfectly usable by a younger person might be very difficult to manipulate by an older or disabled person with different and unique usability needs related to ageing and/or physical and cognitive impairment [2,121-126]. App designers and content developers tend to focus on the “more obvious” types of usability and accessibility (i.e., visual, auditory and motor), and often overlook or give little attention to the cognitive accessibility aspects of their content and user interfaces, which have to do with users’ average reading age [127] and general, digital and health literacy levels [128]. Almost half of all Europeans show limited health literacy, according to the European Health Literacy Survey results published in 2011 [129], and this should be taken into consideration when designing and evaluating apps intended for consumption by the general public. Presenting correct, unbiased information but in a way that is hard to understand by the intended audience not only renders this information useless, but also makes misunderstanding a likely possibility, which can have serious negative health consequences [128]. For apps that connect to devices, such as glucometers, heart rate and blood pressure monitors, integration with Continua Health Alliance’s [130] home and mobile telehealthcare ecosystem of devices from different manufacturers and suppliers would be highly desirable and future- proof. Frohner et al. [131] describe one such an app, and Microsoft ‘HealthVault’ apps for Windows Phone and Windows 8/RT [132] already support Continua certified devices [133]. Biometric fingerprint identification is now available on some smartphones and can help verifying and ensuring the identity of the person or patient using the device or a particular app running on it. Some apps allow patients to manually log and edit their health and lifestyle data before submitting these details electronically to their treating clinicians. Problems may arise in relation to selective and subjective reporting of data by patients and/or patient compliance in maintaining such manual logs, resulting in incomplete data being submitted to clinicians. Sensors can automate the logging of some clinical/health data and partially solve issues related to patient compliance and reporting, but patients may still forget or choose not to wear the corresponding sensor devices, particularly when perceived as intrusive, cumbersome and/or threatening their individual privacy. ‘Patient-centred design’ and appropriate involvement of end users in app and device design, which are now mandatory requirements explicitly stated in FDA regulation 21 CFR 820.30, can help mitigate these latter issues. Happtique Health App Certification Happtique [134] is a US mobile health (mHealth) solutions company aimed at integrating mHealth into patient care and daily life. The company released their ‘Happtique Health App Certification Program’ (HACP) in an attempt to address many of the above discussed app evaluation criteria, as well as some additional relevant and equally important aspects, beginning from where FDA (and MHRA) usually leave off. In other words, Happtique’s scope includes, but also goes beyond, the ‘software as a medical device’ definition used by FDA and MHRA/‘CE mark’. HACP is meant to assist healthcare providers and consumers in identifying medical, health and fitness apps that “deliver credible content, contain safeguards for user data, and function as described”. The final HACP Certification Standards and Associated Performance http://ojphi.org/ Mobile medical and health apps: state of the art, concerns, regulatory control and certification 14 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014 OJPHI Requirements were released on 27 February 2013. They assess operability, privacy, security (collectively referred to as the ‘Technical Standards’) and content (‘Content Standards’), and are available at [116,117,135]. App Operability (OP) Standards (OP1 to OP9) cover issues such as ensuring the app installs, launches, and runs consistently on target device(s) and operating system(s). HACP has some very comprehensive privacy requisites, App Privacy (P) Standards (P1 to P6), requiring, for example, the app to disclose to its users the type(s) and full details of all data it (or any in-app advertiser) collects or accesses on user devices, either pertaining to the usage of the app and/or to the specific user, including user-generated data and data that are collected automatically about the user, and how and by whom all that data are used [135]. App Security (S) Standards (S1 to S7) deal with issues such as verifying that the app, including without limitation, any advertisement displayed or supported through it, is free of malware. App Content (C) Standards (C1 to C11) require (among other things) that content of apps “be written and presented in a manner that is appropriate for the intended audience”. Content Standards also cover in-app advertisements, demanding that “an app that contains advertisements clearly identifies the advertising and complies with any and all applicable regulatory requirements, particularly advertisements that involve or relate to products or services that are clinical or related to health” [135]. Happtique has partnered with third party organisations to serve as HACP Partners for the evaluation of apps against the Certification Standards, based upon each organisation’s area of expertise. The testing of the Technical Standards was assigned to Intertek, a multinational inspection, product testing and certification company headquartered in London, United Kingdom [136]. Apps that meet all of the Technical Standards are then evaluated for the Content Standards by the Association of American Medical Colleges (AAMC) [137] and CGFNS (Commission on Graduates of Foreign Nursing Schools) International [138], with the help of clinical specialists selected based on the app’s specific subject matter [117]. HACP remains a voluntary programme. In order to qualify for submission, an app’s content must be written in English, run natively on iOS (Apple), Android, Blackberry or Windows devices, be intended for sale or use within the US (Happtique is currently solely focused on the US market), and pertain to at least one of the following areas: provision of healthcare, health monitoring and management, and/or advancement of healthcare or medical knowledge. Apps requiring any public or private certification, registration, clearance or similar approval (e.g., by FDA) must obtain those approvals prior to submitting an application for Happtique Certification [117]. Apps successfully meeting all of HACP’s Standards and Associated Performance Requirements are granted the Happtique Certification and Seal, valid for a two-year period and specifically associated with the app version that was submitted for evaluation. The app developer may then use the Seal for promotional, advertising or marketing purposes, as long as the Certification is valid and such use is in compliance with Happtique’s guidelines. All active Seals contain a link back to the ‘Happtique Certified App Directory’ that includes relevant information about the app and its certification history [117] (cf. Health On the Net Foundation’s HONcode certificate and seal [139]. http://ojphi.org/ Mobile medical and health apps: state of the art, concerns, regulatory control and certification 15 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014 OJPHI Conclusions Apps may have deficiencies and limits. App development, support, maintenance and regular updating may entail significant costs. Interactions may require substantial effort. Advice may not align with users’ expectations or life activities. Not all content may benefit all users, and getting users to download and engage with mobile apps is an art [1]. Health disparities and low health literacy and numeracy may negatively affect use [128]. Furthermore, many smartphone apps are not based on behavioural change theories or guidelines (where these could have made a significant, positive difference, if implemented) [30,44]. Relatively few studies exist on the effectiveness (clinically and cost-wise) of mobile smartphone apps and more research is needed to properly address this issue. Assessment of some aspects of specific apps or types of apps may require a full blown clinical trial or evaluation study and the necessary resources to conduct it, which is well beyond what can be evaluated by a single person or a few people using a checklist of criteria to look for. Software applications (apps) and ‘software as a medical device’ are not new concepts. But the mobile Social Web is now enabling millions of people to more easily share, rate, recommend, and find software applications about almost any topic under the sun (‘there is an app for that’). Before the advent of smartphones, small-form-factor tablets and the latest generations of mobile operating systems and Web browsers that support the concept of apps and associated ‘app stores’, downloading and installing software (including shareware [cf. trial versions of paid apps today] and freeware [cf. free apps]) was always possible, but not as easy or as popular (among average Internet users) as it is today. However, this ubiquity (ease-of-installation and popularity) of today’s apps is also bringing in additional risks to less experienced users who might find themselves tricked to download apps that contain malware, or violate their online privacy, or offer them dubious medical information and advice, hence the need to educate users, particularly the general public, and raise their awareness about the potential negative aspects of mobile apps and how to appraise the quality of an app before installing it and granting it permissions on their devices. Hogan and Kerin [140] consider it crucial to educate patients regarding the “unregulated and potentially harmful content of (some) apps”. Voluntary app certification schemes such as HACP, while very welcome (as standard setters in the domain), are reminiscent of the early days of the Web, when many “similar” quality benchmarking tools and codes of conduct for information publishers were proposed and developed to appraise and rate online medical and health information [141]. Indeed, many apps on offer today are nothing more than custom informational Web sites displayed in their own dedicated app screens (custom Web browser screens). It is probably impossible to rate and police every app on offer today, much like in those early days of the Web, when people quickly reached the same conclusion regarding Web pages offering medical and health information. The best first line of defence was, is, and will always be to educate consumers. Empowered patient groups and consumer leaders can play an important role in this respect, as seen, for example, in the recent release of the ‘European Directory of Health Apps 2012- 2013’ [142]. Postscript A new class of wearable smartphone-connected devices is emerging, namely smart watches (worn like a standard watch around the wrist) and optical head-mounted displays such as http://ojphi.org/ Mobile medical and health apps: state of the art, concerns, regulatory control and certification 16 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014 OJPHI Google Glass (worn like eyeglasses). These devices offer new, unique form factors and affordances, requiring dedicated apps to properly address the associated usability opportunities and limitations [143,144]. Smart watches are being used as physical activity trackers [143], while Google Glass has been experimentally deployed as ‘surgical assistant’ and for medical education purposes [144,145]. As these wearable devices and their corresponding apps continue to develop and mature, dedicated research and expert reviews will soon become necessary to investigate and document their various potential and current clinical and health applications. Authors' contributions MNKB conceived and planned the study, conducted the initial PubMed literature survey, and drafted the manuscript with contributions (mainly to the section entitled ‘Range of mobile applications’) from ACB, CK, DBB and RPD. All authors have read and approved the final manuscript. Competing interests The authors declare that they have no competing interests. Any reference herein to any specific commercial apps, devices, other products, or services by trade name, trademark, manufacturer, provider or otherwise does not necessarily constitute or imply its endorsement, recommendation or favouring by the authors. Acknowledgements Commercial products, services and company/brand names mentioned in this paper are trademarks and/or registered trademarks of their respective owners. References 1. Heron KE, Smyth JM. 2010. Ecological momentary interventions: incorporating mobile technology into psychosocial and health behaviour treatments. Br J Health Psychol. 15(1), 1-39. doi:http://dx.doi.org/10.1348/135910709X466063. PubMed 2. Boulos MN, Wheeler S, Tavares C, Jones R. 2011. How smartphones are changing the face of mobile and participatory healthcare: an overview, with example from eCAALYX. Biomed Eng Online. 10, 24. doi:http://dx.doi.org/10.1186/1475-925X-10- 24. PubMed 3. More Smartphones Were Shipped in Q1 2013 Than Feature Phones, An Industry First According to IDC (Press Release, 25 Apr 2013) - http://www.idc.com/getdoc.jsp?containerId=prUS24085413 4. Millar M. Market for feature phone apps is low-tech goldmine (BBC News, 7 May 2013) - http://www.bbc.co.uk/news/business-22390444 5. Fox S, Duggan M. Mobile Health 2012. Washington, DC, Pew Research Center’s Internet & American Life Project 2012. Available at http://pewinternet.org/~/media//Files/Reports/2012/PIP_MobileHealth2012_FINAL.pdf 6. Pelletier SG. Explosive Growth in Health Care Apps Raises Oversight Questions. AAMC Reporter 2012, Oct. Available at https://www.aamc.org/newsroom/reporter/october2012/308516/health-care-apps.html 7. Conn J. 2012. Most-healthful apps. Mod Healthc. 42(50), 30-32. PubMed 8. Epocrates - http://www.epocrates.com/ http://ojphi.org/ http://dx.doi.org/10.1348/135910709X466063 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=19646331&dopt=Abstract http://dx.doi.org/10.1186/1475-925X-10-24 http://dx.doi.org/10.1186/1475-925X-10-24 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=21466669&dopt=Abstract http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23323343&dopt=Abstract Mobile medical and health apps: state of the art, concerns, regulatory control and certification 17 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014 OJPHI 9. Lexicomp - http://www.lexi.com/ 10. UpToDate - http://www.uptodate.com/ 11. Medscape - http://www.medscape.com/public/mobileapp 12. O'Neill S, Brady RR. 2012. Colorectal smartphone apps: opportunities and risks. Colorectal Dis. 14(9), e530-34. PubMed http://dx.doi.org/10.1111/j.1463- 1318.2012.03088.x 13. Jenny JY. 2013. Measurement of the Knee Flexion Angle with a Smartphone- Application is Precise and Accurate. J Arthroplasty. 28(5), 784-87. doi:http://dx.doi.org/10.1016/j.arth.2012.11.013. PubMed 14. Eye Handbook - http://www.eyehandbook.com/ 15. Lord K, Shah VA, Krishna R. 2013. The Eye Handbook: a mobile app in ophthalmic medicine. Mo Med. 110(1), 49-51. PubMed 16. Outbreaks Near Me - http://www.healthmap.org/outbreaksnearme/ 17. Oehler RL, Smith K, Toney JF. 2010. Infectious diseases resources for the iPhone. Clin Infect Dis. 50(9), 1268-74. PubMed http://dx.doi.org/10.1086/651602 18. Franko OI, Tirrell TF. 2012. Smartphone app use among medical providers in ACGME training programs. J Med Syst. 36(5), 3135-39. PubMed http://dx.doi.org/10.1007/s10916-011-9798-7 19. Kane-Gill SL, Post JP, Smithburger PL, Seybert AL. 2012. "Bump": using a mobile app to enhance learning in simulation scenarios. Simul Healthc. 7(5), 326-27. PubMed http://dx.doi.org/10.1097/SIH.0b013e31825e8bcf 20. Hardyman W, Bullock A, Brown A, Carter-Ingram S, Stacey M. 2013. Mobile technology supporting trainee doctors' workplace learning and patient care: an evaluation. BMC Med Educ. 13, 6. PubMed http://dx.doi.org/10.1186/1472-6920-13-6 21. Waldmann UM, Weckbecker K: Smartphone Application of Primary Care Guidelines used in Education of Medical Students. GMS Zeitschrift fur medizinische Ausbildung 2013, 30(1):Doc6. 22. Microbiology and Immunology Wiz - http://www.medicalappjournal.com/app_details.php?appsname=Microbiology+and+Imm unology+Wiz+2013 23. Lange Microbiology & Infectious Diseases Flash Cards, 2nd Edition, by Modality Inc. (iTunes store) - https://itunes.apple.com/gb/app/lange-microbiology- infectious/id496299826?mt=8 24. Chang AY, Ghose S, Littman-Quinn R, Anolik RB, Kyer A, et al. 2012. Use of mobile learning by resident physicians in Botswana. Telemed J E Health. 18(1), 11-13. PubMed http://dx.doi.org/10.1089/tmj.2011.0050 25. Demidowich AP, Lu K, Tamler R, Bloomgarden Z. 2012. An evaluation of diabetes self- management applications for Android smartphones. J Telemed Telecare. 18(4), 235-38. PubMed http://dx.doi.org/10.1258/jtt.2012.111002 26. Nes AA, van Dulmen S, Eide E, Finset A, Kristjansdottir OB, et al. 2012. The development and feasibility of a web-based intervention with diaries and situational feedback via smartphone to support self-management in patients with diabetes type 2. Diabetes Res Clin Pract. 97(3), 385-93. PubMed http://dx.doi.org/10.1016/j.diabres.2012.04.019 27. Schap TE, Zhu F, Delp EJ, Boushey CJ. 2013. Merging dietary assessment with the adolescent lifestyle. J Hum Nutr Diet. doi:10.1111/jhn.12071. PubMed 28. Pellegrini CA, Duncan JM, Moller AC, Buscemi J, Sularz A, et al. 2012. Study protocol: A smartphone-supported weight loss program: design of the ENGAGED randomized controlled trial. BMC Public Health. 12, 1041. PubMed http://dx.doi.org/10.1186/1471- 2458-12-1041 http://ojphi.org/ http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=22646729&dopt=Abstract http://dx.doi.org/10.1111/j.1463-1318.2012.03088.x http://dx.doi.org/10.1111/j.1463-1318.2012.03088.x http://dx.doi.org/10.1016/j.arth.2012.11.013 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23499407&dopt=Abstract http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23457751&dopt=Abstract http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=20233061&dopt=Abstract http://dx.doi.org/10.1086/651602 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=22052129&dopt=Abstract http://dx.doi.org/10.1007/s10916-011-9798-7 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23032752&dopt=Abstract http://dx.doi.org/10.1097/SIH.0b013e31825e8bcf http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23336964&dopt=Abstract http://dx.doi.org/10.1186/1472-6920-13-6 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=22171597&dopt=Abstract http://dx.doi.org/10.1089/tmj.2011.0050 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=22604278&dopt=Abstract http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=22604278&dopt=Abstract http://dx.doi.org/10.1258/jtt.2012.111002 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=22578890&dopt=Abstract http://dx.doi.org/10.1016/j.diabres.2012.04.019 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23489518&dopt=Abstract http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23194256&dopt=Abstract http://dx.doi.org/10.1186/1471-2458-12-1041 http://dx.doi.org/10.1186/1471-2458-12-1041 Mobile medical and health apps: state of the art, concerns, regulatory control and certification 18 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014 OJPHI 29. Boulos MN, Yang SP. 2013. Exergames for health and fitness: the roles of GPS and geosocial apps. Int J Health Geogr. 12, 18. doi:http://dx.doi.org/10.1186/1476-072X-12- 18. PubMed 30. Abroms LC, Padmanabhan N, Thaweethai L, Phillips T. 2011. iPhone apps for smoking cessation: a content analysis. Am J Prev Med. 40(3), 279-85 PubMed http://dx.doi.org/10.1016/j.amepre.2010.10.032 31. McTavish FM, Chih MY, Shah D, Gustafson DH. 2012. How Patients Recovering From Alcoholism Use a Smartphone Intervention. J Dual Diagn. 8(4), 294-304. PubMed http://dx.doi.org/10.1080/15504263.2012.723312 32. m.Carat - http://www.caratnetwork.org/index.php?option=com_content&view=article&id=67&lan g=en 33. Burnay E, Cruz-Correia R, Jacinto T, Sousa AS, Fonseca J. 2013. Challenges of a mobile application for asthma and allergic rhinitis patient enablement-interface and synchronization. Telemed J E Health. 19(1), 13-18. PubMed http://dx.doi.org/10.1089/tmj.2012.0020 34. Palmier-Claus JE, Ainsworth J, Machin M, Barrowclough C, Dunn G, et al. 2012. The feasibility and validity of ambulatory self-report of psychotic symptoms using a smartphone software application. BMC Psychiatry. 12, 172. PubMed http://dx.doi.org/10.1186/1471-244X-12-172 35. Jacob E, Stinson J, Duran J, Gupta A, Gerla M, et al. 2012. Usability testing of a Smartphone for accessing a web-based e-diary for self-monitoring of pain and symptoms in sickle cell disease. J Pediatr Hematol Oncol. 34(5), 326-35. PubMed http://dx.doi.org/10.1097/MPH.0b013e318257a13c 36. Johnston NW, Lambert K, Hussack P, Gerhardsson de Verdier M, Higenbottam T, et al. 2013. Detection of COPD Exacerbations and Compliance with Patient Reported Daily Symptom Diaries Using a BlackBerry-Based Information System. Chest. doi:http://dx.doi.org/10.1378/chest.12-2308. 37. Sposaro F, Danielson J, Tyson G: iWander: An Android application for dementia patients. Conf Proc IEEE Eng Med Biol Soc 2010, 2010:3875-3878. 38. Ferrero NA, Morrell DS, Burkhart CN. 2013. Skin scan: a demonstration of the need for FDA regulation of medical apps on iPhone. J Am Acad Dermatol. 68(3), 515-16. PubMed http://dx.doi.org/10.1016/j.jaad.2012.10.045 39. Wolf JA, Moreau J, Akilov O, Patton T, English JC, et al. 2013. Diagnostic Inaccuracy of Smartphone Applications for Melanoma Detection. JAMA Dermatol. 149(4), 422-26. PubMed http://dx.doi.org/10.1001/jamadermatol.2013.2382 40. Curaudeau GA, Sharma N, Rovin RA. 2011. Development of an iPhone application for sideline concussion testing. Neurosurg Focus. 31(5), E4. PubMed http://dx.doi.org/10.3171/2011.8.FOCUS11186 41. Muessig KE, Pike EC, Legrand S, Hightow-Weidman LB. 2013. Mobile phone applications for the care and prevention of HIV and other sexually transmitted diseases: a review. J Med Internet Res. 15(1), e1. PubMed http://dx.doi.org/10.2196/jmir.2301 42. InSTEDD Remindem - http://remindem.instedd.org/en 43. Fjeldsoe BS, Marshall AL, Miller YD. 2009. Behavior change interventions delivered by mobile telephone short-message service. Am J Prev Med. 36(2), 165-73. doi:http://dx.doi.org/10.1016/j.amepre.2008.09.040. PubMed 44. Riley WT, Rivera DE, Atienza AA, Nilsen W, Allison SM, et al. 2011. Health behavior models in the age of mobile interventions: are our theories up to the task? Transl Behav Med. 1(1), 53-71. PubMed http://dx.doi.org/10.1007/s13142-011-0021-7 http://ojphi.org/ http://dx.doi.org/10.1186/1476-072X-12-18 http://dx.doi.org/10.1186/1476-072X-12-18 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23561306&dopt=Abstract http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=21335258&dopt=Abstract http://dx.doi.org/10.1016/j.amepre.2010.10.032 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23316127&dopt=Abstract http://dx.doi.org/10.1080/15504263.2012.723312 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23215639&dopt=Abstract http://dx.doi.org/10.1089/tmj.2012.0020 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23075387&dopt=Abstract http://dx.doi.org/10.1186/1471-244X-12-172 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=22627570&dopt=Abstract http://dx.doi.org/10.1097/MPH.0b013e318257a13c http://dx.doi.org/10.1378/chest.12-2308 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23394920&dopt=Abstract http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23394920&dopt=Abstract http://dx.doi.org/10.1016/j.jaad.2012.10.045 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23325302&dopt=Abstract http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23325302&dopt=Abstract http://dx.doi.org/10.1001/jamadermatol.2013.2382 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=22044103&dopt=Abstract http://dx.doi.org/10.3171/2011.8.FOCUS11186 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23291245&dopt=Abstract http://dx.doi.org/10.2196/jmir.2301 http://dx.doi.org/10.1016/j.amepre.2008.09.040 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=19135907&dopt=Abstract http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=21796270&dopt=Abstract http://dx.doi.org/10.1007/s13142-011-0021-7 Mobile medical and health apps: state of the art, concerns, regulatory control and certification 19 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014 OJPHI 45. Klasnja P, Pratt W. 2012. Healthcare in the pocket: mapping the space of mobile-phone health interventions. J Biomed Inform. 45(1), 184-98. doi:http://dx.doi.org/10.1016/j.jbi.2011.08.017. PubMed 46. Armstrong AW, Watson AJ, Makredes M, Frangos JE, Kimball AB, et al. 2009. Text- message reminders to improve sunscreen use: a randomized, controlled trial using electronic monitoring. Arch Dermatol. 145(11), 1230-36. doi:http://dx.doi.org/10.1001/archdermatol.2009.269. PubMed 47. Balato N, Megna M, Di Costanzo L, Balato A, Ayala F. 2013. Educational and motivational support service: a pilot study for mobile-phone-based interventions in patients with psoriasis. Br J Dermatol. 168(1), 201-05. PubMed http://dx.doi.org/10.1111/j.1365-2133.2012.11205.x 48. Suffoletto B, Calabria J, Ross A, Callaway C, Yealy DM. 2012. A mobile phone text message program to measure oral antibiotic use and provide feedback on adherence to patients discharged from the emergency department. Acad Emerg Med. 19(8), 949-58. PubMed http://dx.doi.org/10.1111/j.1553-2712.2012.01411.x 49. Redfern J, Thiagalingam A, Jan S, Whittaker R, Hackett M, Mooney J, De Keizer L, Hillis G, Chow C: Development of a set of mobile phone text messages designed for prevention of recurrent cardiovascular events. Eur J Prev Cardiol 2012, May. doi:10.1177/2047487312449416. 50. Kharrazi H, Chisholm R, VanNasdale D, Thompson B. 2012. Mobile personal health records: an evaluation of features and functionality. Int J Med Inform. 81(9), 579-93. PubMed http://dx.doi.org/10.1016/j.ijmedinf.2012.04.007 51. Microsoft HealthVault - https://www.healthvault.com/gb/en 52. DocbookMD - http://docbookmd.com/ 53. Conde C. 2012. An apt app. Tex Med. 108(11), 53-56. PubMed 54. Rocha A, Martins A, Freire JC. 2011. Junior, Kamel Boulos MN, Vicente ME, Feld R, van de Ven P, Nelson J, Bourke A, Olaighin G, Sdogati C, Jobes A, Narvaiza L, Rodríguez-Molinero A: Innovations in health care services: The CAALYX system. Int J Med Inform. doi:10.1016/j.ijmedinf.2011.03.003. 55. Kamel Boulos MN. Quo vadis smartphones and apps in emergency medicine? International Hospital and Equipment (IHE) 2012, 38:14-15. http://www.ihe- online.com/index.php?id=3315 56. Demaerschalk BM, Vegunta S, Vargas BB, Wu Q, Channer DD, et al. 2012. Reliability of real-time video smartphone for assessing National Institutes of Health Stroke Scale scores in acute stroke patients. Stroke. 43(12), 3271-77. PubMed http://dx.doi.org/10.1161/STROKEAHA.112.669150 57. Takao H, Murayama Y, Ishibashi T, Karagiozov KL, Abe T. 2012. A new support system using a mobile device (smartphone) for diagnostic image display and treatment of stroke. Stroke. 43(1), 236-39. PubMed http://dx.doi.org/10.1161/STROKEAHA.111.627943 58. ResolutionMD - http://www.calgaryscientific.com/resolutionmd/mobile-resmd/ 59. Demaerschalk BM, Vargas JE, Channer DD, Noble BN, Kiernan TE, et al. 2012. Smartphone teleradiology application is successfully incorporated into a telestroke network environment. Stroke. 43(11), 3098-101. PubMed http://dx.doi.org/10.1161/STROKEAHA.112.669325 60. Mitchell JR, Sharma P, Modi J, Simpson M, Thomas M, et al. 2011. A smartphone client-server teleradiology system for primary diagnosis of acute stroke. J Med Internet Res. 13(2), e31. PubMed http://dx.doi.org/10.2196/jmir.1732 http://ojphi.org/ http://dx.doi.org/10.1016/j.jbi.2011.08.017 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=21925288&dopt=Abstract http://dx.doi.org/10.1001/archdermatol.2009.269 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=19917951&dopt=Abstract http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23240729&dopt=Abstract http://dx.doi.org/10.1111/j.1365-2133.2012.11205.x http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=22849787&dopt=Abstract http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=22849787&dopt=Abstract http://dx.doi.org/10.1111/j.1553-2712.2012.01411.x http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=22809779&dopt=Abstract http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=22809779&dopt=Abstract http://dx.doi.org/10.1016/j.ijmedinf.2012.04.007 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23117875&dopt=Abstract http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23160878&dopt=Abstract http://dx.doi.org/10.1161/STROKEAHA.112.669150 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=21998052&dopt=Abstract http://dx.doi.org/10.1161/STROKEAHA.111.627943 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=22968466&dopt=Abstract http://dx.doi.org/10.1161/STROKEAHA.112.669325 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=21550961&dopt=Abstract http://dx.doi.org/10.2196/jmir.1732 Mobile medical and health apps: state of the art, concerns, regulatory control and certification 20 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014 OJPHI 61. Modi J, Sharma P, Earl A, Simpson M, Mitchell JR, et al. 2010. iPhone-based teleradiology for the diagnosis of acute cervico-dorsal spine trauma. Can J Neurol Sci. 37(6), 849-54. PubMed 62. Kumar S, Wang EH, Pokabla MJ, Noecker RJ. 2012. Teleophthalmology assessment of diabetic retinopathy fundus images: smartphone versus standard office computer workstation. Telemed J E Health. 18(2), 158-62. PubMed http://dx.doi.org/10.1089/tmj.2011.0089 63. Börve A, Holst A, Gente-Lidholm A, Molina-Martinez R, Paoli J. 2012. Use of the mobile phone multimedia messaging service for teledermatology. J Telemed Telecare. 18(5), 292-96. PubMed http://dx.doi.org/10.1258/jtt.2012.120206 64. Krpič A, Savanovic A, Cikajlo I. 2013. Telerehabilitation: remote multimedia-supported assistance and mobile monitoring of balance training outcomes can facilitate the clinical staff's effort. Int J Rehabil Res. 36(2), 162-71. doi:http://dx.doi.org/10.1097/MRR.0b013e32835dd63b. PubMed 65. Engel H, Huang JJ, Tsao CK, Lin CY, Chou PY, et al. 2011. Remote real-time monitoring of free flaps via smartphone photography and 3G wireless Internet: a prospective study evidencing diagnostic accuracy. Microsurgery. 31(8), 589-95. PubMed http://dx.doi.org/10.1002/micr.20921 66. Lau JK, Lowres N, Neubeck L, Brieger DB, Sy RW, et al. 2013. iPhone ECG application for community screening to detect silent atrial fibrillation: A novel technology to prevent stroke. Int J Cardiol. 165(1), 193-94. doi:http://dx.doi.org/10.1016/j.ijcard.2013.01.220. PubMed 67. Kamel Boulos MN. Smartphones in diabetes care and management. CyberTherapy & Rehabilitation (C&R) Magazine 2011, 4(3):20-21. http://www.cybertherapyandrehabilitation.com/past-issues/2011/issue-3- 2011/smartphones-in-diabetes-care-and-management/ 68. Cafazzo JA, Casselman M, Hamming N, Katzman DK, Palmert MR. 2012. Design of an mHealth app for the self-management of adolescent type 1 diabetes: a pilot study. J Med Internet Res. 14(3), e70. PubMed http://dx.doi.org/10.2196/jmir.2058 69. Potticary J. 2012. Smartphone apps set to provide next generation of blood-testing technology. Bioanalysis. 4(19), 2326. PubMed 70. Clayton S. Lifelens is testing for Malaria – on a Windows Phone (Next at Microsoft, 2 Aug 2011) - http://blogs.technet.com/b/next/archive/2011/08/02/lifelens-is-testing-for- malaria-on-a-windows-phone.aspx 71. Ozdalga E, Ozdalga A, Ahuja N. 2012. The smartphone in medicine: a review of current and potential use among physicians and students. J Med Internet Res. 14(5), e128. PubMed http://dx.doi.org/10.2196/jmir.1994 72. Glucool - Diabetes management for Android - http://www.glucool.com/ 73. OnTrack Diabetes - Android Apps on Google Play - https://play.google.com/store/apps/details?id=com.gexperts.ontrack&hl=en 74. Diabetes under control - dbees - http://dbees.com/ 75. Track3 Diabetes Planner - http://www.track3.com/android_diabetes_logbook/ 76. Johnson PT, Zimmerman SL, Heath D, Eng J, Horton KM, et al. 2012. The iPad as a mobile device for CT display and interpretation: diagnostic accuracy for identification of pulmonary embolism. Emerg Radiol. 19(4), 323-27. doi:http://dx.doi.org/10.1007/s10140-012-1037-0. PubMed 77. Park JB, Choi HJ, Lee JH, Kang BS. 2013. An Assessment of the iPad 2 as a CT Teleradiology Tool Using Brain CT with Subtle Intracranial Hemorrhage Under Conventional Illumination. J Digit Imaging. doi:http://dx.doi.org/10.1007/s10278-013- 9580-0. PubMed http://ojphi.org/ http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=21059550&dopt=Abstract http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=22304438&dopt=Abstract http://dx.doi.org/10.1089/tmj.2011.0089 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=22802521&dopt=Abstract http://dx.doi.org/10.1258/jtt.2012.120206 http://dx.doi.org/10.1097/MRR.0b013e32835dd63b http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23337324&dopt=Abstract http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=22072583&dopt=Abstract http://dx.doi.org/10.1002/micr.20921 http://dx.doi.org/10.1016/j.ijcard.2013.01.220 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23465249&dopt=Abstract http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23465249&dopt=Abstract http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=22564332&dopt=Abstract http://dx.doi.org/10.2196/jmir.2058 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23251915&dopt=Abstract http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23017375&dopt=Abstract http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23017375&dopt=Abstract http://dx.doi.org/10.2196/jmir.1994 http://dx.doi.org/10.1007/s10140-012-1037-0 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=22450843&dopt=Abstract http://dx.doi.org/10.1007/s10278-013-9580-0 http://dx.doi.org/10.1007/s10278-013-9580-0 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23404630&dopt=Abstract Mobile medical and health apps: state of the art, concerns, regulatory control and certification 21 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014 OJPHI 78. Mc Laughlin P, Neill SO, Fanning N, Mc Garrigle AM, Connor OJ, et al. 2012. Emergency CT brain: preliminary interpretation with a tablet device: image quality and diagnostic performance of the Apple iPad. Emerg Radiol. 19(2), 127-33. doi:http://dx.doi.org/10.1007/s10140-011-1011-2. PubMed 79. Bahsoun AN, Malik MM, Ahmed K, El-Hage O, Jaye P, et al. 2013. Tablet based simulation provides a new solution to accessing laparoscopic skills training. J Surg Educ. 70(1), 161-63. doi:http://dx.doi.org/10.1016/j.jsurg.2012.08.008. PubMed 80. Rassweiler JJ, Müller M, Fangerau M, Klein J, Goezen AS, et al. 2012. iPad-assisted percutaneous access to the kidney using marker-based navigation: initial clinical experience. Eur Urol. 61(3), 628-31. doi:http://dx.doi.org/10.1016/j.eururo.2011.12.024. PubMed 81. Abboud S, Weiss F, Siegel E, Jeudy J. 2013. TB or Not TB: interreader and intrareader variability in screening diagnosis on an iPad versus a traditional display. J Am Coll Radiol. 10(1), 42-44. doi:http://dx.doi.org/10.1016/j.jacr.2012.07.019. PubMed 82. Prue CS, Shannon KL, Khyang J, Edwards LJ, Ahmed S, et al. 2013. Mobile phones improve case detection and management of malaria in rural Bangladesh. Malar J. 12, 48. doi:http://dx.doi.org/10.1186/1475-2875-12-48. PubMed 83. Velthoven MH, Brusamento S, Majeed A, Car J. 2013. Scope and effectiveness of mobile phone messaging for HIV/AIDS care: a systematic review. Psychol Health Med. 18(2), 182-202. doi:http://dx.doi.org/10.1080/13548506.2012.701310. PubMed 84. Parker-Pope T. Giving Chronic Pain a Medical Platform of Its Own (18 Jul 2011) - http://well.blogs.nytimes.com/2011/07/18/giving-chronic-pain-a-medical-platform-of-its- own 85. Kristjánsdóttir OB, Fors EA, Eide E, Finset A, Stensrud TL, et al. 2013. A smartphone- based intervention with diaries and therapist-feedback to reduce catastrophizing and increase functioning in women with chronic widespread pain: randomized controlled trial. J Med Internet Res. 15(1), e5. doi:http://dx.doi.org/10.2196/jmir.2249. PubMed 86. Spyridonis F, Gronli TM, Hansen J, Ghinea G: Evaluating the usability of a virtual reality-based Android application in managing the pain experience of wheelchair users. Conf Proc IEEE Eng Med Biol Soc 2012, 2012:2460-3. doi:10.1109/EMBC.2012.6346462. 87. Stinson JN, Jibb LA, Nguyen C, Nathan PC, Maloney AM, et al. 2013. Development and testing of a multidimensional iPhone pain assessment application for adolescents with cancer. J Med Internet Res. 15(3), e51. doi:http://dx.doi.org/10.2196/jmir.2350. PubMed 88. Anonymous. 2012. Health apps and safety: views from recent sources. Health Devices. 41(10), 330-31. PubMed 89. Buijink AW, Visser BJ, Marshall L: Medical apps for smartphones: lack of evidence undermines quality and safety. Evid Based Med 2012, ebmed-2012-100885 (Published Online First: 25 Aug 2012). doi:10.1136/eb-2012-100885. 90. Cummings E, Borycki EM, Roehrer E. 2013. Issues and considerations for healthcare consumers using mobile applications. Stud Health Technol Inform. 183, 227-31. PubMed 91. McCartney M: How do we know whether medical apps work? BMJ 2013, 346:f1811 (correction in BMJ 2013, 346:f1974). 92. Visvanathan A, Hamilton A, Brady RR. 2012. Smartphone apps in microbiology--is better regulation required? Clin Microbiol Infect. 18(7), E218-20. PubMed http://dx.doi.org/10.1111/j.1469-0691.2012.03892.x 93. SkinVision - https://skinvision.com/ 94. Weaver C. Apps Aim to Detect Skin Cancer (The Wall Street Journal (WSJ), 16 Jan 2013) - http://online.wsj.com/article/SB10001424127887323783704578245973988828066.html http://ojphi.org/ http://dx.doi.org/10.1007/s10140-011-1011-2 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=22173819&dopt=Abstract http://dx.doi.org/10.1016/j.jsurg.2012.08.008 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23337687&dopt=Abstract http://dx.doi.org/10.1016/j.eururo.2011.12.024 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=22209052&dopt=Abstract http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=22209052&dopt=Abstract http://dx.doi.org/10.1016/j.jacr.2012.07.019 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23290673&dopt=Abstract http://dx.doi.org/10.1186/1475-2875-12-48 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23374585&dopt=Abstract http://dx.doi.org/10.1080/13548506.2012.701310 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=22788357&dopt=Abstract http://dx.doi.org/10.2196/jmir.2249 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23291270&dopt=Abstract http://dx.doi.org/10.2196/jmir.2350 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23475457&dopt=Abstract http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23444673&dopt=Abstract http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23388288&dopt=Abstract http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=22563840&dopt=Abstract http://dx.doi.org/10.1111/j.1469-0691.2012.03892.x Mobile medical and health apps: state of the art, concerns, regulatory control and certification 22 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014 OJPHI 95. Robson Y, Blackford S, Roberts D. 2012. Caution in melanoma risk analysis with smartphone application technology. Br J Dermatol. 167(3), 703-04. doi:http://dx.doi.org/10.1111/j.1365-2133.2012.11046.x. PubMed 96. Huckvale K, Car M, Morrison C, Car J. 2012. Apps for asthma self-management: a systematic assessment of content and tools. BMC Med. 10, 144. doi:http://dx.doi.org/10.1186/1741-7015-10-144. PubMed 97. McKinstry B: Currently available smartphone apps for asthma have worrying deficiencies. Evid Based Med 2013, Feb (Published Online First: 16 Feb 2013). doi:10.1136/eb-2012-101194. 98. O'Neill S, Brady RR. 2013. Clinical involvement and transparency in medical apps; not all apps are equal. Colorectal Dis. 15(1), 122. doi:http://dx.doi.org/10.1111/codi.12086. PubMed 99. US Food and Drug Administration. (FDA): Mobile Medical Applications (Last Updated: 22 Dec 2011) - http://www.fda.gov/MedicalDevices/ProductsandMedicalProcedures/ConnectedHealth/ MobileMedicalApplications/default.htm 100. US Food and Drug Administration. (FDA): Mobile Medical Applications: Draft Guidance for Industry and Food and Drug Administration Staff. Silver Spring, MD: US Department of Health & Human Services; 2011 (July). Available at http://www.fda.gov/downloads/MedicalDevices/DeviceRegulationandGuidance/Guidanc eDocuments/UCM263366.pdf 101. Cain M. 2012. One company’s experience: blazing the trail with the first FDA-approved medical imaging app. Biomed Instrum Technol. 46(Suppl), 87-90. PubMed http://dx.doi.org/10.2345/0899-8205-46.s2.87 102. US Food and Drug Administration. (FDA): FDA clears first diagnostic radiology application for mobile devices--Provides wireless access to medical images for iPhone, iPad users (4 Feb 2011) - http://www.fda.gov/NewsEvents/Newsroom/PressAnnouncements/2011/ucm242295.htm 103. FDA approves Welch Allyn foray into iPhone imaging--iExaminer iPhone PanOptic Ophthalmoscope (24 Jan 2013) - http://www.medgadget.com/2013/01/iexaminer-iphone- adapter-for-welch-allyns-panoptic-ophthalmoscope-cleared-in-u-s.html 104. Strickland E. The FDA Takes On Mobile Health Apps--Regulation is coming soon to health apps. Here’s what to expect. IEEE Spectrum 2012, Sep 12: online - http://spectrum.ieee.org/biomedical/devices/the-fda-takes-on-mobile-health-apps 105. ONCOassist - http://oncoassist.com/ 106. European Commission. Enterprise and Industry: CE marking - http://ec.europa.eu/enterprise/policies/single-market-goods/cemarking/index_en.htm 107. Irish app certified by CE as a medical device (6 Mar 2013) - http://www.siliconrepublic.com/innovation/item/31753-medtech 108. Medical Device Directive. (93/42/EEC) - EUR-Lex - http://eur- lex.europa.eu/LexUriServ/LexUriServ.do?uri=CONSLEG:1993L0042:20071011:en:PD F 109. When is an app classed as a medical device? (Mar 2013) - http://www.geneticdigital.co.uk/2013/03/when-should-an-app-be-classed-as-a-device/ 110. Medicines and Healthcare Products Regulatory Agency (MHRA). Medical Device Technology Forum on the use of software as a medical device (12 May 2010) - http://www.mhra.gov.uk/Howweregulate/NewTechnologiesForums/DevicesNewTechnol ogyForum/Forums/CON084987 111. NHS Health Apps Library - http://apps.nhs.uk/ 112. iBreastCheck (NHS page) - http://apps.nhs.uk/app/ibreastcheck/ http://ojphi.org/ http://dx.doi.org/10.1111/j.1365-2133.2012.11046.x http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=22762381&dopt=Abstract http://dx.doi.org/10.1186/1741-7015-10-144 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23171675&dopt=Abstract http://dx.doi.org/10.1111/codi.12086 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23173845&dopt=Abstract http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23173845&dopt=Abstract http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23039783&dopt=Abstract http://dx.doi.org/10.2345/0899-8205-46.s2.87 Mobile medical and health apps: state of the art, concerns, regulatory control and certification 23 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014 OJPHI 113. NHS Health Choices app - http://apps.nhs.uk/app/health-choices/ 114. Read C: Efficiency. The NHS app store. Health Serv J 2012, 122(6332): Suppl 2-5. 115. HANDI (The Healthcare App Network for Development and Innovation). Proposed NHS App Store (18 Oct 2012) - http://handihealth.org/proposed-nhs-app-store/ 116. Horowitz BT. Happtique Publishes Final Standards for Mobile Health App Certification (4 Mar 2013) - http://www.eweek.com/mobile/happtique-publishes-final-standards-for- mobile-health-app-certification/ 117. Happtique: Happtique Health App Certification Standards (Final, 27 Feb 2013) - http://www.happtique.com/app-certification/ - Direct URL to PDF document (October 2013): http://www.happtique.com/docs/HACP_Certification_Standards.pdf 118. Kamel Boulos MN. On quality benchmarking of online medical/health-related information resources (March 2004) - http://healthcybermap.org/MNKB_Quality.PDF 119. Wac K. 2012. Smartphone as a personal, pervasive health informatics services platform: literature review. Yearb Med Inform. 7(1), 83-93. PubMed 120. Payne KF, Tahim A, Goodson AM, Delaney M, Fan K. 2012. A review of current clinical photography guidelines in relation to smartphone publishing of medical images. J Vis Commun Med. 35(4), 188-92. doi:10.3109/17453054.2012.747174. PubMed 121. Caprani N, O’Connor NE, Gurrin C. Chapter 5: Touch Screens for the Older User. In Assistive Technologies. Edited by Auat Cheein FA. Intech; 2012:95-118 (ISBN 978-953- 51-0348-6). Available at http://www.intechopen.com/books/assistive- technologies/touch-screens-for-the-older-user 122. Guillaume L, Nadine V. Influence of age and interaction complexity on touch screen. In Proceedings of 2010 12th IEEE International Conference on e-Health Networking Applications and Services (Healthcom): 1-3 July 2010; Lyon, France. 2010:246-253. doi:10.1109/HEALTH.2010.5556560. 123. Phiriyapokanon T. Is a big button interface enough for elderly users?: Towards user interface guidelines for elderly users. Lambert Academic Publishing; 2011 (ISBN-13: 978-3845421667). 124. Al-Razgan MS, Al-Khalifa HS, Al-Shahrani MD, AlAjmi HH. Touch-Based Mobile Phone Interface Guidelines and Design Recommendations for Elderly People: A Survey of the Literature. In Neural Information Processing (19th International Conference, ICONIP 2012: 12-15 November 2012; Doha, Qatar), Proceedings, Part IV - Lecture Notes in Computer Science. Springer; 2012, 7666:568-574. doi:10.1007/978-3-642- 34478-7_69. 125. Mertens A, Hurtmanns J, Wacharamanotham C, Kronenbürger M, Borchers J, et al. 2012. Swabbing: touchscreen-based input technique for people with hand tremor. Work. 41(Suppl 1), 2405-11. doi:10.3233/WOR-2012-0474-2405. PubMed 126. Zhou J, Patrick Rau P-L, Salvendy G. 2012. Use and Design of Handheld Computers for Older Adults: A Review and Appraisal. Int J Hum Comput Interact. 28(12), 799-826. doi:http://dx.doi.org/10.1080/10447318.2012.668129. 127. Boulos MN. 2005. British Internet-derived Patient Information on Diabetes Mellitus: Is it Readable? Diabetes Technol Ther. 7(3), 528-35. doi:http://dx.doi.org/10.1089/dia.2005.7.528. PubMed 128. Kamel Boulos MN. Social media and mobile health (Chapter 13, under Part B: Taking action to create and strengthen health literacy-friendly settings). In: Ilona Kickbusch, Jürgen Pelikan, Franklin Apfel and Agis Tsouros (Editors). THE SOLID FACTS - Health Literacy. Copenhagen, Denmark: World Health Organization Regional Office for Europe, 2013, pp. 63-67. Available at http://www.euro.who.int/__data/assets/pdf_file/0008/190655/e96854.pdf - URL of unabridged chapter version: http://ow.ly/kpn43 http://ojphi.org/ http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=22890347&dopt=Abstract http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23278420&dopt=Abstract http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=22317077&dopt=Abstract http://dx.doi.org/10.1080/10447318.2012.668129 http://dx.doi.org/10.1089/dia.2005.7.528 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=15929685&dopt=Abstract Mobile medical and health apps: state of the art, concerns, regulatory control and certification 24 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014 OJPHI 129. Almost half of Europeans possess limited health literacy (Maastricht University - press release, 22 Nov 2011) - http://www.maastrichtuniversity.nl/web/Main/Sitewide/PressRelease/AlmostHalfOfEuro peansPossessLimitedHealthLiteracy.htm 130. Continua Health Alliance - http://www.continuaalliance.org/ 131. Frohner M, Urbauer P, Forjan M, Pohn B, Gerbovics F, et al. 2012. Development of an Android App in Compliance with the Continua Health Alliance Design Guidelines for Medical Device Connectivity in mHealth. Biomed Tech (Berl). 57(Suppl 1.). doi:10.1515/bmt-2012-4203. PubMed 132. Microsoft HealthVault. (Windows 8/RT) - http://apps.microsoft.com/windows/en- US/app/healthvault/728f1c88-7e2f-4b40-95c1-74fc09983689 133. Van Hoof B. Connecting Devices to HealthVault (June 2009) - http://download.microsoft.com/download/0/4/9/0498cecf-d0b1-4a72-b9b7- 17eb7d7ada98/Van_Hoof_Connecting_Devices.pdf 134. Happtique - http://www.happtique.com/ 135. Happtique: Health App Certification Program Standards & Performance Requirements - A Side-By-Side Comparison Of Published Draft And Final Standards/Performance Requirements (27 Feb 2013) - http://www.happtique.com/wp- content/uploads/Happtique-Side-by-side.pdf 136. Intertek - http://www.intertek.com/ 137. Association of American Medical Colleges (AAMC) - https://www.aamc.org/ 138. CGFNS International - http://cgfns.org/ 139. HONcode certificate & seal - http://www.hon.ch/HONcode/Patients/sample_certificate.html 140. Hogan NM, Kerin MJ. 2012. Smart phone apps: smart patients, steer clear. Patient Educ Couns. 89(2), 360-61. PubMed http://dx.doi.org/10.1016/j.pec.2012.07.016 141. Boulos MN, Roudsari AV, Gordon C, Muir Gray JA. 2001. The use of quality benchmarking in assessing web resources for the dermatology virtual branch library of the National electronic Library for Health (NeLH). J Med Internet Res. 3(1), e5. doi:http://dx.doi.org/10.2196/jmir.3.1.e5. PubMed 142. PatientView. European Directory of Health Apps 2012-2013 - A review by patient groups and empowered consumers. London, UK: PatientView; 2012. Available at http://www.patient- view.com/uploads/6/5/7/9/6579846/pv_appdirectory_final_web_300812.pdf 143. Google. + posts by MN Kamel Boulos on smart watches - https://plus.google.com/u/0/100165894171992699139/posts/ir1iRBjbm4t 144. Google. + posts by MN Kamel Boulos on Google Glass - https://plus.google.com/u/0/100165894171992699139/posts/7Y1G13Dphrg 145. Vallurupalli S, Paydak H, Agarwal SK, Agrawal M, Assad-Kottner C. 2013. Wearable technology to improve education and patient outcomes in a cardiology fellowship program - a feasibility study. Health Technol. 3(4), 267-70. doi:http://dx.doi.org/10.1007/s12553-013-0065-4. http://ojphi.org/ http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23096286&dopt=Abstract http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=22897983&dopt=Abstract http://dx.doi.org/10.1016/j.pec.2012.07.016 http://dx.doi.org/10.2196/jmir.3.1.e5 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=11720947&dopt=Abstract http://dx.doi.org/10.1007/s12553-013-0065-4