Dermatology: Practical and Conceptual Dermatology Practical & Conceptual Research | Dermatol Pract Concept. 2021;11(2):e2021033 1 Acceptance of Teledermoscopy by General Practitioners and Dermatologists in Denmark Tine Vestergaard1, Merethe K. Andersen2, Anette Bygum3 1 Department of Dermatology and Allergy Centre & Odense Patient Data Explorative Network, Odense University Hospital, Odense, Denmark 2 Audit Project Odense, Research Unit of General Practice, University of Southern Denmark, Odense, Denmark 3 Department of Dermatology and Allergy Centre, Odense University Hospital, Odense, Denmark Key words: provider satisfaction, teledermoscopy, teledermatology, skin cancer, malignant melanoma, general practitioner Citation: Vestergaard T, Andersen MK, Bygum A. Acceptance of teledermoscopy by general practitioners and dermatologists in Denmark. Dermatol Pract Concept. 2021;11(2):e2021033. DOI: https://doi.org/10.5826/dpc.1102a33 Accepted: October 30, 2020; Published: April 12, 2021 Copyright: ©2021 Vestergaard et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License BY-NC-4.0, which permits unrestricted noncommercial use, distribution, and reproduction in any medium, provided the original authors and source are credited. Funding: None. Competing interests: The authors have no conflicts of interest to disclose. Authorship: All authors have contributed significantly to this publication. Corresponding author: Tine Vestergaard, MD, Odense University Hospital, Kløvervænget 15, 5000 Odense C, Denmark. Email: tine.vestergaard@rsyd.dk Background: Teledermoscopy can be used to triage referrals of suspected skin cancers, thereby reduc- ing waiting time and number of face-to-face consultations with a dermatologist. However, the success of the implementation of this technology in part relies on the acceptance of the providers. Objectives: This study assessed the attitudes towards teledermoscopy of referring general practi- tioners and consultant dermatologists. Methods: General practitioners from 48 practices and 3 dermatologists in the region of Southern Denmark, who had previous experience with teledermoscopy, were invited to answer questionnaires on their acceptance of the technology. Results: General practitioners from 23 practices responded. All domains of the questionnaire received high scores, indicating a high degree of acceptance of teledermoscopy among respondents. All 3 der- matologists agreed that teledermoscopy was useful for triaging referrals, but they were less confident in their diagnoses and management plans proposed by teledermoscopy than in traditional face-to-face evaluations of patients. Two of the 3 dermatologists were satisfied with using teledermoscopy as a consult method. Conclusions: This study reports high levels of provider acceptance of teledermoscopy. However, a low response rate among general practitioners may limit its generalizability. ABSTRACT 2 Research | Dermatol Pract Concept. 2021;11(2):e2021033 Introduction The rising incidence of skin cancer affects the lives of many and poses a burden to health care systems across the globe [1,2]. In Denmark, persons who suspect they might have skin cancer usually consult their general practitioner (GP), who acts as a gatekeeper in triaging these lesions for further evaluation by a dermatologist or plastic surgeon. However, early signs of skin cancer may be subtle, and a sensitivity of 50%-55% for the detection of malignant melanoma has been reported for GPs [3]. Telemedicine is useful in dermatology, a medical specialty that relies on visual inspection and pattern recognition [4]. However, telemedicine is currently not recommended in Denmark for the evaluation of possible skin cancers. In the examination of skin tumors, dermatologists use dermoscopy, which has been shown to increase the diagnostic sensitivity and specificity [5]. Given the right equipment, GPs can take dermoscopic images and forward them for evaluation by a dermatologist. This has been reported to be useful as a triage tool and can reduce waiting times [6]. We previously studied the diagnostic accuracy of teleder- matology, including teledermoscopy, in the region of Southern Denmark in 2018 [7]. In the current study, we assessed the acceptance of this new technology among participating GPs. In addition, we collected the views of the participating derma- tologists. Satisfaction of both patient and provider is believed to be an important factor in the successful implementation of novel technologies [8,9]. We previously reported that almost 90% of patients were satisfied with, or neutral towards, the use of teledermoscopy [10]. Materials and Methods Prior Diagnostic Accuracy Study GPs from 50 practices in the region of Southern Denmark had been included in the diagnostic accuracy study [7]. Background information on age, sex, years working as a GP, interest in dermatology and dermoscopy, and distance to the nearest dermatologist was gathered in January 2018. During the study, 2 general practices dropped out and 1 general practice did not include any patients. A total of 519 patients with 600 possible skin cancers were included. On average, each practice photographed 12.5 lesions (range, 0-41) with an iPhone and Handyscope, which were sent for evaluation by a dermatologist using the FotoFinder hub [11]. The study was initialized with a teaching session on dermoscopy by the primary investigator in January 2018 and was completed with an evaluation and feedback session in December 2018 as proposed by the APO method [12]. GPs from 29 gen- eral practices attended this meeting. In February 2019, we e-mailed an electronic questionnaire on provider acceptance of teledermoscopy to all 48 participating general practices, with a reminder 2 weeks later. Questionnaire for GPs The questionnaire used to assess the GPs’ acceptance of teledermoscopy was the modified Technology Acceptance Model (TeleTAM) developed by Orruño et al [13]. TeleTAM was developed based on existing theories on adaptation and acceptance of new technologies, and was face- and content-validated by experts. It consists of 33 questions exploring 8 domains believed to influence the acceptance of teledermatology by physicians (Table 1). The providers’ future intention to use teledermatology (INT) is the main outcome. The other 7 domains consist of items related to the individual context (compatibility, attitude), the technological context (perceived usefulness, perceived ease of use, habits), and the organizational context (facilitators, subjective norm). At the end of the questionnaire, GPs had the option to com- ment in free writing. We translated the English version of the questionnaire into Danish after a forward translation and back-translation process. The word “teledermatology” in the original ques- tionnaire was replaced by “teledermoscopy” in this study. Answers to the questionnaire were given on a 7-point Likert scale ranging from strongly disagree to strongly agree. Ques- tionnaires were created, sent and stored in REDCap (Research Electronic Data Capture) hosted at OPEN (Open Patient data Explorative Network), Odense University Hospital [14,15]. We speculated that a GP’s intention to use teledermoscopy might be influenced by having a dermoscope or a special interest in dermatology before the study, or by the distance to the nearest dermatologist. Questionnaire for Dermatologists Three dermatologists who had participated in the diagnostic accuracy study on teledermoscopy were asked about their views on this novel procedure via a short questionnaire. They had 1, 4 and 9 years of experience post-specialization, and had reviewed 25.8%, 24.8% and 26.2% of the cases referred for teledermoscopy, respectively. A fourth dermatologist, who was an investigator in this study and hence not surveyed, had evaluated the remaining photos. To evaluate the dermatologists’ perceptions of teleder- moscopy, the 5 questions previously applied by Whited and colleagues were used [16]. These questions were considered to have face validity as they treated issues pivotal to the future implementation of teledermoscopy. A paper form with the original English wording and a Danish translation was used. Since the dermatologists evaluated both clinical and dermo- scopic images of each skin lesion, the word “teledermatology” was maintained. Research | Dermatol Pract Concept. 2021;11(2):e2021033 3 Statistical Analysis STATA version 16.0 was used for all statistical methods. P values <.05 were considered statistically significant. In con- cordance with the study by Orruño et al [13], a score was calculated for each domain as the mean of the scores of the questions related to that domain. For the logistic regression analysis, the dependent variable (INT) was dichotomized into 0 = low/moderate and 1 = high intention to use teleder- moscopy. As in the study by Orruño et al [13], we chose the median as the cut-off point between the groups; a score ≥6 was rated as high intention. Because only 29 observations were available, the number of explanatory variables needed to be reduced for the regres- sion analyses. Based on previous studies, 3 domains, namely perceived usefulness (PU), perceived ease of use (PEU) and facilitators (FAC), were considered most important [13,17]. Orruño et al [13] only found FAC to be a significant predictor of INT. PU and PEU were dimensions in the original Tech- nology Acceptance Model proposed by Davis [18], and both domains had high correlations with INT. We found collinear- ity between the variables PU and PEU and decided to reduce our statistical model further by excluding PEU. Cronbach alpha and inter-item correlation were calcu- lated to elucidate the reliability and validity of the TeleTAM questionnaire. Results General Practitioners’ Responses Twenty-nine GPs from 23 different practices completed the TeleTAM questionnaire. GPs from another two practices reporting on technical problems did not respond, for a total of 68 non-respondents. Background information on respon- dents and non-respondents, collected during the diagnostic accuracy study, is shown in Table 2. Respondents were sig- nificantly older than non-respondents and there was a trend towards respondents being more interested in dermatology. Scores on the TeleTAM questionnaire, for the 8 domains, and Cronbach alpha and inter-item correlation are shown in Table 3. All domains received high scores, indicating a high degree of acceptance of teledermoscopy among respondents. Cronbach alpha was acceptably high for all variables except FAC and COM. Correlation with INT was moderate to high for all domains except HAB. Logistic regression with a very reduced model due to the low number of observations did not reveal PU or FAC as significant predictors of INT. Having a special interest in dermatology and the distance to the nearest dermatologist did not significantly predict high INT. Interestingly, GPs who did not own a dermoscope were 10-times more likely to report high INT (95% CI, 1.3-78.1, P = .03). A few comments in free writing were received. Two GPs stressed the lack of reimbursement as a disadvantage, and one mentioned the cost of the equipment as a barrier. One GP commented that he still lacked routine in dermoscopy. One GP commented on the questionnaire and wanted the option to answer “not relevant.” One GP commented that he had invested in a Handyscope. Dermatologists’ Responses The results of the questionnaire for dermatologists are shown in Figure 1. All 3 dermatologists agreed that teledermoscopy was useful for triaging referrals, but they were less confident in their teledermoscopy diagnoses and management plans than in the traditional face-to-face evaluations of patients. Two of the 3 dermatologists were satisfied with teledermos- copy as a consult method. Discussion Nowadays, in the era of social distancing, the interest in teledermoscopy is definitely increasing, not only among dermatologists but also among GPs. Together, these health- care providers proved to easily use a common web platform for teledermoscopy and research purposes [19,20]. In this Table 1. The 8 Domains of the TeleTAM Questionnaire Domain Example questiona PU: perceived usefulness Teledermoscopy could help me to diagnose my patients more rapidly. PEU: perceived ease of use I think that teledermoscopy is a flexible technology to interact with. FAC: facilitators I would use teledermoscopy if I receive adequate training. COM: compatibility The use of teledermoscopy is compatible with my work habits. SN: subjective norm Most of my colleagues will welcome the fact that I use teledermoscopy. HAB: habits I feel comfortable with information and communication technologies. ATT: attitude The use of teledermoscopy is beneficial for the diagnosis of my patients. INT: intention to use I have the intention to use teledermoscopy routinely with my patients. a The questionnaire comprises 33 questions overall. One example question is shown for each domain. 4 Research | Dermatol Pract Concept. 2021;11(2):e2021033 Table 2. Background Information on Participating GPs, According to Whether or Not They Responded to the TeleTAM Questionnairea Characteristic Respondents (n=29) Non-respondents (n=68) PbNo.a Value No.a Value Male sex, n (%) 28 18 (64) 68 31 (46) .10 Age, mean (SD), y 24 50.7 (8.6) 68 46.2 (10.6) .04 Years working as GP, mean (SD) 22 13.7 (9.4) 63 9.7 (9.3) .10 Special interest in dermatology 22 14(63.6) 61 25 (41.0) .07 Have a dermoscope 23 11(47.8) 62 28 (45.2) .83 Distance to nearest dermatologist, mean (SD), km 23 9.8 (11.2) 64 9.3 (11.2) .85 a Number of persons who provided the requested information; b P values calculated by two-sample t test with unequal variance. GP = general practitioner; SD = standard deviation. Table 3. GPs’ Scores on the TeleTAM Questionnaire, and Crohnbach Alpha and Correlation with Intention to use Teledermoscopy Variable Observations, n Score, Mean (SD) Minimum Scorea Cronbach alpha Correlation with INT PU 28 6.10 (0.65) 4.83 0.91 0.80 PEU 29 5.75 (0.80) 4.50 0.89 0.65 FAC 29 6.03 (0.57) 4.67 0.35 0.67 COM 29 5.15 (0.63) 3.75 0.52 0.53 SN 28 5.71 (0.84) 4.00 0.84 0.63 HAB 29 5.84 (0.96) 4.00 0.71 0.38 ATT 29 6.15 (0.57) 5.00 0.80 0.71 INT 28 6.07 (0.77) 4.33 0.90 1.00 a The maximum score for every variable was 7.00. Figure 1. Dermatologists’ views on teledermoscopy (n = 3). *One dermatologist answered both neutral and disagree. I AM LESS CONFIDENT IN MY DIAGNOSES AND MANAGEMENT PLANS USING TELEDERMATOLOGY THAN SEEING PATIENTS IN CLINIC. TELEDERMATOLOGY IS A MORE EFFICIENT USE OF THE TIME THAT I SPEND AS A CONSULTANT. TELEDERMATOLOGY CONSULTS TAKE LONGER TO PERFORM THAN DO CLINIC VISITS. TELEDERMATOLOGY MAKES IT EASIER TO TRIAGE PATIENTS TO CLINIC APPOINTMENTS COMPARED TO TRADITIONAL REFERRALS. OVERALL, I AM SATISFIED WITH USING TELEDERMATOLOGY AS A CONSULT METHOD. 100% 0% 0% 0% 0% 0% 20% 40% 60% 80% 100% 17%* 17%* 0% 0% 0% 33% 67% 67% 100% 100% AGREE NEUTRAL DISAGREE Research | Dermatol Pract Concept. 2021;11(2):e2021033 5 questionnaire study, Danish GPs were asked about their atti- tudes towards the use of teledermoscopy. Overall, there was a very positive attitude towards this new technology, with mean scores for all domains above what has previously been reported on the TeleTAM questionnaire [13,17]. While high levels of satisfaction with teledermatology have been reported for both primary care physicians and dermatologists, only a few studies specifically evaluated teledermoscopy [21]. Ken- ney et al found high levels of satisfaction with teledermoscopy in a survey of 5 primary care physicians in a hospital setting, where a trained nurse photographer took the photos [22]. In a study of mixed quantitative and qualitative measures of provider satisfaction with teledermoscopy, Janda et al concluded that most participants were receptive to the use of mobile teledermoscopy in their practice [23]. Both GPs and dermatologists were surveyed. The study elucidated advan- tages and disadvantages of teledermoscopy and touched on themes such as lesion monitoring, time consumption and record keeping, which we recognize from the comments in the TeleTAM questionnaire and feedback obtained at the evaluation session. Orruño et al found FAC to be the only significant predic- tor of INT [13]. We could not replicate this result, possibly due to the low number of observations. However, we found the lowest Cronbach alpha for FAC, indicating that the reliability of this domain may be low in different health care settings. We also found low reliability for COM, as did Strat- ton and Loescher [17]. Furthermore, Stratton and Loescher reported low reliability for HAB. In our study, the Cronbach alpha for this domain was acceptably high. The validity of the TeleTAM questionnaire seems good. We found the lowest correlation with INT for HAB; this corresponds to the find- ings of both Orruño et al [13] and Stratton and Loescher [17]. We found that GPs without a dermoscope were more likely to have high intentions to use teledermoscopy. This may be because GPs already using dermoscopy have higher confidence in their diagnoses and have less need for specialist assistance through teledermoscopy or a standard referral, as shown by Chappuis et al [24]. Another study reported that 82% of GPs using dermoscopy were confident or very confi- dent in its use [25]. The distance to the nearest dermatologist did not influence INT. However, distances to dermatologist are relatively short in all of Denmark compared to Australia and some Nordic countries, where teledermoscopy may be even more warranted than in Denmark. A major limitation to this study is the low number of observations. GPs have limited time and an increasing work- load; we speculate that this may be an explanation for the low participation rate. Other studies reported response rates of 2%, 62% and 100% in different set-ups, with the highest response rate reported in a study of 5 primary care physicians in a hospital setting [13,22,26]. Furthermore, respondents and non-respondents in our study differed in several ways. Firstly, respondents were older than non-respondents. The background information was collected at the beginning of the diagnostic accuracy study in January 2018. More than 90% of the general practices employed GPs in training (data not shown). In Denmark, this type of employment lasts 6 or 12 months. Therefore, many of these young doctors, who had potentially participated in the diagnostic accuracy study, were no longer employed at the participating practices at the time of this questionnaire study and hence could not answer the TeleTAM questionnaire. This may also in part explain the low number of respondents. In addition, more respondents had a special interest in dermatology, which may bias their attitude towards teledermoscopy. Therefore, our results may not be generalizable, and Danish GPs as a whole may not be as positively minded towards this new technology. However, as some of the barriers towards the use of teledermoscopy may be lacking remunerations, this could be an incentive worth taking into account in future collective bargaining. Finally, our results may not be directly transferable to primary care settings in other parts of the world. In conclusion, this study found a high acceptance of teledermoscopy among GPs and dermatologists. More studies investigating provider accep- tance and satisfaction with teledermoscopy are warranted. References 1. Gordon LG, Rowell D. Health system costs of skin cancer and cost-effectiveness of skin cancer prevention and screening: a systematic review. Eur J Cancer Prev. 2015;24(2):141-149. DOI: 10.1097/CEJ.0000000000000056. PMID: 25089375. 2. Noels E, Hollestein L, Luijkx K, et al. Increasing Costs of Skin Can- cer due to Increasing Incidence and Introduction of Pharmaceuti- cals, 2007-2017. Acta Derm Venereol. 2020;100(10):adv00147. DOI: 10.2340/00015555-3463. PMID: 32189004. 3. Westerhoff K, McCarthy WH, Menzies SW. Increase in the sensi- tivity for melanoma diagnosis by primary care physicians using skin surface microscopy. Br J Dermatol. 2000;143(5):1016-1020. DOI: 10.1046/j.1365-2133.2000.03836.x. PMID: 11069512. 4. Lee KJ, Finnane A, Soyer HP. Recent trends in teledermatology and teledermoscopy. Dermatol Pract Concept. 2018;8(3):214- 223. DOI: 10.5826/dpc.0803a013. PMID: 30116667. 5. Kittler H, Pehamberger H, Wolff K, Binder M. Diagnostic accu- racy of dermoscopy. Lancet Oncol. 2002;3(3):159-165. DOI: 10.1016/S1470-2045(02)00679-4. 6. Dahlen Gyllencreutz J, Paoli J, Bjellerup M, et al. Diagnostic agreement and interobserver concordance with teledermoscopy referrals. J Eur Acad Dermatol Venereol. 2017;31(5):898-903. DOI: 10.1111/jdv.14147. PMID: 28150389. 7. Vestergaard T, Prasad SC, Schuster A, Laurinaviciene R, Andersen MK, Bygum A. Diagnostic accuracy and interobserver concor- dance: teledermoscopy of 600 suspicious skin lesions in Southern Denmark. J Eur Acad Dermatol Venereol. 2020;34(7):1601-1608. DOI: 10.1111/jdv.16275. PMID: 32031277. 8. Schwamm LH. Telehealth: seven strategies to successfully implement disruptive technology and transform health care. 6 Research | Dermatol Pract Concept. 2021;11(2):e2021033 Health Aff (Millwood). 2014;33(2):200-206. DOI: 10.1377/ hlthaff.2013.1021. PMID: 24493761. 9. Al-Qirim NA. Teledermatology: the case of adoption and diffusion of telemedicine health Waikato in New Zealand. Telemed J E Health. 2003;9(2):167-177. DOI: 10.1089/153056203766437507. PMID: 12855039. 10. Gilling S, Mortz CG, Vestergaard T. Patient satisfaction and expectations regarding mobile teledermoscopy in general prac- tice for diagnosis of non-melanoma skin cancer and malignant melanoma. Acta Derm Venereol. 2020;100(8):adv00117. DOI: 10.2340/00015555-3459. PMID: 32179927. 11. FotoFinder. https://www.fotofinder.de/en/technology/skin-can- cer-diagnostics/handyscope/fotofinder-hub/ . Accessed August 21, 2020. 12. Munck AP, Damsgaard JJ, Hansen DG, Bjerrum L, Sondergaard J. [The APO method--a popular form of quality development in general practice]. Ugeskrift for laeger. 2002;164(46):5390-5393. 13. Orruno E, Gagnon MP, Asua J, Ben Abdeljelil A. Evaluation of teledermatology adoption by health-care professionals using a modified Technology Acceptance Model. J Telemed Telecare. 2011;17(6):303-307. DOI: 10.1258/jtt.2011.101101. PMID: 21844171. 14. Harris PA, Taylor R, Minor BL, et al. The REDCap consor- tium: Building an international community of software platform partners. J Biomed Inform. 2019;95:103208. DOI: 10.1016/j. jbi.2019.103208. PMID: 31078660. 15. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377- 381. DOI: 10.1016/j.jbi.2008.08.010. PMID: 18929686. 16. Whited JD, Hall RP, Foy ME, et al. Patient and clinician satis- faction with a store-and-forward teledermatology consult sys- tem. Telemed J E Health. 2004;10(4):422-431. DOI: 10.1089/ tmj.2004.10.422 PMID: 15689645. 17. Stratton D, Loescher LJ. The acceptance of mobile teledermoscopy by primary care nurse practitioners in the state of Arizona. J Am Assoc Nurse Practitioners. 2016;28(6):287-293. 18. Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly. 1989;13(3):319-340. 19. Tognetti L, Cevenini G, Moscarella E, et al. An integrated clin- ical-dermoscopic risk scoring system for the differentiation between early melanoma and atypical nevi: the iDScore. Journal of the European Academy of Dermatology and Venereology : JEADV. 2018;32(12):2162-2170. DOI: 10.1111/jdv.15106. PMID: 29888421. 20. Tognetti L, Cevenini G, Moscarella E, et al. Validation of an integrated dermoscopic scoring method in an European teleder- moscopy web platform: the iDScore project for early detection of melanoma. J Eur Acad Dermatol Venereol. 2020;34(3):640-647. DOI: 10.1111/jdv.15923. PMID: 31465600. 21. Mounessa JS, Chapman S, Braunberger T, et al. A systematic review of satisfaction with teledermatology. J Telemed Telecare. 2018;24(4):263-270. DOI: 10.1177/1357633X17696587. PMID: 28350281. 22. Kenney AS, Yiannias JA, Raghu TS, David PS, Chang YH, Greig HE. Measures of satisfaction for providers and patients us- ing same day teledermoscopy consultation. Int J Dermatol. 2016;55(7):781-785. DOI: 10.1111/ijd.12892. PMID: 26276548. 23. Janda M, Horsham C, Koh U, et al. Evaluating health- care practitioners’ views on store-and-forward teledermos- copy services for the diagnosis of skin cancer. Digit Health. 2019;5:2055207619828225. DOI: 10.1177/2055207619828225. PMID: 30792879. 24. Chappuis P, Duru G, Marchal O, Girier P, Dalle S, Thomas L. Dermoscopy, a useful tool for general practitioners in melanoma screening: a nationwide survey. Br J Dermatol. 2016;175(4):744- 750. DOI: 10.1111/bjd.14495. PMID: 26914613. 25. Jones OT, Jurascheck LC, Utukuri M, Pannebakker MM, Emery J, Walter FM. Dermoscopy use in UK primary care: a survey of GPs with a special interest in dermatology. J Eur Acad Dermatol Venereol.. 2019;33(9):1706-1712. DOI: 10.1111/jdv.15614. 26. Lim D, Oakley AM, Rademaker M. Better, sooner, more conve- nient: a successful teledermoscopy service. Australas J Dermatol. 2012;53(1):22-25. DOI: 10.1111/j.1440-0960.2011.00836.x. PMID: 22309326.