Dermatology: Practical and Conceptual Original Article | Dermatol Pract Concept. 2023;13(2):e2023105 1 Effect of Histopathological Explanations for Dermoscopic Criteria on Learning Curves in Skin Cancer Training: a Randomized Controlled Trial Niels Kvorning Ternov1, Martin Tolsgaard2,3, Lars Konge2,3, Anders Nymark Christensen4, Sigrid Isabella Pilgaard Kristensen1, Lisbet Rosenkrantz Hölmich1,2, Jonathan Stretch5,6, Richard Anthony Scolyer5,6,7,8, Tine Vestergaard9, Pascale Guitera5,6,10, Annette Hougaard Chakera1,2 1 Department of Plastic Surgery, Copenhagen University Hospital, Herlev and Gentofte, Copenhagen, Denmark 2 Department of Clinical Medicine, University of Copenhagen, Copenhagen, Denmark 3 Copenhagen Academy for Medical Education and Simulation, Rigshospitalet, Copenhagen, Denmark 4 Department of Applied Mathematics and Computer Science, Technical University of Denmark, Kongens Lyngby, Denmark 5 Melanoma Institute Australia, The University of Sydney, Sydney, New South Wales, Australia 6 Faculty of Medicine and Health, The University of Sydney, Sydney, New South Wales, Australia 7 Tissue Pathology and Diagnostic Oncology, Royal Prince Alfred Hospital and New South Wales Health Pathology, Sydney, New South Wales, Australia 8 Charles Perkins Centre, The University of Sydney, Sydney, New South Wales, Australia 9 Department of Dermatology and Allergy Center, Odense University Hospital, Denmark 10 The Sydney Melanoma Diagnosis Centre, Royal Prince Alfred Hospital, Sidney, Australia Key words: education, melanoma, pigmented lesions, dermatopathology, carcinoma Citation: Kvorning Ternov N, Tolsgaard M, Konge L, et al. Effect of Histopathological Explanations for Dermoscopic Criteria on Learning Curves in Skin Cancer Training: A Randomized Controlled Trial. Dermatol Pract Concept. 2023;13(2):e2023105. DOI: https://doi .org/10.5826/dpc.1302a105 Accepted: October 20, 2022; Published: April 2023 Copyright: ©2023 Kvorning Ternov et al. This is an open-access article distributed under the terms of the Creative Commons Attribution-NonCommercial License (BY-NC-4.0), https://creativecommons.org/licenses/by-nc/4.0/, which permits unrestricted noncommercial use, distribution, and reproduction in any medium, provided the original authors and source are credited. Funding: None. Competing Interests: NKT is CEO and co-founder of the start-up MelaTech ApS that developed the educational mobile application used in this trial. RAS has received fees for professional services from F. Hoffmann-La Roche Lt d, Evaxion, Provectus Biopharmaceuticals Australia, Qbiotics, Novartis, Merck Sharp & Dohme, NeraCare, AMGEN Inc., Bristol-Myers Squibb, Myriad Genetics, GlaxoSmithKline. Authorship: All authors have contributed significantly to this publication. Corresponding Author: Niels Kvorning Ternov, Dep. of Plastic Surgery, Herlev and Gentofte University Hospital, Borgmester Ib Juuls Vej 1, 2370 Herlev, Denmark. E-mail: niels.kvorning.ternov@regionh.dk Introduction: Case-based training improves novices pattern recognition and diagnostic accuracy in skin cancer diagnostics. However, it is unclear how pattern recognition is best taught in conjunction with the knowledge needed to justify a diagnosis. Objectives: The aim of this study was to examine whether an explanation of the underlying histo- pathological reason for dermoscopic criteria improves skill acquisition and retention during case-based training in skin cancer diagnostics. ABSTRACT 2 Original Article | Dermatol Pract Concept. 2023;13(2):e2023105 Introduction Skin cancer is the most common malignancy among fair-skinned people worldwide [1,2]. Early detection and treatment of skin cancers reduce patient mortality and the associated socioeconomic costs [3]. In most countries, skin cancer triage is performed by personnel without stan- dardized training in the domain [4,5]. Courses that teach structured checklists for skin cancer diagnostics such as the ABCD (Asymmetry, Borders, Colour, and Diameter) algo- rithm improve novices short-term accuracy, but frequent refresher courses are needed to maintain the skill [6,7]. It is unclear if the training effect is caused by the structured checklists or the simultaneous exposure to many skin lesion images. A Cochrane review recently stated that structured checklists do not improve clinicians accuracy in skin cancer diagnostics [8]. Case-based training improves novices ac- curacies in skin cancer diagnostics significantly more than structured checklist practice [9-11]. Exposure to annotated skin lesion images improves novices pattern recognition, which is the primary diagnostic strategy of experts [12-14]. It is unclear how pattern recognition and the declarative knowledge needed to justify a diagnosis, are best taught in conjunction. Related work from odontology suggests that teaching the underlying biomedical reason for visual crite- ria improves the students ability to recognize and remem- ber the criteria [15,16]. Objectives This study investigated a novel approach towards teaching skin cancer diagnostics through a mobile educational appli- cation. Our primary objective was to examine whether an explanation of the underlying histopathological reason for the dermoscopic criteria used in skin cancer diagnostics af- fects medical students’ learning curves and skill retention. We hypothesized that a deeper biomedical understanding of the dermoscopic criteria would improve the students’ ability to recognize and remember them. Methods In this double-blinded randomized controlled trial (alloca- tion ratio 1:1), we enrolled medical students with no prior experience in skin cancer diagnostics that had previously passed an exam in general histology. The students were in- vited to participate through a Facebook group for Danish medical students. Participants were enrolled through virtual meetings between the 8th and 27th of July 2021. During in- clusion, we helped participants download and get started in the educational application (onboarding process). Par- ticipants were automatically and randomly (simple ran- domization) assigned to the intervention or control group during the onboarding process. The principal investigator (N.K.T.), a student assistant (S.K.), and participants were all blinded towards trial group allocation. Following the on- boarding process, all participants underwent a pre-test and were instructed to diagnose 500 skin lesion cases (including the pre-test cases) over 8 days (training phase), pause for 14 days (washout phase 1), diagnose another 100 cases in 2 days (retention phase), pause for another 7 days (washout phase 2), and finally, complete a retention test (Figure 1). All participants received daily reminders during the training and retention phases, and those that completed the entire study received a certificate of completion. During the training and retention phase, participants had access to written learning modules that described the most common skin lesion diag- noses. The learning modules included a dermoscopy sub- section that differed between the trial groups. Participants in the control group saw a brief description of the dermo- scopic criteria, while the intervention group saw the same description supplemented with a histopathological expla- nation (Figure 2). Neither group were informed about the group-dependent learning module differences. We planned and conducted the study per the principles of the Declara- tion of Helsinki. Participants were informed about the study before participation and gave informed consent. The study was voluntary, held no consequences for the participants, and received a waiver from the Regional Ethics Committee Methods: In this double-blinded randomized controlled trial, medical students underwent eight days of case-based training in skin cancer diagnostics, which included access to written diagnosis modules. The modules dermoscopic subsections differed between the study groups. All participants received a general description of the criteria, but the intervention group additionally received a histopathological explanation. Results: Most participants (78%) passed a reliable test in skin cancer diagnostics, following a mean training time of 217 minutes. Access to histopathological explanations did not affect participants’ learning curves or skill retention. Conclusions: The histopathological explanation did not affect the students, but the overall education- al approach was efficient and scalable. Original Article | Dermatol Pract Concept. 2023;13(2):e2023105 3 Figure 1. Trial flow. Participants performed a pre-test (12 cases) at the beginning and a retention test (25 cases) at the end of the trial. During the training and retention phases, each participant practiced skin lesion diagnostics on 500 (including the pre-test cases) and 100 training cases, re- spectively, while accessing the learning modules of their own accord. We instructed participants to abstain from any training during both washout phases. Figure 2. Educational intervention (mobile application). The educational intervention consisted of an educational mobile application that included quizzes and written learning modules. The red circles within the figure indicate where users “press” the mobile screen to proceed towards the next screen, indicated by the red arrow. The “Quiz feature” presents skin lesions for diagnostics. The small images representing the clinical image (A) and the avatar (B) are buttons that open the clinical image and 3D avatar. When users press “Benign” (C) or “Malignant”, an array of new buttons representing the various benign or malignant differential diagnoses appear. When users press one of the diagnosis buttons (D), they receive immediate feedback. The feedback consists of the chosen diagnosis, the correct diagnosis, and access to learning modules on both the chosen and correct diagnoses (E). Each learn- ing module consists of the following sections: introduction, pathology, clinical presentation, dermoscopy, differential diag- noses, and references. The dermoscopy sections included an overview and subsections describing the primary dermoscopic criteria. Each subsection included a detailed description of the dermoscopic criterium (F). Users from both trial groups received descriptions and annotated images representing the dermoscopic criteria. However, the subsections presented to the intervention group participants also explained the underlying histopathological correlation for the dermoscopic crite- ria. When the learning modules in the application are closed (G), users return to the previous training case feedback page. 4 Original Article | Dermatol Pract Concept. 2023;13(2):e2023105 Participants received immediate feedback on their quiz di- agnoses, including the correct diagnosis and access to the aforementioned learning modules. Time spent reading the learning modules and diagnosing the quiz cases were auto- matically registered throughout the study. Pre- and Retention Test The pre- and retention tests consisted of skin lesion cases from a test item library with validity evidence previously described by our group [4]. The pre-test consisted of 12 randomly sam- pled test items (Generalizability coefficient of 0.7), while the retention test included all 25 test items (Cronbach α of 0.83). A former pass-fail test revealed a pass-fail limit of twelve, ie a score above 12/25 is enough to pass the test [4]. Statistics Data was divided into a training (0-500 cases) and a reten- tion phase (501-600 cases). A mixed-effects logistic regres- sion model with correct or false case-answers as an outcome and a random intercept and slope for each individual was applied to estimate the learning curves on the log-odds scale. We sought the most straightforward description of the partic- ipants learning curves by comparing (likelihood-ratio tests) cubic and linear spline models with four, one, or zero knots. The retention phase data were described using a simple line. Once we had located the optimal statistical equation for de- scribing the training phase, we compared the control and intervention groups training and retention learning curves using likelihood-ratio tests. The control group retention test results were compared to those of the intervention group us- ing the Welch t-test. For exploratory post hoc analyses, we used the participants test scores on the retention test to di- vided them into three equally big performance groups; low- (1st tertile), intermediate- (2nd tertile) and high- (3rd tertile) performance. We compared the learning curves, time spent reading, and time spent diagnosing training cases between the 1st and 3rd tertile using likelihood ratio tests and Welch t-tests. All statistical analyses were performed in R version 4.1.0 (R Foundation for Statistical Computing). Results Eighty-seven medical students were enrolled, and 76 com- pleted the entire trial, see the consort diagram in Figure 3. Retention Test Results on the retention test were equal (t= 0.13, degrees of freedom (df) = 71.3, P = 0.90) for the intervention (mean: 13.8, SD: 3.06) and control (mean: 13.9, SD: 3.35) groups. Fifty-nine (78%) out of the 76 participants passed the re- tention test (>12/25 correct answers). The 76 participants of Region Hovedstaden, Denmark (jr nr. H-20066667). The Danish Health Data Authorities and Data Protection Agency approved access, anonymization, handling, and storage of the skin lesion cases (jr. nr. 21/5103 and 18/53664). We sub- mitted a study protocol on clinicaltrial.gov prior to initiating the study (identifier: NCT05087485). Skin Lesions Library We developed a case library consisting of 2,376 anonymous skin lesions for this study. Each lesion belonged to one of the following seven diagnostic groups: nevus, seborrheic kerato- sis/solar lentigo, dermatofibroma, hemangioma, melanoma, basal cell carcinoma, and squamous cell carcinoma. Each case included a clinical and dermoscopic image of the lesion, the lesion location on a human 3D avatar, a diagnosis, and the patient age and gender. The lesions diagnoses were based on either a histopathological assessment (N = 1,293) or a clinical consensus (N = 1,083), consisting of a joint judgment by 2-3 clinicians. All images were captured by nurses and doctors at the Department of Dermatology and Allergy Cen- tre, Odense University Hospital, in Denmark, from the 1st of September 2010 until the 8th of May 2021. Dermoscopic images were photographed using digital dermoscopes ( Medicam 800 and 1000, Fotofinder Systems GmbH). Written Learning Modules The mobile application’s written content consisted of 38 di- agnosis (eg melanoma) and sub-diagnosis learning modules (eg superficial spreading melanoma). Each sub-diagnosis module included the following subsections: introduction, pathology, clinical presentation, dermoscopy, differential di- agnoses, and references (Figure 2). We created two versions of each dermoscopic sub-section; one described the dermo- scopic criteria with annotated images (control group), and another version that additionally explained the histopatho- logical correlation for each dermoscopic criterion (inter- vention group) (Figure  2). All diagnosis and sub-diagnosis modules were written by the first author (N.T.) and reviewed by content experts in pathology, dermatology, and skin can- cer surgery (co-authors: A.C., P.G., T.V., R.S., L.H., and J.S.). Mobile Application In this study, we employed a mobile application for training skin lesion diagnostics, called Dermloop Learn (Melatech ApS), developed in cooperation with our group (Figure 2). A continuously updated version of the application can be accessed online (https://training.dermloop.io/) or through the app store (“Dermloop Learn”). The application included three functionalities: skin lesion quizzes, written learn- ing modules, and user tracking. Each quiz consisted of ten randomly sampled skin lesion cases from the case library. Original Article | Dermatol Pract Concept. 2023;13(2):e2023105 5 intervention and control groups (training phase: χ2= 0.35, df = 2, P = 0.83, retention phase: χ2=0.94, df = 1, P = 0.33). The learning curves of the intervention and control group participants were also equal within the low- and high-perfor- mance groups (training phase: χ2= 0.80, df = 2, P = 0.67, re- tention phase: χ2=0.15, df = 1, P = 0.70). However, there was a significant difference between the overall (intervention + control) learning curves of the low- versus high-performance groups, (training phase: χ2= 25.71, df = 1, P = <0.01, reten- tion phase: χ2=15.29, df = 1, P < 0.01). The mean time spent training was 217 minutes, 117  minutes diagnosing training cases, and 100 minutes reading the learning modules. There was no difference in time spent diagnosing cases (t= -0.03, df = 51.9, P = 0.98) or reading (t= -0.02, df = 73.9, P = 0.98) between the in- tervention and control participants. The high-performance who completed the retention test were split into low-, intermediate-, and high-performance groups (N = 25, 25, and 26) based on their test results. Intervention and con- trol participants were equally distributed across the low (N = 14/11) and high-performance (N = 14/12) groups. Learning Curves and Time Spent Training An almost straight line with a single knot, ie breaking point, at 100 cases provided a significantly better data-fit for the training phase than a straight line without knots (χ2= 125.0, df = 2, P = <0.01). There was no added benefit from adding three additional knots at 200, 300, and 400 training cases (χ2= 7.5, df = 6, P = 0.28), or performing a cubic transforma- tion (χ2= 4.3, df = 4, P = 0.37) (Figure 4). There were no significant learning curve differences in the training or retention phase, when comparing the ASSESSED FOR ELIGIBILITY (N=87) EXCLUDED (N= 0) RANDOMIZED (N= 87) FALLOUT (N=2) FALLOUT (N=3) TRAINING PHASE (N=42) FALLOUT (N= 1) RETENTION PHASE (N=41) FALLOUT (N= 1) RETENTION TEST (N= 40) PART OF THE INTENDED PROGRAM (N=42) PARTICIPANTS WITH DATA FOR ANALYSIS OF: -LEARNING CURVES (N=42) -T-TEST RETENTION TEST (N=40) -TIME USED READING AND PRACTICING (N= 40) PART OF THE INTENDED PROGRAM (N= 40) PARTICIPANTS WITH DATA FOR ANALYSIS OF: -LEARNING CURVES (N=40) -T-TEST RETENTION TEST (N= 36) -TIME USED READING AND PRACTICING (N= 36) RETENTION TEST (N= 36) FALLOUT (N= 0) RETENTION PHASE (N= 36) FALLOUT (N=4) TRAINING PHASE (N= 40) INTERVENTION GROUP (N=44) (DESCRIPTION OF CRITERIA + HISTOPATHOLOGICAL EXPLANATIONS) -MALE (N=9) -FEMALE (N= 35) CONTROL GROUP (N= 43) (DESCRIPTION OF CRITERIA) -MALE (N= 8) -FEMALE (N= 35) Figure 3. Consort diagram. 6 Original Article | Dermatol Pract Concept. 2023;13(2):e2023105 (78%), irrespective of trial groups, passed a reliable test in skin cancer diagnostics, following a mean training time of 217 minutes. Access to explanations of the histopathological correlation for dermoscopic criteria did not affect partici- pants learning curves or skill retention. The rapid diagnostic improvement observed in this study resonates with former studies on case-based pattern recog- nition training for skin cancer diagnostics [9,10]. This pro- posed model for teaching complex visual diagnostics could potentially democratize skin cancer diagnostics. Primary care providers, nurses, and medical students can be educated in mass, improving access to high-quality skin cancer triage group spent the same amount of time reading (t= 0.13, df = 30.5, P = 0.90) and significantly more time diagnosing the training cases (t=-3.8, df = 43.8, P = <0.001) compared to the low-performance group (Table 1). Participants spent 40% of their training time on the first 100 cases (mean: 93.8 min) and the remaining time on the last 500 cases (mean: 146.8 min). Conclusions This study explored a novel approach towards digital training in skin cancer diagnostics. The vast majority of participants SIMPLE LINE ES TI M A TE D P R O B A B IL IT Y O F C O R R EC T A N SW ER NUMBER OF COMPLETED CASES 0 0. 2 0. 3 0. 4 0. 5 0. 6 A 100 200 300 400 500 CONTROL INTERVENTION SPLINE WITH 4 KNOTS KNOTS 1 C KNOTS 2 KNOTS 3 KNOTS 4 CONTROL INTERVENTION 0 0. 2 0. 3 0. 4 0. 5 0. 6 100 200 300 400 500 ES TI M A TE D P R O B A B IL IT Y O F C O R R EC T A N SW ER NUMBER OF COMPLETED CASES SPLINE WITH 1 KNOT KNOT 1 0 0. 2 0. 3 0. 4 0. 5 0. 6 B 100 200 300 400 500 CONTROL INTERVENTION ES TI M A TE D P R O B A B IL IT Y O F C O R R EC T A N SW ER NUMBER OF COMPLETED CASES CUBIC SPLINE ES TI M A TE D P R O B A B IL IT Y O F C O R R EC T A N SW ER NUMBER OF COMPLETED CASES 0 0. 2 0. 3 0. 4 0. 5 0. 6 D 100 200 300 400 500 CONTROL INTERVENTION Figure 4. Learning curve models for the training phase. The figure depicts the various statistical equations applied to the data. Red dashed, and solid black lines represent the intervention and control groups, respectively. (B) An almost straight line with one knot provided a simple yet reliable fit for the data. (C,D) Increasing the complexity of the model by introducing additional knots (C) or a cubic function (D) did not provide any additional value com- pared to model B. Table 1. Time spent training within the mobile application. Activity Time spent reading (min) Time spent diagnosing training cases (min) Group Intervention Control 1st tertile 3rd tertile Intervention Control 1st tertile 3rd tertile Mean 100.7 100.1 94.7 90.4 117.1 116.6 80.9 124.5 SD 121.9 105.7 150.1 56.6 49.2 93 32.2 48.4 min = minutes; SD = standard deviation. Time spent reading learning modules and diagnosing training cases among the participants within both study groups (intervention, control) and performance groups (1st and 3rd tertile). The 1st and 3rd tertile groups consist of the 33% participants with the lowest and highest per- formance on the retention test. Original Article | Dermatol Pract Concept. 2023;13(2):e2023105 7 postpone the deceleration of learning curves, further dis- cussed below. During post hoc analyses, we found that high-performers were more accurate than the low-performers through- out the study. High-performers were more accurate than low-performers already at the beginning of the trial, despite all participants being supposedly equally inexperienced. These findings suggest that some participants were dis- honest about their expertise during inclusion or possessed a superior innate diagnostic accuracy. Regardless of the difference in diagnostic accuracy, both groups maintained parallel learning curves throughout the trial. In theory, the accuracy of low-performers should have increased faster than it did for the high-performers. It is significantly more challenging to increase one’s diagnostic accuracy when it is high compared to low [21]. High-performers likely had a more efficient and intentional learning strategy through- out the trial. This hypothesis is supported by the fact that high-performers spent 35% more time diagnosing training and reducing some of the current inequality in melanoma mortality [17,18]. Improved competencies could also pave the way for clinical implementation diagnostic artificial in- telligence, providing a human safeguard against the algo- rithms erroneous predictions and inherent biases [19]. According to the “deliberate practice framework” for teaching diagnostics, novice learners need assistance from a domain expert, as they lack the competencies needed to identify and address their knowledge gaps [20]. A synthetic algorithm-driven domain expert could, in theory, be devel- oped and integrated as a digital mentor within learning inter- ventions such as the one used in this study. A digital mentor could be taught how to identify a students learning pattern, weaknesses, and strengths based on prior students training data. The identified patterns could then be used to select and present the instructional material (cases, modules) most likely to increase the student competencies at any given time. Such individualized approaches could potentially reduce the observed difference between low- and high-performers and LEARNING CURVES TRAINING PHASE WASHOUT PERIOD (14 DAYS) NUMBER OF COMPLETED CASES ES TI M A TE D P R O B A B IL IY O F C O R R EC T A N SW ER S 0 0,2 0,3 0,4 0,5 0,6 0,25 0,35 0,45 0,55 100 200 300 400 500 600 TOTAL INTERVENTION TOTAL CONTROL LOW INTERVENTION LOW CONTROL HIGH INTERVENTION HIGH CONTROL RETENTION PHASE Figure 5. Learning curves during the training and retention phases. The diagram depicts the learning curves for the entire group (intervention: dark blue, control: red), the high-performance group (inter- vention: light blue, control: orange), and the low-performance group (intervention: green, control: purple). 8 Original Article | Dermatol Pract Concept. 2023;13(2):e2023105 explanations for dermoscopic criteria did not affect the stu- dents knowledge acquisition and retention. Acknowledgments: We wish to extend a special thanks to Assistant Professor Morten Hannemose and statistician Søren Grimstrup for their statistical assistance. References 1. Lomas A, Leonardi-Bee J, Bath-Hextall F. A systematic re- view of worldwide incidence of nonmelanoma skin cancer. Br J Dermatol. 2012;166(5):1069-1080. DOI: 10.1111/j.1365- 2133.2012.10830.x. PMID: 22251204. 2. Whiteman DC, Green AC, Olsen CM. The Growing Burden of Invasive Melanoma: Projections of Incidence Rates and Num- bers of New Cases in Six Susceptible Populations through 2031. J Invest Dermatol. 2016;136(6):1161-1171. DOI: 10.1016/j. jid.2016.01.035. PMID: 26902923. 3. Petrie T, Samatham R, Witkowski AM, Esteva A, Leachman SA. Melanoma Early Detection: Big Data, Bigger Picture. J Invest Dermatol. 2019;139(1):25-30. DOI: 10.1016/j.jid.2018.06.187. PMID: 30482597. PMCID: PMC6685706. 4. Ternov NK, Vestergaard T, Hölmich LR, et al. Reliable test of clinicians’ mastery in skin cancer diagnostics. Arch Dermatol Res. 2021;313(4):235-243. DOI: 10.1007/s00403-020-02097-8. PMID: 32596742. 5. Tschandl P, Codella N, Akay BN, et al. Comparison of the accu- racy of human readers versus machine-learning algorithms for pigmented skin lesion classification: an open, web-based, inter- national, diagnostic study. Lancet Oncol. 2019;20(7):938-947. DOI: 10.1016/S1470-2045(19)30333-X. PMID: 31201137. PMCID: PMC8237239. 6. Friedman RJ, Rigel DS, Kopf AW. Early detection of ma- lignant melanoma: the role of physician examination and self-examination of the skin. CA Cancer J Clin. 1985;35(3):130- 151. DOI: 10.3322/canjclin.35.3.130. PMID: 3921200. 7. HHarkemanne E, Baeck M, Tromme I. Training general prac- titioners in melanoma diagnosis: a scoping review of the literature. BMJ Open. 2021;11(3):e043926. DOI: 10.1136/bm- jopen-2020-043926. PMID: 33757946. PMCID: PMC7993310. 8. Dinnes J, Deeks JJ, Chuchu N, et al. Dermoscopy, with and without visual inspection, for diagnosing melanoma in adults. Cochrane Database Syst Rev. 2018;12(12):CD011902. DOI: 10.1002/14651858.CD011902.pub2. PMID: 30521682. PM- CID: PMC6517096. 9. Speelman C, Martin K, Flower S, Simpson T. Skill acquisition in skin cancer detection. Percept Mot Skills. 2010;110(1):277-297. DOI: 10.2466/PMS.110.1.277-297. PMID: 20391892. 10. Lacy FA, Coman GC, Holliday AC, Kolodney MS. Assessment of Smartphone Application for Teaching Intuitive Visual Diagno- sis of Melanoma. JAMA Dermatol. 2018;154(6):730-731. DOI: 10.1001/jamadermatol.2018.1525. PMID: 29799954. PMCID: PMC6145647. 11. Monteiro SM, Norman G. Diagnostic reasoning: where we’ve been, where we’re going. Teach Learn Med. 2013;25 Suppl 1:S26-S32. DOI: 10.1080/10401334.2013.842911. PMID: 24246103. 12. Norman GR, Rosenthal D, Brooks LR, Allen SW, Muzzin LJ. The development of expertise in dermatology. Arch Dermatol. 1989;125(8):1063-1068. PMID: 2757402. cases than low-performers. Learning curves decelerated fol- lowing the first 100 training cases, possibly due to a faltering engagement, poor clinical contextualization, or a plateau in the trainees ability to recognize and mitigate their diagnos- tic errors. On average, each participant spent almost the same amount of time (40% of total time) diagnosing and reviewing the first 100 training cases as they did on the re- maining 500 cases. This indicates that they shifted from a slow but actively engaged learning strategy towards a faster trial-and-error strategy that yielded a significantly worse outcome [22,23]. Educational engagement is strongly asso- ciated with internal and external motivation [24]. The aca- demic curiosity, which was the sole motivation of the trial participants, may have faltered following 100 training cases, resulting in the learning curve deceleration. Adopting new clinical skills and knowledge becomes easier if the trainee has former clinical experience within the given domain, pos- sibly because it is easier to conceptualize the clinical rele- vance of the skills being taught [22,25]. Participants in this trial had no former clinical experience, potentially impeding their ability to contextualize and acquire complex skills in visual diagnostics. Finally, the deceleration may have been caused by a plateau in the participants ability to analyze and address their knowledge gaps, ie failure of metacognition [26]. Additional studies, such as think-aloud verbal protocol studies, are needed to further our understanding of the low- versus high-performers learning strategies [27,28]. During think-aloud verbal protocol studies, participants are asked to perform a task and “think aloud” while being recorded. The thought processes are later deconstructed by a trained observer and converted into standardized code blocks prior to statistical analyses. Our study has several limitations. First, we did not collect data on which subsections of the learning modules partici- pants had read during the trial, limiting our ability to perform sub-analyses on the effect of the histopathological explana- tions. Secondly, the retention test is a proxy rather than an ac- curate measure of clinical skills. Finally, it is unclear whether our results can be reproduced among clinicians. Thirdly, do- main experience generally increases a student ability to con- textualize knowledge, and we theorize that our results would have been more pronounced among clinicians compared to medical students. Finally, several factors such as the partici- pants general interest, motivation, and social or cultural sta- tus may have introduced biases in our results. This study describes an efficient and scalable approach towards teaching pattern recognition in skin cancer diagnos- tics. We investigated the learning curves of medical students and found a deceleration following approximately 100 train- ing cases and a continuous and stable superiority among the upper compared to lower tertile. Access to histopathological Original Article | Dermatol Pract Concept. 2023;13(2):e2023105 9 13. Norman GR, Coblentz CL, Brooks LR, Babcook CJ. Expertise in visual diagnosis: a review of the literature. Acad Med. 1992;67(10 Suppl):S78-S83. DOI: 10.1097/00001888-199210000-00045. PMID: 1388563. 14. Yoon JS, Boutis K, Pecaric MR, Fefferman NR, Ericsson KA, Pusic MV. A think-aloud study to inform the design of radio- graph interpretation practice. Adv Health Sci Educ Theory Pract. 2020;25(4):877-903. DOI: 10.1007/s10459-020-09963-0. PMID: 32140874. PMCID: PMC7471179. 15. Baghdady MT, Pharoah MJ, Regehr G, Lam EW, Woods NN. The role of basic sciences in diagnostic oral radiology. J Dent Educ. 2009;73(10):1187-1193. PMID: 19805783. 16. Baghdady MT, Carnahan H, Lam EW, Woods NN. Integration of basic sciences and clinical sciences in oral radiology education for dental students. J Dent Educ. 2013;77(6):757-763. PMID: 23740912. 17. Harvey VM, Patel H, Sandhu S, Wallington SF, Hinds G. Social determinants of racial and ethnic disparities in cutaneous mel- anoma outcomes. Cancer Control. 2014;21(4):343-349. DOI: 10.1177/107327481402100411. PMID: 25310216. PMCID: PMC4505912. 18. Ibfelt EH, Steding-Jessen M, Dalton SO, Lundstrøm SL, Osler M, Hölmich LR. Influence of socioeconomic factors and re- gion of residence on cancer stage of malignant melanoma: a Danish nationwide population-based study. Clin Epidemiol. 2018;10:799-807. DOI: 10.2147/CLEP.S160357. PMID: 30022857. PMCID: PMC6044336. 19. Tschandl P, Rinner C, Apalla Z, et al. Human-computer collab- oration for skin cancer recognition. Nat Med. 2020;26(8):1229- 1234. DOI: 10.1038/s41591-020-0942-0. PMID: 32572267. 20. Ericsson KA. Acquisition and maintenance of medical exper- tise: a perspective from the expert-performance approach with deliberate practice. Acad Med. 2015;90(11):1471-1486. DOI: 10.1097/ACM.0000000000000939. PMID: 26375267. 21. Loerch AG. Learning Curves. In: Gass, S.I., Fu, M.C. (eds) Encyclo- pedia of Operations Research and Management Science. Springer, Boston, MA. 2013 DOI: 10.1007/978-1-4419-1153-7_526 22. Schwartz D, Bransford J, Sears D. Efficiency and innovation in transfer. In J. Mestre (Ed.), Transfer of learning from a modern multidisciplinary perspective. Greenwich, CT: Information Age Publishing. 2005:1-51. 23. Davis JM, Leelawong K, Belynne K, Ret al. Intelligent User Interface Design for Teachable Agent Systems. International Conference on Intelligent User Interfaces. Miami, Florida, The Association for Computing Machinery. 2003:26-33. 24. Cook DA, Artino AR Jr. Motivation to learn: an overview of contemporary theories. Med Educ. 2016;50(10):997-1014. DOI: 10.1111/medu.13074. PMID: 27628718. PMCID: PMC5113774. 25. Andreasen LA, Tabor A, Nørgaard LN, Ret al. Multicenter randomized trial exploring effects of simulation-based ul- trasound training on obstetricians’ diagnostic accuracy: value for experienced operators. Ultrasound Obstet Gyne- col. 2020;55(4):523-529. DOI: 10.1002/uog.20362. PMID: 31152560. 26. Flavell JH (1979). Metacognition and cognitive monitoring: A new area of cognitive–developmental inquiry. American Psychologist. 1979;34(10): 906–911. DOI: 10.1037/0003-066X.34.10.906 27. Ericsson KA, Simon HA. Protocol Analysis: Verbal Reports as Data. 1993. DOI:10.7551/mitpress/5657.001.0001. 28. Fox MC, Ericsson KA, Best R. Do procedures for verbal re- porting of thinking have to be reactive? A meta-analysis and recommendations for best reporting methods. Psychol Bull. 10 Original Article | Dermatol Pract Concept. 2023;13(2):e2023105 2011;137(2):316-344. DOI: 10.1037/a0021663. PMID: 21090887.