DERMATOLOGY PRACTICAL & CONCEPTUAL www.derm101.com Research | Dermatol Pract Concept 2012;2(2):2 3 A study assessing the feasibility and diagnostic accuracy of real-time teledermatopathology Elisabeth Riedl, M.D.1, 3, Masoud Asgari, M.D.2, Diana Alvarez, M.D.2, Irina Margaritescu, M.D.2, Geoffrey J. Gottlieb, M.D.2 1 Department of Dermatology, Medical University of Vienna, Vienna, Austria 2 Ackerman Academy of Dermatopathology, New York, NY, USA 3 Department of Dermatology, Mount Sinai School of Medicine, New York, NY, USA Citation: Riedl E, Asgari M, Alvarez D, Margaritescu I, Gottlieb GJ. A study assessing the feasibility and diagnostic accuracy of real-time teledermatopathology. Dermatol Pract Conc. 2012;2(2):2. http://dx.doi.org/10.5826/dpc.0202a02. Received: January 3, 2012; Accepted: February 25, 2012; Published: April 30, 2012 Copyright: ©2012 Riedl et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Funding: This study was supported by the OELZELT Foundation of the Austrian Society of Sciences. Competing interests: The authors have no conflicts of interest to disclose. All authors have contributed significantly to this publication. Corresponding author: Elisabeth Riedl, M.D., Department of Dermatology, Mount Sinai School of Medicine, One Gustave L. Levy Place, New York, NY 10029. Email: elisried@gmail.com. Introduction Dermatopathology is generally regarded as the gold stan- dard for identification with specificity of those skin diseases that cannot be diagnosed on clinical grounds alone. In many instances those same skin diseases are diagnostically vexing when studied by conventional microscopy, and identifica- tion of them often requires expert consultation not readily available. This problem is most prominent in the diagnosis of melanocytic skin lesions. Dermatopathologic diagnosis of melanoma represents a special circumstance because it is not based on a single criterion, but on a constellation of criteria that are not applied consistently by individual dermatopa- thologists [1-3]. That being so, a true gold standard is lack- Dermatopathology represents the gold standard for the diagnosis of skin diseases and neoplasms that cannot be diagnosed on clinical grounds alone. The aim of this study was to test the feasibility and to assess the accuracy of an Internet-based real-time (live) teledermatopathology consultation. Twenty teaching cases and 10 randomly selected routine cases were presented to four expert derma- topathologists, first by real-time teledermatopathology and, subsequently, in a blinded fashion, using light microscopy. Throughout the study the overall diagnostic accuracy did not differ for the two methods. However, the mean level of confidence and the mean observation times differed significantly between real-time teledermatopathology and light microscopy (92.6±0.24% versus 99.5±0.02%, and 96.31±11.55 sec versus 25.47±3.85 sec, respectively). Assessment of routine cases did not produce significant diagnostic differences between the two methods. These results prove that real-time teleder- matopathology offers an affordable and technically simple technology that lends itself to training as well as to diagnosis of lesions from routine practice by experts situated at remote sites. ABSTRACT 4 Research | Dermatol Pract Concept 2012;2(2):2 ing and the availability of experts gains increasing impor- tance to guarantee a specific diagnosis and the best available treatment according to the diagnosis for the patient. The introduction of teledermatopathology into the rou- tine practice of dermatopathology offers several opportuni- ties for monitoring and improving the quality of diagnosis of “difficult cases” not only in the realm of melanocytic neo- plasms, but in every area of pathology in general [4,5]. The broad application of teledermatology in dermatopathology has been hampered by several obstacles, among them techni- cal obstacles, that have prevented the use of it routinely [6]. Methods of teledermatopathology include electronic trans- mission of still images [7], distant control of a motorized microscope, real-time transmission of digital images from the microscope (videoconferencing), and the assessment of digital images with a high resolution in combination with software that enables the user to load only the desired part of the huge image files (“virtual microscopy”) [8]. Although “virtual microscopy“ represents the latest development in telepathology and is an attractive option for the diagnosis of selected cases, it is time consuming and technically chal- lenging [9]. New Internet-based software, like Skype™, now permits establishment of real-time teledermatopathology, a dynamic method that offers an affordable and technically simple alternative to that what was employed before (Ref: http://www.HL7.com.au/Skype-Video-Conferencing.htm). In this study we tested the feasibility and accuracy of such a method. This objective was obtained in two phases, the first one designed to prove the principle of real-time teledermatopathology and the second one to set up a field experiment to verify its feasibility for routine histopatho- logic evaluation of skin lesions. Materials and methods 1. Study cases The Ethics Committee of the Medical University of Vienna approved this study. For the initial phase of the study, 20 specimens of skin lesions with clear-cut diagnoses were selected and designated as “teaching cases” after they had been reviewed and had been diagnosed without any ambigu- ity by at least two dermatopathologists at the Department of Dermatology, Medical University of Vienna, Vienna, Austria. These “teaching cases” included benign and malignant skin lesions and biopsies of inflammatory skin diseases (Table 1). For the second phase of the study, a total number of 10 cases was randomly selected during routine consultation over a period of two weeks and designated as “routine cases.” For each set of cases, i.e., teaching cases and routine cases, a Skype™ conference was initiated with the remote specialist at the Ackerman Academy of Dermatopathology, New York, NY, USA, and cases were presented in real-time to each of the four participating dermatopathologists who made the diag- nosis of the cases “on the screen“ over the Internet. Upon request, information regarding medical history, clinical set- ting and clinical diagnosis was provided. The diagnosis, dif- ferential diagnoses, level of confidence in the diagnosis, and the time needed to make a diagnosis, were recorded. After two weeks, the actual slides were reviewed by the same der- matopathologists in blinded fashion using conventional light microscopy. The same parameters that had been recorded during the teledermatopathology session were noted. The quality of diagnosis rendered remotely compared to the diag- nosis “under the microscope“ was subjected to statistical analysis. 2. Technical equipment Skype™ is an Internet-based communication software that offers free Internet calls using a headset or free video calls over a web camera. Teledermatopathology sessions were initiated by sending a request for teleconsultation to the remote study site. During the session, a corporate analysis of sections by video conferencing took place in real-time. The system consisted of a remotely controlled microscope (Olympus microscope BX41-TF5; Olympus, Tokyo, Japan) attached to a digital video camera equipped with live video stream transmitted at 800 x 600 pixels at 30 frames per sec- ond. An Internet-connected personal computer with a Win- dow XP operating system (Microsoft, Redmond, VA) was used for the study. LAN connected the two study sites at a maximum transmission rate via Internet of 54 Mbit/sec. This system allowed remote operation of all the movable parts of the light microscope; the video signal was shared between the client at the center where the session was initiated and the expert center and was displayed on a viewing screen. In addition, an audio connection was established via Skype™. 3. Outcome, quality measurements, and statistical analysis The observer’s diagnosis “on the screen” was contrasted with the diagnosis “under the microscope” (looking at the same section of tissue in a blinded fashion). Each time the observer was asked to provide the following information: a specific diagnosis, the level of confidence in that diagno- sis, and one or more differential diagnoses. A comparison between the proportion of correct specific diagnoses by each mode of examination, the plausibility of the specific and dif- ferential diagnoses and the level of confidence in the specific diagnosis was performed. With regard to the specific diagno- sis, the agreement between both modalities was calculated. In addition, the time needed for each case by teleconsultation was recorded and compared with the time needed for the diagnosis of the sections “under the microscope.” Each par- Research | Dermatol Pract Concept 2012;2(2):2 5 ticipant was asked to assess the quality of transmission and of the slides, respectively. Finally, the request of the observer for additional clinical information was recorded. Results Characterization of study cases for study phases 1 and 2 According to the study protocol, cases differed between the two parts of the study in regard to diagnostic difficulty, types of diseases and unambiguity of diagnosis. Since all specimens came from the same institution, the quality of the slides was comparable throughout the study. Accordingly, study partici- pants rated the technical quality of the slides as “good” or “excellent” (data not shown). Moreover, all specimens were prepared from punch biopsies, biopsies or excisions. While the “teaching cases” included stereotypic presentations of common skin diseases (Table 1), “routine cases” that were randomly collected during a two-week period also included cases without a clear-cut diagnosis (Table 1). Specifically, case 3 was diagnosed as pityriasis lichenoides chronica, but a drug eruption or skin lesion of lupus erythematosus could not be ruled out by the dermatopathologist on site. Similarly, case 5 came with the provisional diagnosis “pityriasis rosea, rule out eczema or psoriasis.” Finally, for case 10 no distinc- tion between a primary melanoma and a melanoma metasta- sis was provided during initial diagnosis. All study participants were experienced dermatopa- thologists. Two were board-certified dermatologists and two TABLE 1. Histopathologic diagnoses of slides used for teledermatopathology and microscopy including 20 “teaching cases” (left column) and 10 “routine cases” (right column) Part a “Teaching Cases” Part b “Routine Cases” 1 Granuloma annulare 1 Desmoplastic melanoma 2 Reed’s nevus 2 Basal cell carcinoma 3 Invasive melanoma 3 Pityriasis lichenoides chronica* 4 Scabies 4 Irritated seborrheic keratosis 5 Squamous cell carcinoma 5 Pityriasis rosea* 6 Isthmus catagen cyst 6 Basal cell carcinoma 7 Basal cell carcinoma (BCC) 7 Eczema/dermatitis 8 Syringoma 8 Lichen sclerosus et atrophicans 9 Xanthogranuloma 9 Neurofibroma 10 Melanoma in situ 10 Nodular melanoma* 11 Blue nevus 12 Chronic pigmented purpura 13 Hidrocystoma 14 Psoriasis 15 Dermatofibroma 16 Herpes simplex 17 Leukocytoclastic vasculitis 18 Lichen planus 19 Congenital melanocytic nevus 20 Keratoacanthoma * Diagnosis of these cases included one ore more differential diagnosis as discussed in the text. 6 Research | Dermatol Pract Concept 2012;2(2):2 board-certified pathologists. Furthermore, there was some heterogeneity in regard to duration of professional experi- ence and places of dermatopathology training. However all participants shared at least a one-year period of dermatopa- thology training at the same institution. Assessment of “teaching cases” proves the principle that real-time teledermatopathology compares to conventional light microscopy in regard to diagnostic accuracy The first phase of the study was set up as a proof of prin- ciple to test the method of real-time teledermatopathology. Remote “teleconsultation” diagnoses of 20 teaching cases (Table 1) were compared with diagnoses when assessing the same sections directly “under the microscope.” Overall diag- nostic accuracy, level of confidence and time needed to come to a diagnosis were evaluated. Figure 1a illustrates that the overall diagnostic accuracy did not differ for the two meth- ods. However, the overall mean level of confidence differed significantly between teleconsultation and direct slide assess- ment by light microscopy (92.6±0.24% versus 99.5±0.02%, p=0.008). As can be seen from Figure 1b, teledermatopathol- ogy was also associated with a higher degree of interobserver variability in terms of confidence in the diagnosis. In line with this, observers relied more on additional clinical infor- mation (i.e., biopsy site, number of lesions, age of patient) when making the “on screen” diagnosis (data not shown). Finally, as shown in Figure 1c, the time that was needed to come to a diagnosis was significantly longer for telecon- sultation sessions (96.31±11.55 sec versus 25.47±3.85 sec, p<0.001). Real-time teledermatopathology and direct slide assessment by light microscope of routine cases yield a high degree of interobserver agreement Randomly selected routine specimens were used to compare the accuracy of teleconsultation with direct slide assessment by light microscopy. This field experiment was designed to mirror the actual situation of routine dermatopathology consultation. Of the 10 randomly selected cases, three speci- mens had been signed out originally without a final diagnosis (Table 1). According to the “real-life” setting, rather than giv- ing the number of correct diagnoses, interobserver agreement between teleconsultation and direct slide evaluation and the time needed to come to a diagnosis were assessed. Figure 2a shows that the proportion of cases in which all observers agreed did not differ significantly between the two methods: Observers agreed in six out of 10 cases with the “on screen” Figure 1. Proof of principle of live teledermatopathology evaluating diagnostic accuracy, level of confidence and time to diagnosis during as- sessment of “teaching cases.” a. Bars represent the proportion of correct (corr) diagnoses during “online” (open bars) and “slide” (closed bars) assessment under the microscope by individual observers (A, B, C, D). b. Bars represent the percentages of cases that yielded a confidence level of 100% by individual observers (A, B, C, D) during online” (open bars) and “slide” (closed bars) evaluation. c. Bars represent time for diagnosis (sec) needed by individual observers (A, B, C, D) during teleconsultation (left panel) and direct slide assessment (right panel). P<0.001 between the two groups. A B C Research | Dermatol Pract Concept 2012;2(2):2 7 diagnosis compared to seven out of 10 cases of “under the microscope” diagnosis. Comparable to the results obtained during phase 1, Figure 2b shows that the duration to come to a diagnosis was significantly longer for the method of real-time teledermatopathology compared to direct slide evaluation by light microscopy (112.05±19.95 sec versus 35.43±7.47 sec, p<0.001). Moreover, the time for diagnosis differed signifi- cantly between study observers (p=0.027, Figure 2b). Further analysis revealed that the time difference between teleconsul- tation and direct slide assessment is observer dependent and statistically significant (p= 0.001). This data show that tele- consultation compares to direct slide assessment under the microscope in terms of diagnostic accuracy. Discussion Real-time teledermatopathology represents a novel technique allowing for remote interactive teaching, expert consultation of difficult cases and second opinion consultation in sites where availability of pathologists is limited [10]. The live teledermatopathology sessions include the interactive control of a microscope by a remote presenter and a viewing patholo- gist. Slides can be viewed in their entirety at different magni- Figure 2. Comparison of level of agreement and time needed for diagnosis in the evaluation of “routine cases.” a. Bars represent percentage of cases with perfect agreement among observers dur- ing teleconsultation (left) and direct slide assessment (right). b. Bars represent time needed for diagnosis (sec) by individual observers (A, B, C, D) during teleconsultation (left panel) and direct slide assess- ment (right panel). fication levels, and the images displayed on the video screen represent the actual slide. Furthermore, the viewing patholo- gist can direct the remote operator to areas of specific interest within the sample. Prerequisites for the successful implemen- tation of such a method into daily teaching and dermatopa- thology consultation practice are high diagnostic accuracy and precision, cost and time effectiveness, and practicability. The results of this preliminary study prove that real-time (live) teledermatopathology offers an affordable and simple technology that lends itself to training as well as to diagnosis of difficult lesions by experts situated at remote sites. During the first study phase, the feasibility of telederma- topathology as a diagnostic tool was assessed. Using teaching cases with common, clear-cut diagnoses we could demon- strate that the diagnostic accuracy was comparable between the two methods. This is the first and major requirement when searching for a method that could expand or, in some instances, replace direct slide assessment by light microscopy. Furthermore, we found that although the confidence in the final diagnosis revealed some interobserver differences for the method of teleconsultation, the overall confidence levels did not differ for the two methods. Another critical factor when it comes to practicability of a method is how time consuming it is likely to be. In this study we observed that in both settings, namely, evaluation of “teaching cases” and assessment of “routine cases” study participants needed significantly longer to reach a diagnosis when using teledermatopathology. Still, on average it took less than two minutes to make a diagnosis “on screen.” The impressively short time for the “under the microscope” diag- nosis that ranged between 25 and 35 seconds underlines the study participants’ high level of expertise. However, due to the relatively short interval between the teledermatopathol- ogy consultation and the direct microscopic evaluation, we cannot rule out that participants remembered some cases during the second evaluation and that this bias contributed to the unusually short time needed for “under the micro- scope” diagnosis. The fourth parameter that critically influences the practi- cability of a method of teledermatopathology is its cost effec- tiveness. We started out with the aim of finding a setup that includes a technically superior method that was at the same time affordable even for small-sized dermatopathology prac- tices and hospital departments. The method of live telederma- topathology that we chose actually fulfills these requirements. The total cost was estimated at approximately €20,000, which included a camera-equipped light microscope, a per- sonal computer (PC) and the necessary image software. The free Internet communication software that was used during the study allowed top quality sound and image communica- tion during teleconsultation. Of course, a fast Internet con- nection is a prerequisite for this setup. The main advantage of A B 8 Research | Dermatol Pract Concept 2012;2(2):2 this method over the more sophisticated method of “virtual microscopy” that uses digital image files that have to be gen- erated and then uploaded on the computer or on a server [8] are the comparative low costs and time requirements asso- ciated with real-time teledermatopathology. A limitation is the dependency on the operator at the presenting site. On the other hand, this presumed limitation is advantageous if the presenting physician wants to consult with the remote expert. “Virtual microscopy” allows assessing the uploaded specimen directly without being dependent on direct interac- tion with the remote site presenting the slide. Additionally, this method allows storage of image files for future evalua- tion. However, in regard to its practical use for remote expert consultation, one has to bear in mind that this method is cost intensive especially for the site that seeks expert consultation. This might be one of the limiting factors for its integration into daily practice, especially in rural areas. The validity of real-time telepathology has been tested in a recent study in China [11]. In that study four patholo- gists evaluated 600 specimens from 16 organ systems first by telepathology and subsequently by light microscopy. Comparable to our results, the investigators found a high level of diagnostic agreement between both methods. In line with our results, slide review by teleconsultation took three to four times longer compared to direct assessment under the microscope. In the case of virtual microscopy several studies have shown its value in regard to feasibility and diagnostic accu- racy [12,13]. Our study is the first to evaluate the use of real-time teledermatopathology for both teaching purposes and remote expert diagnosis. From our data we conclude that live teledermatopa- thology is a suitable method for histopathologic diagnosis of skin diseases. The differences seen in the time needed to make a diagnosis “on screen” was likely due to the variable degrees of experience in this method among participants. Future studies will need to evaluate the effect of adequate training in live teledermatopathology and to assess its true potential in everyday dermatopathology practice. Acknowledgments This study was supported by a grant from the OELZELT Foundation of the Austrian Academy of Sciences. The authors thank Dr. Harald Kittler for expert statistical analy- sis of study data. References 1. Ackerman AB. Discordance among expert pathologists in diagno- sis of melanocytic neoplasms. Hum Pathol. 1996;27(11):1115-6. 2. Brochez L, Verhaeghe E, Grosshans E, et al. Inter-observer varia- tion in the histopathological diagnosis of clinically suspicious pigmented skin lesions. J Pathol. 2002;196(4):459-66. 3. Urso C, Rongioletti F, Innocenzi D, et al. Interobserver reproduc- ibility of histological features in cutaneous malignant melanoma. J Clin Pathol. 2005;58(11):1194-8. 4. Leinweber B, Massone C, Kodama K, et al. Teledermatopathol- ogy: a controlled study about diagnostic validity and technical requirements for digital transmission. Am J Dermatopathol. 2006;28(5):413-6. 5. Deagustin D, Sanmartin J, Varela-Centelles P, Vidal S. Seoane J. Technological bases for teledermatopathology: state of the art. Semin Cutan Med Surg. 2008;27(1):25-31. 6. Furness P. A randomized controlled trial of the diagnostic ac- curacy of internet-based telepathology compared with conven- tional microscopy. Histopathology. 2007;50(2):266-73. 7. Nordrum I, Johansen M, Amin A, Isaksen V, Ludvigsen JA. Di- agnostic accuracy of second-opinion diagnoses based on still im- ages. Hum Pathol. 2004;35(1):129-35. 8. Costello SS, Johnston DJ, Dervan PA, O’Shea DG. Development and evaluation of the virtual pathology slide: a new tool in tele- pathology. J Med Internet Res. 2003;5:e11. 9. Massone C, Soyer PH, Lozzi GP, et al. Feasibility and diagnostic agreement in teledermatopathology using a virtual slide system. Hum Pathol. 2007;38(4):546-54. 10. Kaplan KJ, Burgess JR, Sandberg GD, Myers CP, Bigott TR, Greenspan RB. Use of robotic telepathology for frozen-section diagnosis: a retrospective trial of a telepathology system for in- traoperative consultation. Mod Pathol. 2002;15(11):1197-204. 11. Li X, Gong E, McNutt MA, et al.. Assessment of diagnostic ac- curacy and feasibility of dynamic telepathology in China. Hum Pathol. 2008;39(2):236-42. 12. Koch LH, Lampros JN, Delong LK, Chen SC, Woosley JT, Hood AF. Randomized comparison of virtual microscopy and tradi- tional glass microscopy in diagnostic accuracy among dermatol- ogy and pathology residents. Hum Pathol. 2009;40(5):662-7. 13. Nielsen PS, Lindebjerg J, Rasmussen J, Starklint H, Waldstrøm M, Nielsen B. Virtual microscopy: an evaluation of its validity and diagnostic performance in routine histologic diagnosis of skin tumors. Hum Pathol. 2010;41(12):1770-6.