SKIN March 2018 Volume 2 Issue 2 Copyright 2018 The National Society for Cutaneous Medicine 122 ORIGINAL RESEARCH Comparison of Survey Modality and Response Rate in Dermatologists’ Perceptions and Opinions of Sunscreens Alex M. Glazer MD a , Aaron S. Farberg MD b , Ryan M. Svoboda MD MS c , Darrell S. Rigel MD MS c a University of Arizona College of Medicine, Division of Dermatology, Tucson, AZ b Department of Dermatology, Icahn School of Medicine at Mount Sinai, New York, NY c National Society for Cutaneous Medicine, New York, NY d Ronald O. Perelman Department of Dermatology, NYU School of Medicine, New York, NY ABSTRACT Background. Survey instruments are valuable tools for research and provide insight into real-world practice and areas of knowledge deficits in ways that traditional observational studies and randomized trials cannot. Despite some experts espousing survey response rates ≥60% as valid, there is no consensus as to what an adequate response rate is for a scientific survey. Furthermore, little is known what effect the interaction of survey administration modality and response rate has on results. Objective. To compare the results of differing survey modalities (which typically feature different response rate ranges). Methods. A validated, 21-item survey assessing perceptions of, recommendations regarding, and usage of sunscreen was distributed to three samples of dermatologists using three different modalities: pen/paper via mail, online via email, and in real-time via an audience response system at a national conference. Results. Response rates varied widely by survey modality (30% mail, 9% email 95% live). However, dermatologists’ responses to individual survey questions were largely consistent across modalities, with a statistically significant difference seen for only three questions (recommending sunscreens based on cosmetic elegance, recommending sunscreens based on photostability, and recommending vitamin D supplementation as a means to avoid sun exposure. Conclusions. In this study evaluating dermatologists’ perceptions of, recommendations for, and personal use of sunscreen, survey results were largely consistent across three different modalities (mail, email, live) despite widely variable response rates, from 9% to 95%. These results suggest that when a scientific survey sample is representative of the target population, minimum response rate and survey modality appear to have negligible impact on results. SKIN March 2018 Volume 2 Issue 2 Copyright 2018 The National Society for Cutaneous Medicine 123 Physician management practices, knowledge gaps, and target areas for improvement are often assessed through research tools such as surveys. Surveys can be conducted using multiple methods including mail, internet, live audience response systems, or face-to-face interviews. Studies specifically comparing the differences in medical survey results based upon mode of administration are lacking. There is no consensus as to what an adequate response rate is for a survey to be considered publishable in scientific literature. There have been suggestions that high response rates are required for results to be valid and the JAMA network suggests a minimum rate of 60% for publication. 1,2 The purpose of this study was to compare differing survey modalities with lower response rates to assess if such high rates should be required. A 21 question survey assessing dermatologists’ perceptions, recommendation factors, and usage of sunscreen was derived and validated. The survey was administered between May and July 2016 to different subsets of US dermatologists using 1) a mailed paper survey (mailed to 500 randomly selected US AAD members), 2) a real-time computer administered survey at a national dermatology conference (165 US dermatologists) 3 , and 3) an emailed survey (sent to 6350 US dermatologists). 4 Stratification measures were taken to minimize respondent overlap. Results for each question were compared by modality using chi square and proportional difference testing where appropriate. The estimated sample size (n) to test a proportion to achieve a 95% confidence interval of less than 5% was computed using the standard statistical formula: Where N=population size and p=0.85 (lower a priori estimate of average affirmative response) and determined for our surveys (mail [28.2%], email [3.0%] and live [54.3%]). The survey was institutional review board exempt. The survey questions and comparative results are presented in the Table 1. The response rates for each modality (mail 30%, email 9%, live 95%) exceeded the determined threshold for validity. Respondent demographics were representative of US AAD membership. Only 3 of 63 responses had statistically significant (but not clinically relevant) differing responses. All other questions had results that were statistically consistent. Determining dermatologists’ perceptions, recommendations and usage of sunscreen appears to be independent of survey modality. Despite differing sample sizes and response rates for each modality employed, the same results were generally achieved. INTRODUCTION METHODS RESULTS DISCUSSION SKIN 124 Table 1. Dermatologists’ perceptions, recommendations, and use of sunscreen compared through differing survey modalities. Dermatologists’ Sunscreen Perceptions Live Interactive Audience (n=156) Mail Survey (n=150) Email Survey (n=540) Regular use of sunscreen helps lower skin cancer risk. 97% (CI: 93-99%) 100% (CI: 98-100%) 99% (CI: 98-100%) Regular use of sunscreen helps reduce subsequent photoaging. 100% (CI: 98-100%) 100% (CI: 98-100%) 99% (CI: 98-100%) Consider FDA approved sunscreens currently available in the US safe. 96% (CI: 92-99%) 95% (CI: 91-98%) 96% (CI: 94-97%) Consider oxybenzone in sunscreen to be safe. 91% (CI: 85-95%) 89% (CI: 83-93%) 86% (CI: 83-89%) Consider retinyl palmitate in sunscreen to be safe. 87% (CI: 81-82%) 85% (CI: 79-91%) 85% (CI: 82-88%) Patients generally under-apply sunscreen. 99% (CI: 95-100%) 99% (CI: 95-100%) 96% (CI: 94-97%) High SPF sunscreens (SPF 50+) provide additional margin of safety. 83% (CI: 77-89%) 76% (CI: 68-83%) 74% (CI: 70-78%) Best form of sunscreen is one used regularly. 99% (CI: 95-100%) 97% (CI: 93-99%) 97% (CI: 95-98%) Dermatologists’ Sunscreen Recommendations and Factors SPF - criteria used for sunscreen recommendation: 99% (CI: 95-100%) 98% (CI: 94-100%) 96% (CI: 94-97%) Broad Spectrum - criterion used for sunscreen recommendation: 96% (CI: 92-99%) 99% (CI: 96-100%) 98% (CI: 97-99%) Cosmetic Elegance/Feel - criterion used for sunscreen recommendation: 71% (CI: 63-78%)* 88% (CI: 82-93%) 85% (CI: 82-88%) Photostability - criteria used for sunscreen recommendation: 42% (CI: 34-51%)* 67% (CI: 59-74%) 68% (CI: 64-72%) Recommend sunscreen containing: Oxybenzone: 88% (CI: 82-93%) 86% (CI: 79-91%) 83% (CI: 80-86%) Recommend sunscreen containing: Retinyl palmitate: 78% (CI: 71-84%) 79% (CI: 72-86%) 80% (CI: 77-84%) Recommend sunscreen containing: Zinc oxide or titanium dioxide: 100% (CI: 98-100%) 100% (CI: 98-100%) 99% (CI: 98-100%) Recommend sunscreen with SPF 50 or higher: 97% (CI: 93-99%) 92% (CI: 86-96%) 92% (CI: 90-94%) Recommend Spray Formulations: 73% (CI: 65-80%) 74% (CI: 66-81%) 69% (CI: 65-73%) Recommend vitamin D by oral supplement rather than sun exposure. 80% (CI: 73-86%)* 86% (CI: 79-91%) 91% (CI: 88-93%) Percentage of patients that sunscreen is recommended for: 79% for 80%+ of patients (CI:72-85%) 78% for 80%+ of patients (CI:71-84%) 79% for 80%+ of patients (CI:75-82%) SPF level typically recommend to patients who are outdoors: 100% recommend SPF30 or more (CI:98-100%) 35% recommend SPF50 or more (CI:28-43%) 99% recommend SPF30 or more (CI:95-100%) 37% recommend SPF50 or more (CI:30-46%) 98% recommend SPF30 of more (CI:96-99%) 36% recommend SPF50 or more (CI:32-40%) Dermatologists’ Sunscreen Self/Family Usage When outdoors, the SPF level dermatologists typically choose: 100% choose SPF30+ (CI:98-100%) 69% choose SPF50+ (CI:61-76%) 100% choose SPF30+ (CI:98-100%) 67% choose SPF50+ (CI:59-74%) 99% choose SPF30+ (CI:98-100%) 63% choose SPF50+ (CI:59-67%) Dermatologists typically wear sunscreen themselves: 76% at least half time (CI:69-83%) 53% daily (CI:45-61%) 77% at least half time (CI:70-84%) 43% daily (CI:35-52%) 82% at least half time (CI:79-85%) 44% daily (CI:40-48%) Recommend family/friends use sunscreen to help protect their skin. 99% (CI:96-100%) 99% (CI:96-100%) 99% (CI:98-100%) SKIN March 2018 Volume 2 Issue 2 Copyright 2018 The National Society for Cutaneous Medicine 125 125 Although all three surveys had different population sizes and response rates, because each met statistical thresholds and was demographically representative, the results were not materially different. Standard inferential mathematical methods demonstrate that the percentage of responses required for a sample to be statistically valid varies inversely with the population of interest size. 5 For example, election polling typically utilizes a very small proportion to predict outcomes. 6 In fact, the reference cited to support a 60% minimum response rate was actually a “viewpoint” editorial based upon a paper sampling very small populations (and also asserts that the sample size percentage should decrease as the population size increases). 1 Our findings confirm this in contrast to a fixed-minimum survey response rate of 60% that is being used in some journals independent of population size. 1,2 Study limitations include possible non- responder, selection, acquiescence and social desirability bias that could have affected the survey results and potentially impacted the comparisons. When surveying dermatologists’ sunscreen opinions, live, online and traditional mail surveys provided similar results. Minimum response rates needed to achieve validity can be calculated as a function of population size using standard statistical methods when the sample is demographically representative of the studied population and should not be limited to a fixed minimum percentage. Conflict of Interest Disclosures: None. Funding: None Corresponding Author: Alex M. Glazer, MD Division of Dermatology, University of Arizona PO Box 245024 1515 N. Campbell Avenue Tucson, AZ 85724-5024 212-685-3252 (Office) alex.glazer@gmail.com References: 1. Fincham JE. Response rates and responsiveness for surveys, standards, and the Journal. Am J Pharm Educ. 2008;72:Article 43. 2. Instructions for Authors. The JAMA Network website. http://jamanetwork.com/journals/ja ma/pages/instructions-for-authors. Updated May 4, 2017. Accessed May 17, 2017. 3. Farberg AS, Glazer AM, Rigel AC, et al. Dermatologists’ perceptions, recommendations, and use of sunscreen. JAMA Dermatol. 2017;153(1):99-101. 4. Farberg AS, Rigel AC, Rigel DS. Online survey of US dermatologists’ sunscreen opinions: perceptions, recommendation factors, and self- usage. J Drugs Dermatol. 2016;15(9):1121-1123. 5. Survey research design. in: P.P. Phillips, J.J. Phillips, B. Aaron (Eds.) Survey basics. ASTD Press,Alexandria (VA); 2013. SKIN March 2018 Volume 2 Issue 2 Copyright 2018 The National Society for Cutaneous Medicine 126 126 6. Gaissmaeier W, Marewski JN. Forecasting elections with mere recognition from small, lousy samples: a comparison of collective recognition, wisdom of crowds, and representative polls. Judgment and Decision Making. 2011;6(1):73-88.