Microsoft Word - 1172_final.doc 1 Volume 16, Number 2 August 19, 2013 ISSN 1099-839X The Effect of Survey Mode on High School Risk Behavior Data: a Comparison between Web and Paper-based Surveys Shobana Raghupathy Sociometrics Corporation- Los Altos Stephen Hahn-Smith Contra Costa Behavioral Health Division There has been increasing interest in using of web-based surveys—rather than paper based surveys--for collecting data on alcohol and other drug use in middle and high schools in the US. However, prior research has indicated that respondent confidentiality is an underlying concern with online data collection especially when computer-assisted surveys are administered in group settings such as computer labs and classrooms. Using a sample of 341 high school students, we conducted a study comparing alcohol risk behavior prevalence rates from the web-based surveys with paper-form version of the same survey. The online surveys used in the study implemented several programming safeguards to enhance privacy. The goal of our study was to see if the risk-behavior items—when transitioned to an online format with these additional safeguards—would provide prevalence estimates that are comparable to its paper form version. Results showed that overall alcohol prevalence rates did not vary significantly between the two conditions. However, there was a significant interaction between gender and survey mode. Females reported lower mean risk indicator rates compared to males, possibly indicating stronger privacy concerns among the former group. The results suggest that online survey administration may require targeted efforts to help alleviate confidentiality concerns among adolescent girls. Keywords: online, alcohol, youth risk behavior, school health, web-based survey, methods. Access to computers and the Internet has become almost universal in schools in the United States (Wells & Lewis, 2006). Consequently, there has been increased research on the potential use of web-based surveys in elementary, middle and high schools for collecting data on high-risk health behaviors (Boyd, McCabe, Cranford, & Young, 2007; Cooper et al., 2006; McCabe, Boyd, Young, Crawford, & Pope, 2005). Computer and Internet- based survey administration has several significant advantages over traditional paper and pencil data collection methods (Lessler, Caspar, Penne, & Barker, 2000; O’ Reilly, Hubbard, Lessler, Biemer, & Turner, 1994; Turner et al., 1998). For example, it is quicker, and more cost-effective as it completely eliminates the need for manual data entry thus reducing transcription errors as well as workload (Turner, Rogers, Hendershot, Miller, & Thornberry, 1996; Turner et al., 1998; Lessler, Caspar, Penne, & Barker, 2000; O’ Reilly et al., 1994). Moreover, unlike paper surveys, the computerized survey allows for complex questionnaire designs by providing automated skip patterns (i.e. questions that do not apply to a specific respondent are automatically skipped). This feature makes computer-based surveys especially attractive for interviewing younger children who may be unable to Current Issues in Education Vol. 16 No. 2 2 maneuver complex branching in questionnaires (Beebe, Harrison, McRae, Anderson, & Fulkerson, 1998). One particular concern has been the effects of mode of data collection (web-based surveys, paper surveys) on respondents’ actual and perceived privacy and anonymity regarding data collected online. These perceptions are especially critical as they impact respondents’ willingness to disclose socially undesirable behaviors such as unsafe sexual practices and/or substance abuse (Sudman, 2001). The evidence is mixed in this regard. Several recent school-based studies in the United States and elsewhere using school computer labs have found that web-based surveys and paper modes generally produced equivalent youth risk behavior prevalence estimates (Bates & Cox, 2008; Denscombe, 2006; Eaton et al., 2010; McCabe et al., 2005; Van de Looij-Jansen & de Wilde, 2008). However, confidentiality remains an underlying concern especially when responses are visible on monitor screens and the online surveys are administered in group settings such as computer labs and classrooms (Beebe et al., 1998; Brenner et al., 2006). In particular, substance use prevalence among adolescents has varied depending on their perceptions of privacy (Aquilino, Wright, & Supple, 2000), anonymity (Supple, Acquilino, & Wright, 1999), and trust (Wright, Aquilino, & Supple, 1998). Recently, Denniston and his team of researchers (2010) conducted a comprehensive study comparing paper and pencil vs. web administration of the Youth Risk Behavior Survey (YRBS), which is the primary instrument used by schools in the United States for collecting data on a variety of adolescent high-risk behaviors. The research, sponsored by the Center for Disease Control, involved 85 high schools in the US. The study found that students taking an online health-risk behavior survey expressed greater concerns about privacy than those who took the same survey in paper form (Denniston et al., 2010). The study also found that the web-based survey produced more missing data due to nonresponses, compared to in-class administrations of paper surveys. The authors recommended against the administration of the YRBS in an online format: however, they indicated that steps to improve respondent privacy would possibly enhance the feasibility of using web-based surveys in schools, especially for small surveys that can be completed quickly. Similar observations about privacy were also noted by Beebe et al. (1998) who found that teen risk behavior reporting was higher among paper and pencil respondents –as compared to computer-based respondents-- when computers were placed closer (Beebe et al., 1998). Further examination of the feasibility of web- based data collection in schools is needed because there is evidence that students themselves prefer taking online surveys despite privacy concerns (Mangunkusumo et al., 2006; Case & Haines, 2004; Hallfors, Khatapoush, Kadushin, Watson, & Saxe, 2000; McCabe et al., 2005; McCabe, Boyd, Young, & Crawford, 2004). This preference may be based on the fact that web-based surveys—especially those that are audio-enhanced—are in a format that is more comprehensible to junior-high and vocational high school students, who might have lower literacy skills than senior-high school students (McCabe et al., 2004). In fact, the Office of Management and Budget (OMB) requires federally sponsored data collection efforts to offer an electronic mode of response whenever feasible, as this process is believed to reduce respondent burden (Office of Management and Budget, 1998). Indeed, given the cost and time advantages of online surveys compared to paper formats, the trend towards electronic data collection seems inevitable. Study Purpose Our study extends the research on the mode effects of web-based data collection of adolescent health data in school settings. Recently, our project team was invited to evaluate the feasibility of web-based data collection in a high school in rural Washington. Each year, the school collected risk indicator data from 9-12th graders by administering a 17-item paper survey on alcohol use risk behavior and attitudes. These items are drawn from the widely used Youth Risk Behavior Survey (YRBS), the same instrument that was the focus of the study by Denniston et al. (2010) discussed earlier. Our study incorporated several survey design features that were specifically addressed privacy concerns of the survey respondents. First, our survey was much smaller in scope and included just 17 items focused primarily on alcohol risk indicators (instead of all 90 plus items in the YRBS) as the school was mainly interested in measuring alcohol risk-behavior. We also incorporated programming features that promoted response confidentiality. The survey layout was designed to hold just one question at a time on the computer screen. Once a response was entered by the student, the survey automatically progressed to the next question thereby instantly refreshing the screen. This auto progression feature also eliminated nonresponses as the students could move on to the next question only after the previous one was answered. Online respondents were also automatically logged out after the last question. These privacy safeguards programmed into the web-based survey system were far more comprehensive than those used in prior research using online YRBS instruments, and were expected to enhance the confidentiality and anonymity of survey respondents1. Consequently, we hypothesized that alcohol use risk prevalence estimates would be higher among the group taking the online survey, compared to those using the paper survey, as the former group would be more forthcoming about their high risk behaviors when assured of complete confidentiality and anonymity. The Effect of Survey Mode on High School Risk Behavior Data: A Comparison Between Web and Paper-based Surveys 3 Methods Sample The study used a convenience sample of high school students in grades 9-12. Students were drawn from a single school from a predominantly rural school district in Washington State. A total of 341 high school students participated in this study comparing online versus paper form administration of an abbreviated version of the YRBS. The sample was fairly evenly split between male (50.4 percent) and female (49.6 percent) students. Almost two-thirds of students (63.6 percent) were in either 9th or 10th grade. About 60% of the sample was White, with 30% Native American2. About half the students were eligible for free or reduced priced school lunches. The school was given a stipend to assist with the study, and for supplying their computer lab for the study. Students were randomly assigned into two research conditions: one that took the online version of the survey (Condition A), and the other that took the paper form version (Condition B). The sample size of Condition A (online version) was 160 students, while Condition B consisted of 181 students. Data Collection Instruments and Procedures Project staff programmed the High School Youth Risk Behavior Survey as an online survey. The survey included 17 items on a number of alcohol use and risk indicator measures including short and long-term use, binge drinking, and perception of harm. Online surveys were administered in the school computer lab, while paper surveys were administered in classrooms. Several measures were undertaken to maximize privacy of responses. Each student was assigned a unique ID by a study coordinator hired by the project. Students in Condition A logged into the survey by entering his or her unique student ID and self-administered the survey. Responses entered into the online survey were directly deposited on an external server ensuring that teachers did not have access to the data. The survey screen held just one question at a time and progressed automatically to the next page upon completion of each question. This ensured that the screen was continually refreshed and did not display answers that were visible to everyone. Besides enhancing privacy, this auto progression feature also meant that students had to complete each question before moving on to the next one, which completely eliminated nonresponses as students not wishing to answer a question had to check on a “decline to answer” box. Students took about 25-30 minutes to complete the survey, which did not feature any skip patterns, and were automatically logged out upon completion. Data collection for each test was conducted over 1-2 days in order to account for lab availability. All data collection procedures were approved by the study organization’s full Institutional Review Board for the rights of human subjects in research. Active consent was sought from parents, while students turned in assent forms. Teachers were responsible for parental notifications at least two weeks before the study and for devising the schedules for computer lab use. A study coordinator hired by the project explained survey procedures and supervised the students in the computer lab during the survey administration process. Students were reminded that their answers would be kept confidential and that they would be returning to take a retest in two weeks’ time. Once the test and retest data were collected, project staff conducted the data analysis. Results: Alcohol Use Risk Prevalence rates – online versus paper-form The goal of the present study was to determine whether there were significant differences in the prevalence of high risk behaviors and attitudes regarding alcohol use depending on whether survey questions were presented on a computer versus on paper. Would the method of data collection have an effect on reporting tendencies of respondents? To examine this question, risk prevalence rates for each condition were examined. To simplify the analysis, questions were recoded into “high risk” versus “no risk” following coding used in past studies using the YRBS (e.g., Brenner et al., 2002). The coding procedure is outlined in Table 1. A Chi-Squared analysis was used to assess whether prevalence in each of the behavior risk factors varied between the two conditions. Contrary to our hypothesis, risk indicator estimates using the above coding scheme were found to be comparable between the two conditions. In other words, mode of survey administration did not impact alcohol risk prevalence rates and no significant differences were found between the two conditions. Out of the 17 individual items, just three registered significant differences between different modes of administration (Table 2). Analysis of Variance (ANOVA) was used to examine differential responses by subgroups, including Gender, Race, and Grade Level. Race and Grade Level did not vary significantly between conditions; however, a significant difference did emerge for Gender. 1 For example, in the study by Denniston et al. (2010), up to four questions appeared on each screen and response choices remained visible until students clicked the ‘‘next” button on the bottom of each screen. Also, students could advance to subsequent questions regardless of whether they left questions blank, which led to substantial nonresponses. 2 The school district lay in the outskirts of a reservation and has a high proportion of Native American students. Current Issues in Education Vol. 16 No. 2 4 Table 1 Coding Scheme for Computing Alcohol Risk Prevalence Estimates ITEM HIGH RISK NO RISK Lifetime alcohol use 1 or more times 0 times Alcohol use last 12 months 1 or more times 0 times Alcohol use last 30 days 1 or more times 0 times Binge drinking last 2 weeks 1 or more times 0 times Parents upset if they found you had been drinking after party Not at all upset to somewhat upset Very to extremely upset Been to party with other kids drinking over last 12 months 1 or more times 0 times Driven a car after drinking over last 12 months 1 or more times 0 times Ridden in car with someone who had been drinking over last 12 months 1 or more times 0 times Number of friends who drink alcohol once a week or more 1 or more friends 0 friends Easy for kids to get alcohol Very easy Sort of easy, difficult, or not sure Believe people risk harming themselves with 1-2 drinks per day No or not sure Yes Believe people risk harming themselves with 3-4 drinks per day No or not sure Yes Believe people risk harming themselves with 5 or more drinks per day No or not sure Yes Is underage drinking harmful to your health No or not sure Yes Is binge drinking dangerous No or not sure Yes Wrong for kids under 21 to drink No or not sure Yes Able to say “no” when all friends are drinking No or not sure Yes The Effect of Survey Mode on High School Risk Behavior Data: A Comparison Between Web and Paper-based Surveys 5 Table 2 Prevalence of Substance use Variables: Overall and Within Each Survey Mode Total (n=341) Condition A Online (n=161) Condition B P & P (n=180) X2 p-value Lifetime alcohol use 73.1% 70.0% 75.8% NS Alcohol use last 12 months 56.6% 55.3% 57.9% NS Alcohol use last 30 days 31.9% 31.1% 32.6% NS Binge drinking last 2 weeks 19.8% 18.0% 21.3% NS Parents upset if they found you had been drinking after party 71.9% 71.9% 71.9% NS Been to party with other kids drinking over last 12 months 53.1% 47.8% 57.9% NS Driven a car after drinking over last 12 months 7.7% 8.7% 6.7% NS Ridden in car with someone who had been drinking over last 12 months 41.6% 36.6% 46.1% NS Number of friends who drink alcohol once a week or more 60.7% 57.8% 63.3% NS Easy for kids to get alcohol 66.0% 68.8% 63.5% NS Believe people risk harming themselves with 1-2 drinks per day 44.1% 37.5% 50.0% p<.05 Believe people risk harming themselves with 3-4 drinks per day 24.2% 23.6% 24.7% NS Believe people risk harming themselves with 5 or more drinks per day 10.9% 8.7% 12.9% NS Is underage drinking harmful to your health 19.8% 16.1% 23.0% NS Is binge drinking dangerous 14.2% 10.6% 17.4% p<.05 Wrong for kids under 21 to drink 47.6% 46.3% 48.9% NS Able to say “no” when all friends are drinking 27.1% 22.4% 31.5% p<.05 Current Issues in Education Vol. 16 No. 2 6 Table 3 Prevalence Rates by Condition and Gender Question Item Male Female Online (n=87) P&P (n=84 ) X2 p-value Onlin e (n=74 ) P&P (n=94 ) X2 p-value Lifetime alcohol use 76.7% 78.6% NS 62.2 % 73.4 % NS Alcohol use last 12 months 59.8% 59.5% NS 50.0 % 56.4 % NS Alcohol use last 30 days 37.9% 35.7% NS 23.0 % 29.8 % NS Binge drinking last 2 weeks 21.8% 26.2% NS 13.5 % 17.0 % NS Parents upset if they found you had been drinking after party 67.8% 73.8% NS 76.7 % 70.2 % NS Been to party with other kids drinking over last 12 months 54.0% 59.5% NS 40.5 % 56.4 % p<.05 Driven a car after drinking over last 12 months 10.3% 6.0% NS 6.8% 7.4% NS Ridden in car with someone who had been drinking over last 12 months 43.7% 44.0% NS 28.4 % 47.9 % p<.01 Number of friends who drink alcohol once a week or more 62.1% 70.2% NS 52.7 % 57.0 % NS Easy for kids to get alcohol 68.6% 65.5% NS 68.9 % 61.7 % NS Believe people risk harming themselves with 1-2 drinks per day 45.3% 52.4% NS 28.4 % 47.9 % p<.01 Believe people risk harming themselves with 3-4 drinks per day 31.0% 22.6% NS 14.9 % 26.6 % p<.05 Believe people risk harming themselves with 5 or more drinks per day 10.3% 10.7% NS 6.8% 14.9 % NS P0Is underage drinking harmful to your health 23.0% 22.6% NS 8.1% 23.4 % p<.01 Is binge drinking dangerous 18.4% 17.9% NS 1.4% 17.0 % p<.001 Wrong for kids under 21 to drink 55.2% 47.6% NS 35.6 % 50.0 % p<.05 Able to say “no” when all friends are drinking 29.9% 32.1% NS 13.5 % 30.9 % p<.01 Risk Index 42.1% 42.6% NS 31.2 % 40.5 % p<.01 Notably, females reported lower mean risk indicator rates (M=36.4 percent) compared to males (M=42.7 percent), F(1,337)= 8.15, p<.01. More importantly for this study, there was a significant interaction between gender and whether students took the survey online or on paper-and-pencil, F(1,335)=4.40, p<.05. To analyze this trend further, a new variable, Risk_Index, was created to assess for mean differences between all four conditions (Table 3). The Risk_Index was simply the mean prevalence across all 17 items for each student. While males had non-significant differences in responses across question items between the two survey methods, females had significantly higher risk prevalence rates in the paper-and-pencil condition (Risk index=40.5 percent) relative to the online condition (Risk index=31.2 percent). To examine these differences further, a Chi- square analysis was performed for each question separately by gender (Table 3). For males, there were no question items that varied significantly between survey The Effect of Survey Mode on High School Risk Behavior Data: A Comparison Between Web and Paper-based Surveys 7 modality types. For females, however, 9 of the 17 risk prevalence questions varied significantly in their prevalence rates between survey administration modality. In all cases, females reported higher risk prevalence rates in the paper-and-pencil administration compared to the online method, suggesting that former group was more comfortable about revealing sensitive information about their alcohol risk behaviors. Data Quality In a prior study, Denniston et al., (2010) saw far greater percentage of incomplete surveys and missing data among online respondents. Our efforts at improving data confidentiality by programming privacy safeguards and restricting survey length seem to have had a positive effect on data quality. The number of missing or invalid responses across all questions was nearly non-existent for both the online and paper and pencil version of the survey. These results indicate that students in both groups were responsive to questions. This could be due to short length of the survey as well as because of the additional programming safeguards taken to enhance respondent confidentiality in the computer condition. General Discussion The goal of this study was to gain a better understanding of differences that may exist when students take a health risk behavior survey using the traditional paper-and-pencil method of administration, compared to taking it online. Risk indicator rates for alcohol-related risk behaviors were examined between the online and the paper-and-pencil versions of a 17-item survey whose items were drawn from the YRBS. We developed innovative programming safeguards to enhance respondent privacy and anonymity. These included elements such as auto-progression, forced response, instant screen refreshing and automatic logout. Despite these features, our study did not detect higher risk prevalence among the online respondents, as was hypothesized. The alcohol risk indicators were comparable between the two survey modes, which is similar to the result found in earlier studies that used the YRBS (Eaton et al., 2010; Denniston et al., 2010). However, these additional privacy safeguards did improve data quality, and eliminated problems of missing data due to incomplete surveys and nonresponses that was found in the study by Denniston et al. (2010). The most significant finding in our study, however, was revealed when examining the data separately by gender. On the majority of questions, females were more likely to endorse high risk items in the paper-and-pencil condition compared to the online condition. These results suggest that while overall risk prevalence rates for alcohol use do not vary much between survey conditions, there can be significant differences within specific subgroups (e.g., females). In this case, risk behavior prevalence rates for females were higher in the paper and pencil condition, indicating that perhaps female respondents were more confident in disclosing high risk behaviors on paper than they were when responding on a computer. These results are worthy of more investigation. There is some evidence in the field of ecommerce and consumer behavior that shows male-female differences in Internet privacy risk perception and protection behaviors. For example, Youn and Hall (2008) found that girls perceived themselves as more susceptible to unsolicited email than boys, and perceived privacy risks to be more severe than boys. Girls were also less likely to read unsolicited emails and tended to perceive provide inaccurate information boys online. It is therefore unsurprising that such online privacy concerns manifested themselves in self-reports of sensitive risk behavior data as well. The practical implications of our finding are clear. Special attention should be paid to the specific environmental conditions, which may affect privacy concerns when students are completing surveys on sensitive information. This includes computer spacing, viewable area, the progression of survey questions, and other environmental factors that contribute to perceived privacy, which can have a profound impact on questions of a sensitive nature. Equally important, attention should be paid to understanding the deeper dynamics at play regarding how different subgroups of students respond to survey questions in light of the mode in which they are administered. Higher levels of risk perception and privacy concerns can cause girls to be less than forthcoming in their responses, and special efforts may needed to help girls alleviate these concerns, over and above environmental conditions. It would be a great contribution to the literature to create an established standard and best practice to follow when students are responding to surveys containing sensitive data. This issue is particularly important now as computers are ubiquitous and computer-based surveys are becoming more and more prevalent. References Aquilino, W. S., Wright, D. L., & Supple, A. J. (2000). Response effects due to bystander presence in CASI and paper-and-pencil surveys of drug use and alcohol use. Substance Use and Misuse, 35, 845–67. Bates S. C., & Cox, J. M. (2008). The impact of computer versus paper and pencil survey and individual versus group administration, on self-reports of sensitive behaviors. Computers in Human Behavior, 24, 903-916. Beebe T. J, Harrison, P. A., McRae, J. A., Anderson, R. A., & Fulkerson, J. A. (1998). An evaluation of computer-assisted self-interviews in a school setting. Public Opinion Quarterly, 62, 623-632. Boyd, C. J., McCabe, S. E., Cranford, J. A., & Young, A. (2007). Prescription drug abuse and diversion Current Issues in Education Vol. 16 No. 2 8 among adolescents in a Southeast Michigan School District. Archives of Pediatrics and Adolescent Medicine, 161(3), 276-281. Brener, N. D., Eaton, D. K., Kann, L., Grunbaum, J. A., Gross, L.A., … & Kyle T. M. (2006). The association of survey setting and mode with self- reported health risk behaviors among high- school students. Public Opinion Quarterly, 70(3), 354-374. Case, S., & Haines, K. (2004). Developing a computer- based interactive research questionnaire for educational consultation with young people. The Journal of Society and Information, 1(2). Cooper, C. J., Cooper, S. P., del Junco, D. J., Shipp, E. M., Whitworth, R., & Cooper, S. R (2006). Web- based data collection: detailed methods of a questionnaire and data gathering tool. Epidemiologic Perspectives & Innovations, 3(1). Denniston, M. M., Brener, N. D., Eaton, D. K., McManus, T., Kyle, T. M., Roberts, A. M., Flint, K. H., & Ross, J. G. (2010). Comparison of paper and pencil vs. web administration of the Youth Risk Behavior Survey (YRBS): Participation, data quality and perceived privacy and anonymity. Computers in Human Behavior, 26, 1054-1060. Denscombe, M. (2006). Web-based questionnaires and the mode effect: An evaluation based on completion rates and data contents of near- identical questionnaires delivered in different modes. Social Science Computer Review, 24(2), 246-254. Eaton, D. K., Brener, N. D., Kann, L., Denniston, M., McManus, T., Kyle, T. M., Roberts, A. M., Flint, K. H., & Ross, J. G. (2010). Comparison of paper-and-pencil versus web administration of the Youth Risk Behavior Survey (YRBS): Risk behavior prevalence estimates. Evaluation Review, 34, 137-152. Hallfors, D., Khatapoush, S., Kadushin, C., Watson, K., & Saxe, L. (2000). A comparison of paper vs. computer-assisted self-interview for school alcohol, tobacco, and other drug surveys. Evaluation and Program Planning, 23, 149-155. Lessler, J., Caspar, R., Penne, M., & Barker, P. (2000). Developing computer assisted interviewing (CAI) for the National Household Survey on Drug Abuse. Journal of Drug Issues, 30, 9-34. Mangunkusumo, R. T., Duisterhout, J. S., de Graaff, N., Maarsingh, E. J., de Koning, H. J., & Raat, H. (2006). Internet versus paper mode of health and health behavior questionnaires in elementary schools: Asthma and fruit as examples. Journal of School Health, 76(2), 80-86. McCabe, S. E., Boyd C. J., Young, A., Crawford, S., & Pope, D. (2005). Mode effects for collecting alcohol and tobacco data among 3rd and 4rth grade students: a randomized pilot study of web- form versus paper-form surveys. Addictive Behaviors, 30(4), 663-671. McCabe, S. E., Boyd, C. J., Young, A., & Crawford, S. (2004). Feasibility study for collecting alcohol and other drug use data among secondary school students: A web-based approach. Journal of Drug Education, 34(4), 373-383. Office of Management and Budget. (1998). Government Paperwork Elimination Act. 44 USC 3504. Retrieved from http://159.142.166.204/Documents/paperwork_el imination_act.html. O'Reilly, J. M., Hubbard, M. L., Lessler, J., Biemer, P., & Turner, C. F. (1994). Audio and video computer- assisted self-interviewing: Preliminary tests of new technologies for data collection. Journal of Official Statistics, 10, 197-214. Sudman, S. (2001). Examining substance abuse data collection methodologies. Journal of Drug Issues, 31, 695-716. Supple, A. J., Aquilino, W. S., & Wright, D. L. (1999). Collecting sensitive self-report data with laptop computers: Impact on the response tendencies of adolescents in a home interview. Journal of Research on Adolescence, 9, 467–88. Turner, C., Ku, L., Rogers, S., Lindberg, L., Pleck, H., & Sonenstein, F. (1998). Adolescent sexual behavior, drug use, and violence: Increased reporting with computer survey technology. Science, 280, 867-873. Turner, C., Rogers, S. M., Hendershot, T. P., Miller, H. G., & Thornberry, J. P. (1996). Multilingual audio-CASI: Using English-speaking field interviewers to survey elderly Korean household. Public Health Reports, 111, 276-279. Van de Looji-Jansen, P. M., & de Wilde, E. J. (2008). Comparison of web-based vs. paper and pencil self-administered questionnaire: Effects on health indicators in Dutch adolescents. Health Research and Educational Trust, 43, 1-13. Wells, J., & Lewis, L. (2006). Internet access in U.S. public schools and classrooms: 1994-2005 (NCES 2007-020). U.S. Department of Education. Washington, DC: National Center for Education Statistics. Wright, D. L., Aquilino, W. S., & Supple, A. J. (1998). A comparison of computer-assisted and paper-and- pencil self-administered questionnaires in a survey on smoking, alcohol, and drug use. Public Opinion Quarterly, 62, 331–53. Youn, S., & Hall, K. (2008). Gender and online privacy among teens: risk perception, Privacy concerns, and protection behaviors. CyberPsychology & Behavior, 11, 763-765. The Effect of Survey Mode on High School Risk Behavior Data: A Comparison Between Web and Paper-based Surveys 9 Article Citation Raghupathy, S., & Hahn-Smith, S. (2013). The effect of survey mode on high school risk behavior data: A comparison between web and paper-based surveys. Current Issues in Education, 16(2). Retrieved from http://cie.asu.edu/ojs/index.php/cieatasu/article/view/1172 Author Notes Shobana Raghupathy Sociometrics Corporation 201 Main Street, Suite 100, Los Altos, CA 94022 Shobana@socio.com Dr. Raghupathy is Senior Research Associate at Sociometrics Corporation. Since 1997, when she secured her Ph.D degree in Sociology from Brown University, Dr. Raghupathy has played a leading role in transferring prevention research into practice, primarily in the area of adolescent health. Her work includes development of science-based resources for youth-based drug prevention; multimedia tools for evaluation and implementation; practitioner assistance and training on program replication and adaptation. Dr. Raghupathy has secured numerous grants from the National Institute of Health for creating school-based health education tools for youth. These included an online data collection system for school districts; a drug prevention program for Native American elementary school children; and currently, a suicide prevention intervention for Native American high schoolers. Dr. Raghupathy has established a solid track record as a prevention researcher, having served as Research Advisors to several external public and private research organizations (California Healthy Kids Resource Center, California Department of Education; Pacific Institute for Research and Evaluation and California Department of Health Services; NIDA and Center for Scientific Review). Prior to joining Sociometrics, Dr. Raghupathy worked in international organizations such as the World Bank and the UNICEF. Stephen Hahn-Smith Behavioral Health Services Contra Costa Behavioral Health Division 1340 Arnold Dr. #200 Martinez, CA 94553 Steve.Hahn-Smith@hsd.cccounty.us Stephen Hahn-Smith, Ph.D. is a Research and Evaluation Manager at the Mental Health Division, Contra Costa County. This role involves oversight of all research activity for the division of mental health, including the collection of performance measures for consumers in the system, monetary and revenue analysis, productivity reporting, and consumer satisfaction surveys. Recently, he has been part of the planning and evaluation team for the Mental Health Services Act programs, a new initiative for innovative programs for consumers of the public mental health system. He has also been involved in several large scale cross-site studies, including the CAASP Infrastructure grant in New Mexico; the statewide system of care evaluation while at the University of California, San Francisco; and several prevention related cross-site studies while at EMT Associates. Currently, he is the Coordinator of Research and Evaluation in Contra Costa County for the public mental health plan. Dr. Hahn-Smith received his Ph.D from the University of New Mexico in 1996; Master’s in Education from Harvard University, and Bachelor of Arts from University of Colorado, Boulder. Current Issues in Education Vol. 16 No. 2 10 Manuscript received: 01/24/2013 Revisions received: 06/03/2013 Accepted: 07/13/2013 The Effect of Survey Mode on High School Risk Behavior Data: A Comparison Between Web and Paper-based Surveys 11 Volume 16, Number 2 August 19, 2013 ISSN 1099-839X Authors hold the copyright to articles published in Current Issues in Education. Requests to reprint CIE articles in other journals should be addressed to the author. Reprints should credit CIE as the original publisher and include the URL of the CIE publication. Permission is hereby granted to copy any article, provided CIE is credited and copies are not sold. Editorial Team Executive Editors Melinda A. Hollis Rory Schmitt Assistant Executive Editors Laura Busby Elizabeth Reyes Layout Editors Bonnie Mazza Elizabeth Reyes Recruitment Editor Hillary Andrelchik Copy Editor/Proofreader Lucinda Watson Authentications Editor Lisa Lacy Technical Consultant Andrew J. Thomas Section Editors Ayfer Gokalp David Isaac Hernandez-Saca Linda S. Krecker Carol Masser Bonnie Mazza Constantin Schreiber Faculty Advisors Dr. Gustavo E. Fischman Dr. Jeanne M. Powers