Layout 1


ISDS Annual Conference Proceedings 2012. This is an Open Access article distributed under the terms of the Creative Commons Attribution-
Noncommercial 3.0 Unported License (http://creativecommons.org/licenses/by-nc/3.0/), permitting all non-commercial use, distribution, and
reproduction in any medium, provided the original work is properly cited.

ISDS 2012 Conference Abstracts

An Experimental Study Using Opt-in Internet Panel
Surveys for Behavioral Health Surveillance
Carol A. Gotway Crawford*1, Catherine A. Okoro1, Haci M. Akcin1 and Satvinder Dhingra2, 1

1Centers for Disease Control, Atlanta, GA, USA; 2Northrop Grumman, Atlanta, GA, USA

Objective
To present the design and preliminary results of a pilot study to in-

vestigate the use of opt-in Internet panel surveys for behavioral health
surveillance.

Introduction
Today, surveyors in both the private and public sectors are facing

considerable challenges with random digit dialed (RDD) landline
telephone samples. The population coverage rates for landline tele-
phone surveys are being eroded by wireless-only households,
portable telephone numbers, telecommunication barriers (e.g., call
forwarding, call blocking and pager connections), technological bar-
riers (call-blocking, busy circuits) and increased refusal rates and pri-
vacy concerns. Addressing these issues increasingly drives up the
costs associated with dual-frame telephone surveys designed to be
representative of the target population as well as hinders their ability
to be fully representative of the adult population of each state and ter-
ritory in the United States. 

In an effort to continue to meet these challenges head on and as-
sist state and territorial public health professionals in the continued
collection of data that are representative of their respective popula-
tions, novel approaches to behavioral health surveillance need con-
tinued examination. Both private and public sector researchers are
evaluating the use of Internet opt-in panels to augment dual-frame
RDD survey methods. Compared to dual-frame RDD, opt-in Internet
panels offer lower costs, quick data collection and dissemination, and
the ability to gather additional data on panelists over time. However,
as with dual-frame RDD, this mode has similar challenges with cov-
erage error and non-response. Nevertheless, survey methodologists
are moving forward and exploring ways to reduce or eliminate biases
between the sample and the target population.

Methods
A collaborative pilot project was designed to assess the feasibility

and accuracy of opt-in Internet panel surveys for behavioral health
surveillance. This pilot project is a collaboration between the CDC,
four state departments of health, opt-in Internet panel providers and
the leads of several large surveys and systems such as the Patient-
Reported Outcome Measures Information System (PROMIS) and the
Cooperative Congressional Election Study (CCES). Pilot projects
were conducted in four states (GA, IL, NY, and TX) and four Metro-
politan Statistical Areas (Atlanta, Chicago, New York City, and Hous-

ton). Data were collected using three different opt-in Internet panels
and sampling methods that differ with respect to recruitment strat-
egy, sample selection and sample matching to the adult population of
each geography. A question bank consisting of 80 questions was de-
veloped to benchmark with other existing surveys used to assess var-
ious public health surveillance measures (e.g., the Behavioral Risk
Factor Surveillance System, the PROMIS, National Survey on Drug
Use and Health, and the CCES).

Results
We present comparative analyses that assess the advantages and

disadvantages of different opt-in Internet panels sampling method-
ologies across a range of parameters including cost, geography, time-
liness, usability, and ease of use for technology transfer to states and
local communities. Recommendations for future efforts in behavioral
health surveillance are given based on these results.

Keywords
Random Digit Dialing; BRFSS; Survey Methods; Sample Matching;
Representativeness

References

Ansolabehere S, Schaffner BF. 2010. Re-examining the validity of dif-
ferent survey modes for measuring public opinion in the U.S.: Find-
ings from a 2010 multi-mode comparison. http://projects.iq.harvard.
edu/cces/files/ansolabehere_schaffner_mode.pdf.

Curtin R, Presser S, Singer E. 2005. Changes in telephone survey nonre-
sponse over the past quarter century. Public Opinion Quar-
terly;69(1):87-98.

Liu H, Cella D, Gershon R, Shen J, Morales LS, Riley W, Hays RD. 2010.
Representativeness of the Patient-Reported Outcomes Measurement
Information System Internet panel. J Clin Epidemiol 63(11):1169-78.

Rivers, D., 2007. Sampling for Web Surveys, Paper presented at the Joint
Statistical Meetings. http://www.laits.utexas.edu/txp_media/html/
poll/files/Rivers_matching.pdf

*Carol A. Gotway Crawford
E-mail: cdg7@cdc.gov

Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(1):e24, 2013