44 - iassist quarterly The sampling bias in random digit dialing by A. Dianne Schmidley' Bell Atlantic Corporation All research is based on information from data collection efforts, whether the data result from the use of qualitative approaches, such as, content analysis, focus group sessions, or in-depth interviewing by psychologists, social workers or ethnographers, or from methods which produce data more amenable to quantitative analyses, such as, sample surveys and censuses (technically, a census is a 100% sample stirvey). A study-, conducted by Derek Phillips in the early 1970s, concerning the preference of sociologists for either qtiantitative or qualitative data collection approaches, revealed that more than 90 percent of the research conducted by those social scientists resulted from the analysis of data collected through the administration of interview schedules and/or questionnaires developed for use in a survey setting. While sociologists are generally viewed as major collectors and users of the data resulting from surveys, they are not the only individuals interested in survey data. Pollsters, advertisers, program administrators and others have important uses for survey data. In fact, there are several broad categories of surveys including: 1. Attitude and public opinion polls, 2. Marketing research, including advertising and public relations surveys, 3. Government surveys, especially those conducted for the purposes of developing legislation and administering and evaluating the effectiveness of programs, 'Presented at the International Association for Social Science Information Service and Technology (IASSIST) Conference held in Manna Del Rev. California on Mav 21-24 1986 4. Special surveys conducted by researchers in university settings, usually the basis of primary research, and often government funded, 5. Other surveys, including internal ' Knowledge from What? Fall/Winter 1987 tassist quarterly - 45 organizational surveys conducted to monitor attitudes and opinions. Individuals with training in survey research methods direct many of these collection activities; however, many surveys, particularly those in the areas of marketing, public relations and opinion research, are conducted by individuals who have no formal training and are not aware of the many pitfalls of survey data. Once the ill-gotten information is translated into copy, it acquires a life of its own and the 'facts' become almost impossible to erase. The use of data from a survey, for some purpose other than that for which it was collected, is common. For example, data from govemmeni surveys such as the Cunent Population Survey (CPS) are often used as the basis of marketing decisions, while data from a public opinion survey may influence the decisions of government policy makers. The data from all five categories of surveys are utilized to support policy decisions affecting the expenditure of millions of dollars. Given the penchant of policy makers for basing decisions on data collected through the use of surveys, the need for a critical appraisal of each of the various stages of the survey process has evolved. Questionnaire/interviewer schedule design, sample selection, the adminisuation of the survey process, the collection, tabulation and interpretation of the data, and testing of the reliability- and validity- of the information collected from the survey, compared to some benchmark, have all become subareas of the survey research process and are carefully scrutinized by trained social scientists. The purpose of this paper is to shed additional light on the problem of selecting a representative sample of a population to be surveyed, using the procedure of simple random sampling. When a sample is selected for a survey, attention must be given to the parameters of the universe the researcher hopes to study, and the method of eliciting information from that universe. In the case where the universe to be studied is a human population, there are a limited number of ways of operationally defining the universe and collecting information from the individuals comprising that universe. Each type of definition will affect the method of sample selection chosen. If the universe is operationally defined by street address records, the researcher can mail a questionnaire to each individual respondent If telephone numbers constitute the universe, the researcher can telephone respondents and administer a questionnaire/interview schedule. With street address records, the researcher can meet with each respondent and conduct interviews face to face. Generally, evaluations of each of these three approaches suggest that the higher the quality of the data collected (given proper development of the survey instrument), the greater is the expense of the survey. Mail-back questionnaires are the least expensive way in which to collect information and result in the poorest quality- data with the lowest response rates, while in-person interviews are the most expensive way in which to collect the information, but produce better respondent cooperation. Given this set of circumstances most researchers employ the telephone as a means of collecting survey information, since it produces medium quality data at medium cost I mentioned earlier that when drawing a sample from a universe, the researcher operationalizes the sampling process by giving the members of the populaton s/he hopes to study concreteness in the form of a street address or a telephone number. Client/customer specific lists, collected by some agencies and firms, can be used when the researcher is interested in some subset of the population, and thereby has access to the specific names, telephone numbers and/or Fall/Winter 1987 46 - lassist quarterly address records of that group. In most instances, however, the universe to be studied is only vaguely known. It may be comprised of all the individuals in some specific geographic location such as a trading area, county or school zone, or it may be a subset of the population, such as households with a certain income, or people of certain ages or ethnicity. At any rate, owing to a lack of specific information which could be used to contact and interview the individuals of interest in the population, the researcher more often uses a method of simple random sampling (SRS) of available street records or telephone numbers in order to delineate those units which will be sampled for the purpose of collecting information which can then be ascribed to the larger universe from which the sample was drawn. The preference of survey researchers for telephone interviewing as a means of collecting information from respondents, coupled with the need to employ SRS in order to to identify the individuals to be surveyed within the universe in which they are located, has led recently to the development of a technique called 'Random Digit Dialing' (RDD). RDD IS believed by the naive to be a cure for drawing biased or non-representative samples from a universe. Misinformed advocates of RDD persist in believing that universal telephone service means that everyone has a telephone in their home. Secure in this belief, RDD practicioners promote the idea that by simply dialing the telephone in a random manner one can draw a random sample of the population of any given geographic area. Census data, which are used by experienced survey researchers, to design sampling frames and to calibrate survey results, indicate that universal telephone service does not mean that a telephone is found in every housing unit 'Universal telephone service' is akin to the notion of 'full employment'. Full employment is defined by many as the situation in which about 5% of the population in the labor force, ages 16 through 64, and desiring employment, is unemployed. There aie many persons between the ages of 16 and 64 who are not employed, and are not seeking employment; so full employment does not mean that 5% of this entire age group is unemployed. Similarly, imiversal telephone service does not mean that every housing unit contains telephone network access. There are a variety of reasons why a imit may be without access to the telephone network. Attachment A to this paper gives the reader the sources utilized to determine something called telephone penetration. As can be seen in this attachment, there are basically four ways in which to measure telephone penetration: (1) using decennial census data, (2) using information from the CPS, (3) using GBF-DIME records, and (4) comparing household counts with counts of telephone access lines in given geographic areas. Examples of the decennial census and CPS measures of telephone penetration of each of jurisdictions served by Bell Atlantic appear in Table 1, entitled, "Telephone Availabilit}' For Selected Areas: United States and Bell Atlantic Ser\'ed States, 1980-1985". Figures 1 and 2 illustrate changes over time in telephone penetration in these geographic locations. Table 2 provides and indication of the type of statistic that can be developed using the Access Lines/Households ratio. Several conclusions are possible from a careful examination of these various measures of telephone penetration: 1. Data from different questions result in different statistics; i.e., the two censuses produced different results partly owing to question wording. These differences. fall/Winter 1987 iassisl quarterly - 47 although found in a comparison of data from the 1970 and 1980 censuses, are illustrated in Table 1 in the 'unit' and 'availability' meastires from the CPS, 2. In addition to these difTerences, however, there are major difTerences between geographic locations (Figtires 1 and 2) and, 3. Differences have occurred over time (Table 2), 4. DifTerences result from the sample selection process. Table la reveals the differences in sample drawn from 1980 census data (Colunms a and b); Table lb illustrates the range associated with each confidence interval computed for the various point estimates of penetration computed from CPS data. Thus fai, we have examined the penetration rate for the total nimiber of households. When differences in ty-pes of households is taken into account, the variation in telephone penetration becomes even greater. The data in in Table 3 are from a 1980 Census Public Use Microdata Sample (PUMS). Table 3 contains a cross-tabulation of households by age of householder, household income, and telephone penetration for the state of Virginia. These data indicate that the presence of a telephone in the housing unit is not a random event, but rather thai one is less likely to have a phone if one is poor and/or young. Figure 3 illustrates the ratio of telephone access lines to households in the Bell Atlantic region from the early 1950s to the 1990s forecast period. A surprising event occurred in the late 1970s: the number of access lines increased to the point of outnimibering households. How can this be? The answer is quite simple, it's called multiple lines. In the state of New Jersey alone, more than 10% of households have ai least two lines, and these households are not poor and they are not yoimg. Can one draw a random sample, utilizing RDD? Yes, but a random sample of what? Certainly one will not draw a random sample of the households in a given geographic location. One will draw a random sample of telephone hnes and, as we have seen, these are not evenly distributed across the population. Does this matter? It depends on what one is trying to accomplish. A pollster, trying to predict an election, will want to examine age specific voting patterns and calibrate these to the universe of households with telephones, allowing for multiple lines, of course. Since the people who vote tend to be those with higher incomes, and voter participation increases with age, you will reach the group you wish to sample using telephone interviewing, although you raa\ overstate the case, especially if you are ttacking Republican candidates. (Table 4) A marketing researcher, must be careful to calibrate the resulting data with census data, because the upscale population will be over-represented and other groups imder-represented in the sample. An academic or government researcher, conducting a survey designed to produce information for the development of legislation or the operation of a social program, would need to calibrate the data with census or CPS data in order to enstiie a representativeness. In conclusion, it seems highly unlikely that telephone interviewing is a reasonable replacement for the Bureau of the Census traditional decetmial census methods of data collection employing a housing address hst, mailout questionnaires and enumerators. Given the problems associated with the universe of telephone access lines, if telephone interviewing becomes the mainstay of data collection for the U.S. Government, we will have lost one of the most important means we have of analysing the composition of our population, n Fall/Winter 1987 iassist quarterly Attachment A METHODS OF MEASURING TELEPHONE AVAlLABlLlTr/PESETRATION (STATE LEVEL) I. Decennial Census of Population and Housing Questionnaire 1970 IHl Is there a telephone on which people living in your quarters can be called? I I yes what is the number? Phone number no 1980 fHZd Do you have a telephone in your living quarters? yes n "° Caveats: * Timliness of the data • Validity I Reliability issues II. Current Population Survey Interview Supplementary questions regarding telephone availability asked in March, July & November of each year of a state based sample of households Caveats : * Sampling error at state level * Coverage of non-Bell Atlantic served areas III. Geobased Files/Dual Independent Map Encoding Matching telephone customer address records with street address records in the GBF Caveats : * Non-urbanized areas not covered * New housing not covered, although it can be inferred from telephone company records IV. Residence Acess Lines/Household Estimates; Use telephone company access line information in the' numerator and household estimates prepared by company demographer in the denominator Caveats : * Second line development; failure of business office to identify second line • Accuracy of household estimates; OK for internal purposes, may not stand up in legal/regulatory setting fall/Winter 1987 iassist quarterly - 49 Table 1-A s. s s ^ 5 ^ oi oi /l o - r- £ « a f^ 1/1 o o f^ ri^ ct! « $ i » S 3: $ $ > OJ CS >s— *3 E '£ S. i<. - O >, E fall/Winter 1987 50 - iassisl quarterly Table 1-B Confidence Intervals Associated With Point Estimates Of Telephone Penetration for Bell Atlantic Served Jurisdictions Prepared Demogr by D. Schmidley aphic Studies x8638 (1) State (2) Unit Measure Nov. 1985 95.6 (3) CVx (4) SEx 1.42 (5) Range 68^ (6) Range 95+ % DC 0.0149 94.2 - 97.0 92.8 - 98.4 DEL 93.4 0.0119 1.11 92.3 - 54.5 91.2 - 95.6 MD 95.3 0.0120 1.14 94.2 - 95.4 93.0 - 97.6 NJ 94.1 0.0093 0.88 93.2 - 95.0 92.3 - 95.9 PA 95.8 0.0059 0.57 95.2 - 95.4 94.7 - 96.9 VA 92.0 0.0156 1.44 90.6 - 93.4 89.1 -94.9 WVA 65.1 0.0183 i.58 84.5 - 87.7 82.9 -89.3 Formula from U.S. Bureau of Census SE ^ X • CV X X XI = 68% Confidence Interval X2 = 95% Confidence interval SE = Standard error X = Penetration rate CV ^ Coefficient of variation Fdl/Winler 1987 iassist quarterly - 51 Table 1-C Tins tdble is from the U.S. Bureau of the Census Current Population Surve/ FERlENTAGE Uf- HCJUSLHQLDS WITH A TELLFHUNL E-Y HQUBcHOLDEK S AilTABLE I C ALL RACES WHITE bLAC^ HISF -iTJ Ur.i t Ava 1 Ur.lt Ave 1 Unit Ava 1 Ur. t. 8 MQNIH AVERAGE TOTAL H0USEHULD5 91 .7 9r> B 9r. 7 95 ,;j B'j. 4 B4 B Bl 1 16-LM YKS OLD 77.4 B- 6 80.0 85 6 57. 9 69 7 62 2^-54 YRS OLD 91 .B 93 B 97.5 95 1 B.J . 2 84 5 62 7 S5-5'9 YRS OLD 94.9 96 1 96.0 96 9 87.2 89 6 87 4 60-64 YRS OLD 95. 1 96 96.0 96 B B7.9 B9 8 B8 a 65-69 VRS OLD 95.9 96 7 96. 9 97 5 87.9 90 2 90 4 70-99 YRS OLD 95.5 96. 6 96. 1 97 -) 89. 1 91 3 Bl, 8 NOVEMBER 83 TOTAL HOUSEHOLDS YRS OLD YRS OLD YRS OLD 60-64 YRS OLD 65-69 YRS OLD 70-99 YRS OLD 16-24 25-54 55-59 91 . 76. 91. 95. 95. 93 7 93 1 95 . 78. B 83. 9 BCi 84 1 80 2 86. 2 49.9 68.2 64 93 7 93 4 95. 2 78.7 83.3 81 96 1 96 1 97.0 86. 3 88.5 89 96 4 96 4 97. 2 89.5 90.7 87 96 2 96 5 97.0 87. 2 89.0 90 96 5 96 97.0 90. 1 92.3 85 85. 89. 90. MARCH B4 TOTAL HOUSEHOLDS 91 .8 93 .6 93 . 3 94.9 80. 1 B4. 1 80 .7 83.6 16-24 YRS OLD 77 .6 84 . ':i 80 . 3 85.5 57.9 71.5 59 . 66. 2 25-54 YRS OLD 91 .9 93 . 7 93 . 5 95.0 BO. 4 84.0 83 -> 85. 6 55-59 YRS OLD 94 .9 95 .9 95 .7 96.6 87.6 B9.9 88 . 7 90.5 60-64 YRS OLD 94 2 95 . -^ 95 9 96.7 81.7 B5.0 67 4 89.6 65-69 YRS OLD 96 1 96 6 97 97.4 87.8 69. 3 85 8 87.8 70-99 YRS OLD 95 3 96 -^ 96 2 97. 1 87.2 88.8 82 = 85.5 JULY 84 TOTAL HOUSEHOLDS 91 6 93 8 93 2 95.0 80.5 85.3 81 1 64. o 16-24 YRS OLD 77 83 3 79 4 85.3 60.4 70.0 62 9 7.j.a 25-54 YRS OLD 91 7 93 8 93 4 95. 1 79.8 84.9 83. 1 65.6 55-59 YRS OLD 95 1 96 3 96 1 97. I 87.5 90.2 87. 4 91 . 4 60-64 YRS OLD 95 96 2 95 8 96.9 87.7 69.5 BB. 1 9.;. . 5 65-69 YRS OLD 96 4 97 1 97 3 97.9 B9.3 91.3 66. 7 90.6 70-99 YRS OLD 95 - 96 ^ 95. 9 96.9 89.6 93. 1 64. 68.5 NOVEMBER 84 TOTAL HOUSEHOLDS 91. 4 93. 6 93. 1 95.0 78.9 84.0 61. 1 64.5 16-24 YRS OLD 76. 1 83. 4 79. 85.4 56. 3 70.8 60. 8 70 a 25-54 YRS OLD 91. 4 93. 6 93. 3 95. 1 78.5 83.3 83. 1 85 B 55-59 YRS OLD 94. 9 96. 2 96. 3 97.5 84.7 87.4 85. 3 88 z 60-64 YRS OLD 95. 6 96. 5 96. 5 97.3 90. 3 92. 1 86. ij 67 2 65-69 YRS OLD 96. 96. 7 97. 1 97.6 86.7 69. 1 96. 2 96 2 70-99 YRS OLD 95. 3 96. 6 96. 1 97.2 88.0 90.7 87. 1 BB MARCH 85 TOTAL HOUSEHOLDS 16-24 YRS OLD 25-54 YRS OLD 55-59 YRS OLD 91 .6 77. 3 95 . 80. 1 64.4 81.2 EJ4. 1 84.8 59.6 70.0 62.4 67. 1 95.2 79.5 83.9 83.0 b:,.z 96.7 67.3 89. 1 86.5 B9. 1 Fall/Winter 1987 52 — iassist quarterly Table 2 •a OF HOUSEHOLDS WITH TELEPHONES IN THE C&P SERVED AREA OF VIRGINIA Year (1) Residence Access Lines-ln Service (June 30) (2) Bell Served Households (July 1) (3) Telephone Penetration (2)/(3)X100 (4) 1960 564.824 763.000 74.0 1961 588.783 782.000 75.3 1962 616.381 802,000 76.9 1953 640,932 823,000 77.9 1964 669.764 843,000 79.5 1965 702,113 853.000 81.4 1966 736.798 882,000 83.5 1957 776.020 904,000 85.8 195B 819.205 933,000 87.8 1969 860.518 975,000 88.3 1970 896.711 1,011,000 88.7 1971 933,814 1.027,000 90.9 1972, 972.258 1.058.000 91.0 ~ 1973 1.025.160 1.104.000 92.9 1974 1,069.475 1,134.000 94.3 1975 1.097.747 1,158.000 94.0 1976 1,132,060 1.202,000 94.2 1977 1,166,408 1.225,000 95.2 1978 1,212,158 1.263,000 96.0 1979 1,255,582 1,305,000 96.2 1980 1,288,870 1,355,000 • 95.1 1981 1,319.998 1.386,000 95,2 1982 1,343,961 1,416.000 94.9 Source: (1) Data for Residence Access Lines-In-Service are taken from the Virginia monthly No. 7 report, years 1960-80, and taken from the Virginia No. 2705 report for years 1981-82; (2) Household data is based on household counts from the Decennial Census 1960, 1970 and 1980; and are based on the Current Population Survey and P-25 Population Reports in other years. These reports originate at the U.S. Bureau of the Census. October 19£2 Fall/Winter 1987 lasit.U quarterly - 53 Table 3 T(le»io«t Penetration Dy Hoe and Incow, Viroinia, l96t 16 to 24 25 to 29 3« to 34 35 to 39 48 to 44 45 to 49 58 to 54 55 to 59 be to iA £5 to 69 78 to 74 75 to 79 riSriLKR HSHLDER HSHLDER HSHLDER HSHLDER HSHLDER HSHLDER HSHLDER HSHLDER HSHLDER mSKJES HS-1J1ES nousenoic Ircoae Less Tnan ilB.eee t.ii 8.79 e.83 e.92 e.93 e.9i i.ee e.94 e.95 e.98 e.7a (.75 e.67 8.92 8.9t 8.9i 8.98 8.9i 8.97 8.99 t.Si 1.69 8.77 8.68 8.94 8.97 8.98 8.96 8.96 1.08 8.99 1.88 8.92 8.79 8.68 8.94 8.97 8.99 8.99 i.ea 8.95 8.99 8.99 8.94 8.76 8.69 8.94 8.98 8.96 8.99 8.96 1.88 8.95 8.95 8.94 8.78 8.98 8.94 8.97 8.99 8.99 8.99 1.88 8.99 8.99 8,94 8.68 8.98 8.95 8.96 8.97 8.99 8.99 8.95 1.88 8.99 8.94 8.63 8.94. 8.97 8.96 8.99 8.99 8.99 8.96 1. 88 i.e« 8.95 8.66 8.95 8.96 8.96 8.99 8.99 l.»« 8.99 8.96 8.95 8.94 8.69 8.97 8.97 8.97 8.96 8.96 8.95 1.88 1.88 8.96 8.94 8.9e 8.97 8.96 8.99 8.99 8.99 1.88 1.88 8.97 8.99 8.93 8.91 ili.m - H9S9 8.97 IIMM - 19999 e.95 »2«M - £4999 8.99 $e5«ee - 29999 8.98 am - 34999 8.96 I35W!« - 39999 8.97 »4»W - 44999 1.88 »45eee - 49999 1.88 »5«e8e or (tore 8.97 Total 8.93 data S MC data bst generated from the 1980 Public Use Microdata Sample by the Pennsylvania tate Data C;nter, at the request of A.D. Schmldley, Bell Atlantic Fall/Winter 1987 54 — lassist quarterly Table 4 TELEPHONE PENETRATION BY AGE OF HOUSEHOLDER (%) (1980 CENSUS) AGE RATE AGE RATE 18-2A 83.8 25-29 92.0 30-34 94.5 35-39 95.3 AO-44 95.5 45-49 95.8 50-54 96.0 55-59 96.2 60-64 96.2 65-69 96.0 70-74 96.2 75* 95.6 «»is data was generated from the Public Ut.e Microdata Samples FUM3 for the Bell Atlantic served jurisdictions of Washington, D.C., Delaware, Maryland, New Jersy, Pennsylvania, Virginia and West Virginia, combined. Fall/Winter 1987 lassisl quarterly - 55 Figure 1 o o Q N O Z hi O LiJ 'IW/vL (Vol '1^^ data *». from the U. ?. Decennial Censuses of 1970 and 19H0 UJ h- < o I- < h- LU Z bJ D_ LlI z o I D_ LiJ _J LlJ B'^^I Q E-6B k\\\\\\\\\V\\\\^^\\\\^^m\^>^ ^ T56 £'56 cs k^^<\^^^-H^^^v^^vH\\; ^ fall/Winter 1987 56 - lassis! quarterly Figure 2 CO to (D (0 at CD r> fH > > o o z 2; 'TIHL data aB-from the Current Populc Survey CPS for the dates shown m <: E- :z; w 0. c-96^^mM^mm^mMS L§ ^^^^^^^ zn 6'ie| tl6 ig Fall/Winter 1987 iassisl quarterly - 57 Figure 2 Residence Access Lines vs. Households Bell Atlantic Region Millions 1^- 11- '^10- -iSSi 9- 8- 7- 6- Households ^^^^ 5- ^^^ Residence Access Lines Ik 4- 3- 'I I 1 1 1 1 1950 '55 '60 '65 '70 '75 '80 '84 '90 Fall/Winter 1987