Attrition and the Longitudinal Surveys of Labor Force Behavior: Avoidance, Control, and Correction by Dr. Patricia Rhoton Data Archivist National Longitudinal Surveys Center for Human Resource Research Ohio State University Since 1966 the Center for Human Resource Research has been analyzing the longitudinal surveys conducted by the Census Bureau for the Department of Labor. The main purpose of these surveys is to study the labor force activity of differ- ent population groups. The original groups included men who were 45-59 years old in 1966, women who were 30-44 years old in 1967, men who were 14-24 years old in 1966 and women who were 14-24 years old in 1968. In 1979 a new survey, conducted by the National Opinion Research Center in Chicago, was added for young men and women who were 14-21 in that year. Each of the five surveys is designed to collect information on all phases of the respondent's labor force activity and on other characteristics such as educational attainment, health, family composi- tion, and financial status that are known to be related to such activity. The original plan in 1965 was to interview the same respondents each year for a period of five years. Because of the usefulness of the data and the relatively small sample attrition, the decision was made at the end of the first five year period to continue for another five years. The interview pattern was changed at this time from a face to face yearly interview to a 2-2-1 pattern. Each respon- dent was contacted by phone e^ery two years, then again in person one year after the second phone interview. This pattern was used again in the third five-year extension obtained in 1976 and during the fourth five-year extension, obtained in December, 1982. At the time of this most recent extension a study was done look- ing specifically at attrition within the different cohorts. Longitudinal studies in general have several advantages over the more frequent cross-sectional studies. While longitudinal studies are very expensive, the data are collected in great detail over time with the respondent reporting events and attitudes as they occur rather than retrospectively. Gathering the data in this way also enables the researcher to go beyond issues of correlations to address the more urgent issues of causality. The main advantage of a longitudinal survey, following the same set of respondents year after year, creates its two major problems, however. The first is the difficulty of locating the respondents for the subsequent interview and the second is maintaining respondent cooperation over repeated interviews. Attrition in the NLS Table 1 shows the number and percentage of respondents for all interviews up to and including the 1983 questionnaire. The base year shows only those respon- dents who were interviewed that first year. Between the original screening and the first interview, part of the eligible respondents were lost: 9.0 percent for the Older Men, 5.5 percent for the Older Women, 8.3 percent for the Young Men, 5.8 percent for the Young Women, and 11.5 percent for the New Youth. While Table 1 shows the distribution of noninterviews between and among the five cohorts. Tables 2-5 show interview/noninterview status for the four older cohorts by reason for noninterview. While there are shifts in the distribution of a particular noninterview reason during a particular year, a consistency appears in the rate of attrition within each of the four older panels. The method of interview, whether face to face or by telephone, does not seem to affect the attrition rate. Some of these losses to the sample are unavoidable. For the Mature Men (Table 2), for example, an increasing percentage of the sample losses have been due to respondent's death. The Mature Women's Survey (Table 3) has the second highest retention rate among the four older cohorts. This high rate is probably due to the fact that this group is very stable and has low geographic mobility. The Young Men's Cohort has the lowest rate of retention and has been the test case for new attempts to stop the gradual decline in sample size. A variety of factors account for the difficulty in locating these respondents: completion of school, acquisition of new jobs, formation of families and movement in and out of the military services. The higher rates of attrition in the earlier years were attributed to the influx into military since the sample was drawn and the initial interviewing done during the Vietnam War. However, rates remained high even as the respondents returned from the military. The Young Women's Cohort, which is similar to the Young Men's with respect to completion of school, acquisition of new jobs and formations of families, had the added challenge of name changes accompanying changes in marital status, yet the overall response rate has remained high. The New Youth Cohort has benefited greatly from the lessons taught by experi- ence with the older four cohorts. In 1983, the response rate for this group was 96.3 percent. Comparing this cohort with the Young Women in the first five years, the cohort that had the best retention rates of the older cohorts, shows that a difference in procedures and techniques can decrease attrition on a substantial basis. Not only does NORC have a higher overall interview rate, the organization seems to be better at retrieving respondents. In 1982, of the original 1979 sample, 96.0 percent were interviewed. Some of these had not been interviewed in previous years: 2.2 percent in 1980, 1.1 percent in 1981, and 0.5 percent in 1980 or 1981. Only 165 respondents (one percent) of the original sample has had only one interview after four rounds of the survey. In 1983, the number of respondents who had had only one interview dropped to 115. Over 11 thousand (90.7) of the respondents were interviewed every year, and 5.5 percent had com- pleted four out of the five interviews. ^2 01 t) Is 01 o> I t-l O 1-1 v 0) I cc 0> o *J w o 00 •— »H .-( .1 00 £•:: SJ^ NJf: 0^ 2^ 52- y.^ aj # o c — t > o CO m c C L. 01 ^ «- o (gX s £^85 »8S as s 2^ I o 00 tn I c^ CO tn es) CD I tn ^r I I I I I t- OS c- oo c^ to • . u^ I I I I I I I in I I I 1 I I I CD I CM 00 o I CD e^ I CD e^ m 11 I CO I-" I ^ C^ I O^ I o ^^ c^CDCDCD*0__,__osososososososososososos 0) W i) CI V 01 ai T3 0) x: OD c =/ II 1^ a*' ffl-J^ •0 2R ?'^ « c s l-H ^ O ^ O W «-! ffi C- O Ol O O .-I ^ ^ ;D o r- o ^ CD I T-i 0> I 0» t> I ^ t- 00 CJ CO ^ *rt CD t— oo t- b- t- t- t- t- t- O^ O O^ O) 93 O) O^ ^ CO CO CO 00 oo f 2 to &-Z I CD ^ ^' t- irt in oj ^ o* ift 0> 0> 0> 00 00 « 2J? Oi I Oi I I I OS -^ t- ift <*5 c>) ec c^ I O) t« 00 O 1-« f-H I C4 x: t. 03 c SB 9* ?* a> - «- o a*' e.jp sz be — _ ._ t- :|^ oo ^-> tn » O£ bo o> « ^ be C » «: o « C'l S 3 rH « # x: he ^ ?i* CO w OS Qj a> o OJ as c -^ — 0; o a> oo CO e^ ift c^ ^ Oi es) ,-, .-H >oioo ^<-^o^ oooico^t-r-tci' > (C ^ f^ C^ C^ t- C>3 C^ rt 1-H I/? Oi CC to r5 oi CD 00 i-i oi c-:> <* O"' cc ^ o> ^ ^-i Ca CO ^ O CD ^ V C> '"f C^IOOO OCOCO e^OlU^OCDOOOO*-! cD^co r5t-«m OiOOcoiiTs^coc- irt*rs«— • oic>ao5 »-Hoot/>i/5POoooi'»'CD ^•OOCD OC-OO C^^COO^COtTOI cocorn t~-<-iC- •-i,-OJin. •-i CJ CV)m wi t- C^ CD ^ CO .-< »ft O TT to O^C^O »—'O^'—"Gi^-'QOOOOl^ 1 CO N l> m o ^ o > r- ,- Qj O '^' — ^ 03 - St >.— •- >. = bt 01 gOD-Oi-CC oo.£»-wi-c__ C5 o c^ Irt »-' • be O c c — o — — ._ t. ffl »- 3 »- I*- oie-oi— E c3 C (0 CC C L. .o 1. I, — O) 3 E- H (1- w n. beo — V 01 j: tt. "^ tie c <£> 0) w en S = '- -j(? »—•ooiftin^t- int-»—ICO «-<^0^01»-H I 0)01000) oe CM CO in —1 CO — -v CO CO o '- o t- in c* 00 CO O) CO ' (O in o in r- t- n en CD e>) CO ^ ^ 00 ^ o t» o t- 00 CO be C (O :«t: O o>cooOf-iai^< > (Dooosfc o> o> *T O) ^ r— 2 2*5 CO C~- 0> CO 9:> I «-< «r 05 ^H in oo f-i ^ rH 0> OO rH CO O to <-" 00 CO O 00 ^ c- t- to V^OOtftinO I cocotooo mcj-NCNioJo e^jo^oi UO 0> CO irt c^ t-H o> in « t-« E.J^ »-it-co^CT>co I caot-^ i-ic^inifi'J't- int-*-i^ in in oi o C4 t- O CO »-* t^ o 00 <-< ^H m t^ m coin^i^HOOm »—iCo «-h 00 —• Oi 00 c«i -N to ^r ^ ^ ^^ in c^ ^r t> CM »-t in CO CM to I o O) o> to t- to O ^ •** ^ OS 00 "C to a> O o) ^ o) in »-" ^ o to ^ 1-< *» ^ 05 to «-' 1 CM 0> t- ^ in t- r- -^ o o 0> 1-1 CM oi in •-• ca I ^ 0) a> a> J3 > Q be 0) — c to 00 t- ^ o r- «-H o 'S' in CM oi CO in ^ <-" t^ ^ O) OO o o> ,-. ^ CO ^^ 1-1 "U to a> O o> Oi CO CM ^^ to^ ,— , ,_ 1—I in e-s C o; . 01 >— 1- « ' ^ ffl CJ 3 ' C/2 U O > — - c _ »- £ o t- — OJ . O 0) ffi O 03 X cri b b. ^ - < c - a> CT> I I (— oi ' o O ( CB 3 - o o o oO - - CM 10 £=# 2= 3 ^ w *- 5 ^ A> « 05 *— ' S =>-• ^ s< ^ ^ — o — c — •- 1- c !? ^ *-• oo CO CM in CD o »-l to CS ^ CM CM CM Ift i-» o in 1-1 ^ o CD oi 00 c^ in «D o a> ^ o to CO inO CM O CM rt 00O CO t- C^ CM i-« I f-H — CM *T Ol «-i CM ca CO 00 t- TT c- CO • oi in CO CO CO 00 00 ^ C* CO CM CM CO i-H in CM CO in in 00 c^ r- »-< CO CM ^H CM m ^ C* CM CO ^T •-K O O ^ CO o •-«»-" CO CO in t- C^ 0> 00 OO C^ 00 •— ' CM CM CO 00 C- m c^ 1-1 ^ t- a. CM to CM ^J- CM 1-1 CC S - .— .— o "C *- Oi Ol - _ C 05 I o o o c — cri o o o * 03 - O O O O Cfi 3 CM O - - CM Oj D- I * o m bcw r^ eo ^ ^ + 11 The Impact of Attrition in Representativeness This gradual decline in sample size over time becomes very important if it results in a biased sample. While each cohort was checked at the end of the first five year series of interviews and smaller checks were made in the context of reports on occupational distribution, educational attainment, age distributions and marital status with nationally represented published data, no one looked at all the cohorts systematically until 1982. At this point the issue of representa- tiveness had to be addressed as part of the proposal to extend the cohorts for another five years. Such a study could be done in essentially two ways. First, the remaining sample could be compared against some outside group, such as one from the Decennial Census or the Current Population Survey. Comparison with an outside sample was difficult given time contstraints and the fact that the Decennial Census data were not yet ready for release. While the CPS data were available, differences between the CPS and each of the four older cohorts had already been documented in the first year. The second alternative was to compare the charac- teristics of all respondents interviewed in the initial year to see how much difference, if any, there actually was. Each cohort was checked for differences in the age distributions, educational attainment levels, employment status, industry and occupation distributions, educational attainment levels, employment status, industry and occupation distributions, marital status, SMSA status, annual income distribution and wages and salary distribution. The Young Men and Young Women were also checked on enrollment status. A separate evaluation was done by race for each of the four cohorts. Table 6 is an example of the type of table constructed for each group. The ten year sam- ple was weighted using two methods: the entry level weight and a ten year weight, which includes successive adjustments for each year's noninterview. For all the cohorts except the Young Men the relevant comparison was between the entry year weighted figures and the ten year sample using the ten year weight. In the Young Men's Cohort, the 1966 sample using the 1966 weights was compared to the 1976 sample using the 1966 weights because the 1976 weight had been adjusted to include individuals formerly in the military. Since young men already in the military had been deliberately excluded from the Young Men's sample, using the 1976 weight could create apparent differences where none existed. For this group alone, it was more appropriate to use the 1966 weight. Table 7 summarizes the distribution of differences by cohort and shows that for most of the characteristics the differences between the two samples were less than two percentage points. After the differences were identified, statistical tests of significance were computed for each of the comparisons. Table 8 shows the number of statistically significant differences at various levels for each cohort by race. While the number of differences were higher than would be expected by chance, several were based upon small sample cases in the initial year and characteristics with only two values. In the latter cases a statis- tically significant result in one category means the other category will also be statistically different. After reviewing the entire set of tables it was clear that the noninterviews had not seriously distorted the representativeness of the sample. Given this finding and the ability to change the weights to eliminate any potential bias, the decision was made to continue all four surveys for another five years. 12 Table 7 Nimber and Percentatge of Differences by Panel Absolute differences (%) Panel 0^2 2^3 3+ Total Mature men Black 34 (73.9 8 (17.4) 4 (8.7) 46 (100.0) White 43 (95.6) 2 (4.4) 45 (100.0) Mature wcmen Black 42 (93.3) 3 (6.7) 45 (100.0) White 45 (100.0) 45 (100.0) Young men Black 30 (73.2) 5 (12.2) 6 (14.6) 41 (100.0) White 43 (97.7) 1 (2.3) 44 (100.0) Young wcmen Black 33 (82.5) 6 (15.0) 1 (2.5) 40 (100.0) White 40 (95.2) 2 (4.8) 42 (100.0) Table 8 Nunber and Percentage of Statistically Significant Differences by Panel Level of significance Panel 1% 2% 3% Mature men Black 4 (9.1) 7 (15.9) 12 (27.3) White 4 (9.1) 7 (15.9) 14 (31.8) Mature wcmen Black 2 (4.5) 2 (4.5) 3 (6.8) White 1 (2.3) 4 (9.3) 5 (11.6) Young Men Black 1 (2.6) 4 (10.3) 6 (15.4) White 2 (4.7) 4 (9.3) 6 (14.0) Young wcmen - Black 1 (2.6) 3 (7.9) 4 (10.5) White 1 (2.6) 2 (5.1) 2 (S.l) 13 It is unclear, however, how further erosion of the samples will affect this representativeness. Concern with this issue, together with the high noninterview rates that NORC was having with the New Youth sample, led to an evaluation of the rules that had been established in the original five year period and an attempt to see if it was possible to retrieve some of the noninterview cases. Retrieving Former Noninterview Cases Since the Young Men panel had lost the most respondents, it was the target for the first attempt at retrieval. Respondents from the 1975, 1976, 1978 and 1980 survey years who normally would not have been included in the workload (i.e., attempted to be contacted) because of their noninterview status for those years (refused, unable to contact, institutionalized, moved outside the U.S.) were sorted and a sample of 279 respondents selected. Several changes occurred in procedures for contacting these special respon- dents. No restrictions were placed on the number of telephone calls, mileage or time spent in locating and retrieving these respondents. Each interviewing packet included the respondent's most recently completed interview and household record card, as well as the most recent questionnaire and all record cards for any other household members participating in any of the other cohorts. In addi- tion, an expanded list of methods of locating respondents was included. As a result of these additional steps, 104 (37.3 percent) respondents were interviewed. These have been identified and will be checked as soon as the data tapes are available from the Census Bureau to see if they differ in any way from the rest of the respondents. If these respondents remain in the sample for the next round of interviews in the last part of 1983, a concerted effort may be made to use these procedures during the regular interviews and in similar attempts to retrieve noninterviews in the other three cohorts. Differences Between Census and NORC One of the biggest differences between Census and NORC is the amount of locating information obtained from the respondent. NORC gets more information and asks for individuals with specific relationships depending upon the respon- dent's circumstances. The interviewer starts by asking the name, relationship, address, and phone number of the person most likely to know where the respondent is. If the respondent is living in a dormitory, fraternity, sorority, hospital or other temporary situation, the interviewer is instructed to obtain the name and relationship of a householder at a permanent home address. If the respondent is married and living apart from a spouse, the spouse's address and telephone number are requested. If the respondent is not living with a parent and has not provided a parent's name, this information is obtained, including whether or not the parents live together. The name of another relative with whom the respondent is in contact and the names of friends and places where the respondent goes when not spending spare time at home are also obtained. Respondents are also asked nicknames, maiden names if they are married women, and whether or not they expect to move in the next 12 months. This extensive list gives the NORC interviewer a real advantage when contact- ing someone on the list, since the ability to mention the respondent's parents, relatives, friends, hangouts or nicknames demonstrates that the interviewer knows the respondent to some degree and may make the reference more willing to give out 14 information about the respondent. Another major advantage that the NORC inter- viewer has over the Census interviewer is the existence of a centralized locating shop in Chicago. The person working at the locating shop has access to all pre- vious questionnaires, original copies of locator documents and information about the respondent's brothers and sisters. Working with this additional data, the respondent can usually be located by phone and reassigned to the same or another interviewer. The Census interviewer starts out with less information to locate the respondent. S/he has a questionnaire with a label indicating the respondent's name and most recent home address. In addition, there is a household record card for each respondent that contains the telephone numbers, all the addresses where the respondent has lived since the survey began, the names of all persons who have lived with the respondent, and the names, addresses and telephone numbers of all persons who have lived with the respondent, and the names, addresses and telephone numbers of only two persons who will always know where s/he can be reached. Besides the more extensive locating supplement that NORC builds in the inter- view, several other differences appear. For the New Youth cohort, each respon- dent is paid $10.00 for a completed interview, since many researchers believe that even a small amount of money helps in obtaining cooperation, especially among younger respondents. The New Youth respondents also had the opportunity to take a series of tests that the Department of Defense needed to evaluate tests given to individuals in the military. For these tests, which take several hours, the respondents were paid $50.00. When the four older cohorts were first inter- viewed, paying respondents was not as well accepted. Now there are fears that starting this procedure with the older cohorts would cause concern on the part of the respondents. Another procedural difference is that in the New Youth cohort, the respondents are told up front that they will be interviewed each year for the next several years and are therefore aware that they will be contacted about the same time next year. The Census interviewers are told only that they may be conducting additional surveys, and should not tell the respondents that this is the last time s/he will be interviewed. The lack of an answer to give the respondent, in addition to the 2-2-1 pattern, probably leaves the respondent without a sense of when or if s/he will be contacted again. While this ambiguity may not have an impact on their cooperation in the survey, the NORC approach leaves the respon- dent with a greater feeling of certainty about the interviewing schedule. Revising the Rule for Dropping Respondents After the first year respondents from the four older cohorts who refused to participate or had died were dropped from the Census sample. Those who were not interviewed for any reason for two consecutive years were also dropped. The only exception was made for the Young Men's sample with the respondents who were in the Armed Forces. Since the sample was to represent the national civilian, non- institutionalized population, the young men were not interviewed while they were in the Armed Forces but they were retained in the sample and picked up the first interview after they left the services. However, NORC's success in retrieving respondents even after they refused and the success in the Young Men retrieval effort resulted in a change in these rules. Currently no respondent will be dropped except those who have died. NORC goes back each year and attempts to interview all living respondents. 15 Maintaining Respondent Cooperation While both Census and NORC send out advance letters about the entire survey stressing the importance of the respondents' cooperation, NORC also sends out a newsletter that tells respondents in a very "chatty" format about some general results of the previous survey. The Census Bureau had a short, formal fact sheet that goes out with the cover letter, but the interviewers reported that the respondents did not feel it was very useful. For the 1982 Young Women's Survey, a more extensive description of the surveys and a list of the research results from the survey were sent to any respondent who filled out a postcard requesting additional information. Over one-third of the respondents interviewed in that wave mailed in the postcard. A variable will be created identifying these respondents and if reception of the handbook increases the response rate for the next round, the handbook will be offered to respondents in the other three cohorts. Conclusions The New Youth Survey at this time has considerably better response rates than any of the four older cohorts. A great part of this success can be traced to solving problems that developed over time in the older four cohorts. While the necessity of keeping the same measures over time prevented change in the older four cohorts, these problems were corrected in the first wave of the New Youth. Questions that the respondents or the interviewer had difficulty with in the older four cohorts were altered so that there was no confusion from the very beginning. Perhaps most important, given the highly mobile nature of this age group, much more detail was obtained on individuals who would always know where the respondent was. In addition, more information about the survey was given to the respondent before, during and after each interview. All of these factors combined have resulted in a response rate that is very good for any survey and exceptional for a longitudinal survey in its fifth year. 16