THE USE OF PROGRAM MAILIT IN THE TEACHING OF MAIL SURVEY RESPONSE RATE PROBLEMS AND STRATEGIES Computer Simulation and Learning Theory, Volume 3, 1976 346 THE USE OF PROGRAM MAILIT IN THE TEACHING OF MAIL SURVEY RESPONSE RATE PROBLEMS AND STRATEGIES Alvin C. Burns, University of West Florida PROBLEM SETTING The nonresponse problem associated with mail questionnaire surveys is perhaps their significant shortcoming. Virtually every marketing textbook dwells on this characteristic. Wentz (1972), for example, states that the common return rate is 5-to-10 percent. Students find this attribute disconcerting, particularly when they are brought to realize that the resultant sample suffers from self-selection bias. Several texts suggest ways ~n which the response rate may be increased; the following are representative. Green and Tull (1975) mention that preliminary contacts, cover letters, and monetary inducements are positive influences. Zaltman and Burger (1975) note these three factors while including follow-up contacts, personal delivery and pickup, and self-addressed, stamped envelope. Luck, Wales and Taylor (197k) discuss the use of personalized cover letters and postagepaid return mail. Boyd and Westfall (1972) mention several factors which have been reported to affect the response rate. Whereas the marketing research student (or the practitioner, for that matter) acquires the knowledge that the response rate may be enhanced by any one or a combination of options, he does not acquire an appreciation for the degree of benefit each option entails, nor does he become acquainted with unique cost characteristics. In short, he is not provided with an analytical framework. The author believes that valuable learning is derived from a simulation exercise which allows a student to experiment with the several options and to examine the effects of each or any combination on survey accuracy, speed, and cost. Basically, the interrelationships among these three criteria are the central concern; however, such an approach has residual benefits, and may be applied to illustrate! teach several different concepts. A REVIEW OF RESEARCH FINDINGS IN THE MARKETING LITERATURE Preliminary investigation into the feasibility of developing a computer simulation approach to the teaching of these concepts took the form of a review of several articles dealing with mail questionnaire response rates (in marketing research). Four questions guided the literature search: (1) what is the shape of the returns curve over time; (2) what options have been researched; (3) what is the effect of the various options on mail survey response rate; and (1+) what degree of interdependence exists among the effects of the various options? The review was thorough but not exhaustive. Its intent was to uncover common themes and findings. Computer Simulation and Learning Theory, Volume 3, 1976 347 Figure 1 contains a summary of the results of the literature search which revealed that the effects of at least six options have been reported. These are: an advance letter or postcard informing the prospective respondent of the coming questionnaire; a monetary incentive either included in the letter or promised in return for completing the questionnaire; a follow-up postcard or letter to non-respondents; a personalized cover letter; a stamped return envelope instead of the customary business reply; and an offer of a summary of the survey results. At this point in the development of the simulation, the presence of several difficulties became apparent. One major problem was the fact that the time periods spanned by the several surveys reported ranged considerably. As can be seen in the chart, the shortest period reported was 2 weeks while the longest was 6 weeks. Furthermore, three reports do not indicate specific time periods. Another problem emerged in assessing the scope of each survey. Generally, authors do not report the geographic region involved nor are they equally precise in their descriptions of the sample unit. Where the sample unit is described, it is evident that some studies involved special and nonrepresentative segments. Stafford (1966), for instance, used students in his study, while Keane (1963) surveyed panel members. Other troublesome considerations arose in comparing specific options. Several years are spanned by the studies and inflation undoubtedly is a consideration in the comparison of the effects of monetary inducements; advance letters were sent at varying days tn advance; follow-up letters were mailed at different time intervals; and the concept of personalizing was not described in sufficient detail. As a consequence of these problems, it was decided to concentrate on three articles which provide appreciable detail and/or analysis beyond the descriptive level. Cox (1966) offers a detailed account of the cumulative response rate over a twenty-eight day period. Cox, Anderson, and Fulcher (l97L~) report cumulative rates at various time intervals in a comparison of four alternative strategies. They report no significant interaction between personalization and follow-up postcards. They also conceptualize a decision model treating costs, response rates, and various survey objectives. Wiseman (1973) compares four options: Offer of survey results, 10¢ monetary incentive, follow-up postcard, and stamped return envelope. He concludes that the variables operate independently and that interaction effects are relatively unimportant. DESCRIPTION OF THE COMPUTER PROGRAM Basic Logic Basically, the computer program allows the student to manipulate several mail survey research decision variables and to compare the results of various strategies. He may select any one or a combination of the six options mentioned above (or none). Certain Computer Simulation and Learning Theory, Volume 3, 1976 348 Computer Simulation and Learning Theory, Volume 3, 1976 349 options require additional decisions such as the size of a monetary inducement or the time period before a follow-up letter is mailed. Additionally, he must specify the number of initial mailouts and the number of days involved in the survey (cutoff day). The program simulates a binomial sampling distribution ~ associated with each option. The results are summed (using the independence assumption) and a cumulative return factor associated with the cutoff day is applied. Common costs and direct costs -associated with each option are stored internally and computed in -the form of a total direct cost for the survey. Output takes the form of number of responses, percent response, total direct cost, cost per response, and particulars associated with specific optfons. The student may submit a number of survey options cards in a single run. Each survey is simulated five times to demonstrate variability. The average response rate and number of days for each survey are presented graphically at the end of the complete run. Figure 2 is a sample of the output. The Response Function A pattern of cumulative responses over time is described in detail by Cox (1966) who points out that the response curve for consumer mail surveys is s-shaped. It was decided to use the equation for a Gompertz curve of the form: Where: rt = the response rate at time t c = the growth at maturity (the upper asymptote of 100 percent) a = the proportion of initial growth R = the rate of growth Comparisons between the percentages generated by this equation and reported findings found that the values of the parameters for a reasonable fit should be .0004 and a and .775 for R. Thus, .0004 ≤ r ≤ 1.00. Perusal of the reported effects of various options suggested that certain factors seemed to accelerate the response function. While findings are scanty, logical analysis implies that an advance letter and a sufficient monetary inducement affect responses in this manner. Consequently, an accelerated response curve with the values of .0003 for a and .750 for R is used in the simulation if either or both of these options is chosen by the student. Computer Simulation and Learning Theory, Volume 3, 1976 350 Computer Simulation and Learning Theory, Volume 3, 1976 351 Treatment of the Effects of the Options Each option has a unique effect on the response rate. Disregarding random error generated by the simulation, the general form of the net response equation is: Where: p = the response percentage (at time t) B = a simulated base response assuming no options Ri= the simulated response increment associated with option i rt = the response rate at time t (cutoff day) As one can see, the model assumed makes use of Wiseman’s (1973) finding of independence of the main effects. Each option has a unique incremental response rate associated with it. Although comparison across studies was difficult due to the aforementioned problems an endeavor was made to base the response percentages on reported empirical results. A control statement does not allow the final percentage of response to exceed 95 percent. Treatment of Direct Costs Each option has a unique direct cost equation associated with it. The unit costs are communicated to the student and stored within the program. The total direct cost of each simulated survey is computed as a function of direct unit preparation and mailing costs common to all options, additional direct unit costs associated with chosen options, and direct unit costs associated with returned questionnaires under chosen options. Common direct costs associated with mailing out the survey include costs such as paper, duplication, envelopes, stuffing, sealing and postage. Direct unit costs associated with mailing out under various options are exemplified by labor and postage for stamped return envelopes; special typing for personalized cover letters; duplicating, and stuffing, or mailing costs for advance letters. Cost associated with returns under various options include monetary incentives and business reply postage. USE OF THE PROGRAM As mentioned earlier, the program may be, used in a number of different ways; however, its primary purpose is to develop a conceptual framework in students for evaluating mail survey strategies. Students are urged to experiment with the options to gain a feel for the trade-off between the accuracy of information secured (number Computer Simulation and Learning Theory, Volume 3, 1976 352 of returns), the time period, and the cost of the survey. The author has adopted a three-stage approach in using the simulation which systematically familiarizes students with mail survey strategies, develops the decision-making framework, and evaluates the degree of ‘earning. The objectives of each phase and an assessment of student reactions and learned concepts follows. Familiarization Phase The initial phase of the use of the program requires that the student become familiar with the distinguishing characteristics of mail surveys. Lecture and discussion concentrate on the low response problem, the time lag factor, and cost relative to other survey modes. Discussion then turns to methods of improving response rates in mail surveys, whereupon students are introduced to the various options and the program control card structure. The initial assignment requires students to develop an understanding of the general shape of the response rate curve as well as for its characteristics under various options. The author has come to believe that the best teaching strategy at this stage in the use of the simulation is to assign individual students or student teams the task of investigating the effects of specific options or combinations of options. Early experiences with the simulation revealed that an unstructured assignment overwhelmed the average student; there were simply too many variaHes ~nd combinations of variables with which to cope. Students tended to concentrate on comparisons between options at a single point in time rather than to investigate return rates over time. With individual assignments, students have exhibited greater cooperation and understanding. The graphical presentation provided at the end of the output is advantageous, and the more enterprising students put it to good use. Furthermore, the general characteristics of the response curve come forth as a common factor during group presentations. Thus, students soon discover the growth shape of the curve. Class discussion is directed to differences between response rates to various options and students have been quick to speculate on the logical connections between specific: options and response characteristics. Sensitivity Phase Upon becoming familiar with the basic aspects of the response function, students’ attention is directed toward the trade-off characteristics of mail surveys, specifically the interrelationships between survey direct costs, number of responses, and number of days. The interrelationships are not readily seen as a consequence of the discontinuities in the cost functions and nonlinearity of the response rate. To complicate matters, certain costs are themselves dependent on the response rate (the return postage cost in the case of business reply postage, for example). Consequently, the author Computer Simulation and Learning Theory, Volume 3, 1976 353 has assigned a between-phase transitional role to the cost per-response value. Students are encouraged to perform comparative analysis by holding one factor constant and comparing the change in the cost-per-response figure over time. At this point in the analysis it becomes apparent to the student that diminishing returns exists in the form of the downward sloping curve which approaches its minimum somewhere around 30 days in most cases. Thus an outer bound on the number of survey days is established. Conceptualizing the cost-time-number of response trade-offs is more difficult for students as it requires three-dimensional analysis. Due to the interrelationships of the various costs and the differential response rates, certain strategies are more advantageous in terms of minimizing total cost to achieve a desired minimum number of returns. Alternatively it becomes apparent that the minimum number of returns may be gained in fewer days at higher cost. Ideally, the interrelationships should be conceptualized as “strategy surfaces” in three-dimensional space defined by time, total direct cost, and number of returns. The final aspect of the conceptualization requires the imposition of constraints on the survey. Students are made aware that mail questionnaire surveys operate under constraints of the general form: “Obtain a minimum of N returns at a maximum cost of TC within a maximum of t days.” Admittedly, this is a simplistic specification of the objectives and constraints; however, it allows students to apply them as bounds in their analyses. In actuality, of course, the bounds define a feasibility region in three-dimensional space. The student must choose the best strategy by selecting from those that lie within the enclosed area. The presentation is not readily understood by the majority of students initially; however, the method of analysis eventually becomes more clear with discussion and illustrative runs. At the very least, students come to realize that unidimensional analysis is inappropriate. While they do not assimilate the exact shapes of the strategy surfaces, the logic of the alternative evaluation procedure is acquired. “Actual Survey” Phase The final phase of the use of the program is an individual assignment in the form of a scenario case study. The following example is representative: “The mail survey with which you are presently concerned entails a random sample of homeowners in a large regional area. The region is almost 60,000 square miles in area with a population of about 7,000,000 people, the questions on the questionnaire are relatively straightforward although there are some which solicit ‘confidential’ information. The respondents are assured that the confidentiality of their responses will be respected. Management desires this Computer Simulation and Learning Theory, Volume 3, 1976 354 information as soon as possible but no more than 20 days after mailout. Anticipated statistical analysis requirements require that the final sample size be no less than 1+00. Your target budget is $500, but you may spend up to 20% more if you can guarantee sufficient returns in less than 20 days limit.” Each student is required to present his mail survey plan by enumerating the number of mailouts, the specific options, a time table, and the estimated final cost in the form of a detailed budget. Students are not allowed to experiment with the program at this stage, and the final report includes one decision card which is run by the Instructor. Evaluation places emphasis on the rationale of the student’s choices rather than the outcome of his decisions. Specifically, students must demonstrate that they can apply learned concepts to this situation. CONCLUSIONS As a facilitating device, the described mail survey computer simulation represents unique advantages over the cursory treatment found ~n marketing research textbooks. In particular, it familiarizes students with options and tactics often employed to increase the response rate. Through experimentation and comparison, students quickly envision the response curve and develop an appreciation for the relative effects and special considerations of various options. The author has related a three-phase method of using the simulation to teach the interrelationships and trade-offs between cost, time, and the number of responses. The simulation plays a vital role in generating data to illustrate the strategy surfaces and decision constraints. The author believes that the simulation is an invaluable aid in the teaching of these theoretical concepts. REFERENCES 1.. Boyd, Harper W. Jr., and Ralph Westfall. Marketing Research: Text and Cases, Illinois: Richard D. Irwin, Inc., 1972. 2. Cox, Eli III, Thomas W. Anderson Jr., and David G. Fulcher. “Reappraising Mail Survey Response Rates,” Journal of Marketing Research, Vol. 10 (November 19714), 1413-17. 3. Cox, William E. ‘‘Response Patterns to Mail Surveys,” Journal of Marketing Research 2 (November 1966), 392-7. 4. Ford, Neil P1. “Advance Letter in Mail Surveys,” Journal of Marketing Research 3 (May 1967), 207-4. 5. Francel, E. G. “Mail Administered Questionnaires: A success Story,” Journal of Marketing Research 2 (February 1966), 89-92. 6. Frankel, L. R. “How Incentives and Subsamples Affect the Precision of Mail Surveys,” Journal of Advertising Research 1 (September 1960), 1-5. Computer Simulation and Learning Theory, Volume 3, 1976 355 7. Green, Paul E. and Donald S. Tull. Research for Marketing Decisions, New Jersey: Prentice-Hall, Inc., 1975. 8. Heaton, E. E. ‘‘Increasing Mail Questionnaire Returns With a Preliminary Letter,” Journal of Advertising Research 5 (December 1965), 36-9. 9. Keane, John G. “Low Cost, High Return Mail Surveys,” Journal of Advertising Research 3 (September 1963), 28-32. 10. Kephart, William and Marvin Bressher. “Increasing the Response to Mail Questionnaire: A Research Study,” Public Opinion Quarterly (Summer 1958), 123-32. 11. Luck, David J., Hugh G. Wales, and Donald A. Taylor. Marketing Research, New Jersey: Prentice-Hall, Inc., 1974. 12. Roscoe, A. Marvin, Dorothy Lang, and Jagdish N. Sheth. “Follow-Up Methods, Questionnaire Length, and Market Differences in Mail Surveys,” Journal of Marketing, Vol. 39, No. 2 (April 1973), 20-7. 13. Simon, Ramond. “Response to Personal and Form Letters in Mail Surveys,” Journal of Advertising Research 7 (March 1967), 28-30. 14. Stafford, James E. “Influence of Preliminary Contact on Mail Returns,” Journal of Marketing Research 3 (November 1966), 141011. 15. Watson, John J. “Improving the Response Rate in Mail Research,” Journal of Advertising Research 5 (June 1965), 48-50. 16. Wentz, Walter B. Marketing Research: Management and Methods, New York: Harper and Row Publishers, 1972. 17. Wiseman, Fredrick. “Factor Interaction Effects in Mail Survey Response Rates,” Journal of Marketing Research 10 (August 1973), 330-3. 18. Wotruba, Thomas R. “Monetary Inducements and Mail Questionnaire Response,” Journal of Marketing Research 3 (November 1966), 398-400. 19. Zaltman, Gerald and Philip C. Burger. Marketing Research: Fundamental and Dynamics, Illinois: The Dryden Press, 1975.