A COMPUTER SIMULATION APPROACH TO THE TEACHING OF MAIL SURVEY STRATEGY ALTERNATIVES AND CHOICE Computer Simulation and Learning Theory, Volume 3, 1976 297 A COMPUTER SIMULATION APPROACH TO THE TEACHING OF MAIL SURVEY STRATEGY ALTERNATIVES AND CHOICE Alvin C. Burns, University of West Florida PROBLEM SETTING It is customary for marketing research textbooks to describe three modes of survey data collection: personal interview, telephone interview, or mail questionnaire. These three methods are often compared across considerations assumed to be for most in the researcher’s mind. Boyd and Westfall (1972), for example, discuss differences on the bases of flexibility, amount of information obtained, accuracy of Information obtained, sampling considerations, speed, cost, and quality control. They conclude with the recommendation that accuracy of information should be the primary criterion, and speed and cost should determine the choice if the researcher believes that sufficient accuracy will result from two or more methods. Other texts contain similar discussions of relative advantages and disadvantages. It has been the author’s experience that students frequently level and sharpen this information into the conclusions that personal interviews are costly, telephone interviews are inflexible and mail surveys do not obtain sufficient responses. The nonresponse problem associated with mail questionnaires is perhaps the most significant shortcoming. Virtually every marketing textbook dwells on this characteristic. Wentz (1972), for example, states that the common return rate is 5-to-10 percent. Students find this attribute disconcerting, particularly when they are brought to realize that the resultant sample suffers from self-selection bias. Several texts suggest ways in which the response rate may be increased; the following are representative. Green and Tull (1975) mention that preliminary contacts, cover letters, and monetary inducements are Computer Simulation and Learning Theory, Volume 3, 1976 298 positive influences. Zaltman and Burger (1975) note these three factors while including follow-up contacts, personal delivery and pickup, and self-addressed, stamped envelopes. Luck, Wales and Taylor (1974) discuss the use of personalized cover letters and postage paid return mail. Boyd and Westfall (1972) mention several factors which have been reported to affect the response rate. Whereas the marketing research student (or the practitioner, for that matter) acquires the knowledge that the response rate may be enhanced by any one or a combination of options, he does not acquire an appreciation for the degree of benefit each option entails, nor does he become acquainted with unique cost characteristics. In short, he is not provided with an analytical framework. The author believes that valuable learning is derived from a simulation exercise which allows a student to experiment with the several options and to examine the effects of each or any combination on survey accuracy, speed, and cost. Basically, the interrelationships among these three criterias are the central concern; however, such an approach has residual benefits, and may be applied to illustrate/teach several different concepts. A REVIEW OF RESEARCH FINDINGS IN THE MARKETING LITERATURE Preliminary investigation into the feasibility of developing a computer simulation approach to the teaching of these concepts took the form of a review of several articles dealing with mail questionnaire response rates (in marketing research). Four questions guided the literature search: (1) what is the shape of the returns curve over time; (2) what options have been researched; (3) what is the effect of the various options on mail survey response rate; and (4) what degree of interdependence exists among the effects of the various options? The review was thorough but not exhaustive. Its intent was to uncover common themes and findings. Computer Simulation and Learning Theory, Volume 3, 1976 299 Figure 1 contains a summary of the results of the literature search which revealed that the effects of at least six options have been reported. These are: an advance letter or postcard informing the prospective respondent of the coming questionnaire; a monetary incentive either included in the letter or promised in return for completing the questionnaire; a follow-up postcard or letter to nonrespondents; a personalized cover letter; a stamped return envelope instead of the customary business reply; and an offer of a summary of the survey results. At this point in the development of the simulation, the presence of several difficulties became apparent. One major problem was the fact that the time periods spanned by the several surveys reported ranged considerably. As can be seen in the chart, the shortest time period reported was 2 weeks while the longest was 6 weeks. Furthermore, three reports do not indicate specific time periods. Another problem emerged in assessing the scope of each survey. Generally, authors do not report the geographic region involved nor are they equally precise in their descriptions of the sample unit. Where the sample unit is described, it is evident that some studies involved special and nonrepresentative segments. Stafford (1966), for instance, used students in his study, while Keane (1963) surveyed panel members. Other troublesome considerations arose in comparing specific options. Several years are spanned by the studies and inflation undoubtedly is a consideration in the comparison of the effects of monetary inducements; advance letters were sent at varying days in advance; follow-up letters were mailed at different time intervals; and the concept of personalizing was not described in sufficient detail. As a consequence of these problems, it was decided to concentrate on three articles which provide appreciable detail and/or analysis beyond the Computer Simulation and Learning Theory, Volume 3, 1976 300 Computer Simulation and Learning Theory, Volume 3, 1976 301 Computer Simulation and Learning Theory, Volume 3, 1976 302 descriptive level. Cox (1966) offers a detailed account of the cumulative response rate over a twenty-eight day period. Cox, Anderson, and Fulcher (1974) report cumulative rates at various time intervals in a comparison of four alternative strategies. They report no significant interaction between personalization and follow-up postcards. They also conceptualize a decision model treating costs, response rates, and various survey objectives. Wiseman (1973) compares four options: Offer of survey results, 10¢ monetary incentive, follow-up postcard, and stamped return envelope. He concludes that the variables operate independently and that interaction effects are relatively unimportant. DESCRIPTION OF THE COMPUTER PROGRAM Basic Logic Basically, the computer program allows the student to manipulate several mail survey research decision variables and to compare the results of various strategies. He may select any one or a combination of the six options mentioned above (or none). Certain options require additional decisions such as the size of a monetary inducement or the time period before a follow-up letter is mailed. Additionally, he must specify the number of initial mailouts and the number of days involved in the survey (cutoff day). Figure 2 presents a flow chart of the manner in which the simulation converts these inputs into the results of a mail survey. The program simulates a binomial sampling distribution associated with each option. The results are summed (using the independence assumption) and a cumulative return factor associated with the cutoff day is applied. Common costs and direct costs associated with each option are stored internally and computed in the form of a total direct cost for the survey. Output takes the form of number of responses, percent response, total direct cost, cost per Computer Simulation and Learning Theory, Volume 3, 1976 303 Computer Simulation and Learning Theory, Volume 3, 1976 304 response, and particulars associated with specific options. The student may submit a number of survey options cards in a single run. Each survey is simulated five times to demonstrate variability. The response rates and number of days for each survey are presented graphically at the end of the complete run. See Appendix I for instructions provided to the student. Appendix II has two illustrative printouts. The Response Function A pattern of cumulative responses over time is described in detail by Cox (1966) who points out that the response curve for consumer mail surveys is s-shaped. It was decided to use the equation for a Gompertz curve of the form: Where: r = the response rate at time t c = the growth at maturity (the upper asymptote of 100 percent) a = the proportion of initial growth R = the rate of growth Comparisons between the percentages generated by this equation and reported findings found that the values of the parameters for a reasonable fit should be .0004 for a and .775 for R. Thus, .0004 < r < 1.00. Perusal of the reported effects of various options suggested that certain factors seemed to accelerate the response function. While findings are scanty, logical analysis implies that an advance letter and a sufficient monetary inducement affect responses in this manner. Consequently, an accelerated response curve with the values of .0003 for a and .750 for R is’used in the simulation if either or both of these options is chosen by the student. Both response curves are compared graphically in Figure 3. Computer Simulation and Learning Theory, Volume 3, 1976 305 Computer Simulation and Learning Theory, Volume 3, 1976 306 Treatment of the Effects of the Options Each option has a unique effect on the response rate. Disregarding random error generated by the simulation, the general form of the net response equation is: Where: p = the response percentage (at time t) B = a simulated base response assuming no options R1 = the simulated response increment associated with option I r~ = the response rate at time t (cutoff day) As one can see, the model assumed makes use of Wiseman’s (1973) finding of independence of the main effect. Each option has a unique incremental response rate associated with it. Although comparison across studies was difficult due to the aforementioned problems an endeavor was made to base the response percentages on reported empirical results. Table 1 contains the percentages used by the simulation. A control statement does not allow the final percentage of response to exceed 95 percent. Computer Simulation and Learning Theory, Volume 3, 1976 307 Treatment of Direct Costs Each option has a unique direct cost equation associated with it. The unit costs are communicated to the student and stored within the program. The total direct cost of each simulated survey is computed by the following general formula (in simplified form): Where: TC = total direct cost of the survey N = number of letters mailed out O = total direct unit preparation and mailing out cost common to all options or base. Ci = additional direct unit costs associated with mail out under option i p = response percentage Xi = direct unit cost associated with the return of a questionnaire under option i Common direct costs associated with mailing out the survey include costs such as paper, duplication, envelopes, stuffing, sealing and postage. Direct unit costs associated with mailing out under various options are exemplified by labor and postage for stamped return envelopes; special typing for personalized cover letters; duplicating, and stuffing, or mailing costs for advance letters. Costs associated with returns under various options include monetary incentives and business reply postage. See Appendix I for a detailed description of the associated costs. USE OF THE PROGRAM As mentioned earlier, the program may be used in a number of different ways; however, its primary purpose is to develop a conceptual framework in students for evaluating mail survey strategies. Students are urged to experiment with the Computer Simulation and Learning Theory, Volume 3, 1976 308 options to gain a feel for the trade-off between the accuracy of information secured (number of~returns), the time period, and the cost of the survey. The author has adopted a three-stage approach in using the simulation which systematically familiarizes students with mail survey strategies, develops the decision- making framework, and evaluates the degree of learning. The objectives of each phase and an assessment of student reactions and learned concepts follows. Familiarization Phase The initial phase of the use of the program requires that the student become familiar with the distinguishing characteristics of mail surveys. Lecture and discussion concentrate on the low response problem, the time lag factor, and cost relative to other survey modes. Discussion then turns to methods of improving response rates in mail surveys, whereupon students are introduced to the various options and the program control card structure. The initial assignment requires students to develop an understanding of the general shape of the response rate curve as well as for its characteristics under various options. The graphical presentation provided at the end of the output is advantageous at this point, and the more enterprising students put it to good use. Figure 4 shows how two different options may be compared in the s~ run with the graph. Because the author’s students are not intimate with analysis variance at this time in the course, students ate encouraged to develop their own methods of systematizing the results of their experimentations. Generally tabular or graphical presentations are used to compare response rates. Each student team presents its findings to the class and discussion is guided to emphasize unique attributes of each option. Computer Simulation and Learning Theory, Volume 3, 1976 309 Computer Simulation and Learning Theory, Volume 3, 1976 310 The author has come to believe that the best teaching strategy at this stage in the use of the simulation is to assign individual students or student teams the task of investigating the effects of specific options or combinations of options. Early experiences with the simulation revealed that an unstructured assignment overwhelmed the average student there were simply too many variables and combinations of variables with which to cope. Students tended to concentrate on comparisons between options at a single point in time rather than to investigate return rates over time. With individual assignments, students have exhibited greater cooperation and understanding. Furthermore, the general characteristics of the response curve come forth as a common factor during group presentations. Thus, students soon discover the growth shape of the curve and come to realize that carrying the survey beyond four weeks has marginal value in terms of additional returns regardless of the strategy used. Class discussion is directed to differences between response rates to various options and students have been quick to speculate on the logical connections between specific options and response characteristics. Sensitivity Phase Upon becoming familiar with the basic aspects of the response function, students’ attention is directed toward the trade-off characteristics of mail surveys, specifically the interrelationships between survey direct costs, number of responses, and number of days. The interrelationships are not readily seen as a consequence of the discontinuities in the cost functions and nonlinearity of the response rate. To complicate matters, certain costs are themselves dependent on the response rate (the return postage cost in the case of business reply postage, for example). Consequently, the author has assigned a between-phase Computer Simulation and Learning Theory, Volume 3, 1976 311 transitional role to the cost-per-response value. Students are encouraged to perform comparative analysis by holding one factor constant and comparing the change in the cost-per-response figure over time. Figure 5 illustrates its general form. At this point in the analysis it becomes apparent to the student that diminishing returns exists in the form of the downward sloping curve which approaches its minimum somewhere around 30 days in most cases. Thus an outer bound on the number of survey days is established. Conceptualizing the cost-time-number of responses trade-off s is more difficult for students as it requires three-dimensional analysis. Students are already familiar with the general form of the returns curve over time; consequently, the sensitivity phase of the exercise turns to analyses of total direct costs for various total responses under alternative strategies and total direct costs for various tine periods under alternative strategies. Due to the interrelationships of the various costs and the differential response rates, certain strategies are more advantageous in terms of minimizing total cost to achieve a desired minimum number of returns. Alternatively it becomes apparent that the minimum number of returns may be gained in fewer days at higher cost. Figures 6* and 6b presents the manner in which students are advised to compare alternative strategies. Figure 6a compares total direct survey costs to the number of returns at prespecified points in tine, while Figure 6b compares total direct survey costs to the number of survey days. Ideally, the interrelationships should be conceptualized as “strategy surfaces” in three-dimensional space defined by time, total direct cost, and number of returns. Visual aids in the form of transparency overlays are helpful here. The final aspect of the conceptualization requires the imposition of constraints on the survey. Students are made aware that mail questionnaire Computer Simulation and Learning Theory, Volume 3, 1976 312 Computer Simulation and Learning Theory, Volume 3, 1976 313 Computer Simulation and Learning Theory, Volume 3, 1976 314 Computer Simulation and Learning Theory, Volume 3, 1976 315 surveys operate under constraints of the general form: “Obtain a minimum of N returns at a maximum cost of TC within a maximum of t days.” Admittedly, this is a simplistic specification of the objectives and constraints; however, it allows students to apply them as bounds in their analyses. Figures 6a and 6b contain illustrations of these boundaries. In actuality, of course, the bounds define a feasibility region in three-dimensional space. Given these, the student must choose the best strategy by selecting from those that lie within the enclosed area. The presentation is not readily understood by the majority of students initially; however, the method of analysis eventually becomes more clear with discussion and illustrative runs. At the very least, students come to realize that unidimensional analysis is inappropriate. While they do not assimilate the exact shapes of the strategy surfaces, the logic of the alternative evaluation procedure is acquired. “Actual Survey” Phase The final phase of the use of the program is an individual assignment in the form of a scenario case study. The following example is representative: “The mail survey with ‘which you are presently concerned entails a random sample of homeowners in a large regional area. The region is almost 60,000 square miles in area with a population of about 7,000,000 people, the questions on the questionnaire are relatively straightforward although there are some which solicit ‘confidential’ information. The respondents are assured that the confidentiality of their responses will be respected. Management desires this information as soon as possible but no more than 20 days after mailout. Anticipated statistical analysis requirements require that the final sample size be no less than 400. Your target budget is $500, but you may spend up to 20% more if you can guarantee sufficient returns in less than 20 days limit.” Each student is required to present his mail survey plan by enumerating the number of mailouts, the specific options, a time table, and the estimated Computer Simulation and Learning Theory, Volume 3, 1976 316 final cost in the form of a detailed budget. Students are not allowed to experiment with the program at this stage, and the final report includes one decision card which is rim by the Instructor. Evaluation places emphasis on the rationale of the student’s choices rather than the outcome of his decisions. Specifically, students must demonstrate that they can apply learned concepts to this situation. CONCLUSIONS As a facilitating device, the described mail survey computer simulation represents unique advantages over the cursory treatment found in marketing re- search textbooks. Iii particular, it familiarizes students with options and tactics often employed to increase the response rate. Through experimentation and comparison, students quickly envision the response curve and develop au appreciation for the relative effects and special considerations of various options. The author has related a three-phase method of using the simulation to teach the interrelationships and trade-of fs between cost, time, and the number of responses. The simulation plays a vital role iii generating data to illustrate the strategy surfaces and decision constraints. The author believes that the simulation is au invaluable aid in the teaching of these theoretical concepts. REFERENCES Boyd, Harper W. Jr., and Ralph Westfall. Marketing Research: Text and Cases, Illinois: Richard D. Irwin, Inc., 1972. Cox, Eli III, Thomas W. Anderson Jr., and David C. Fulcher. “Reappraising Mail Survey Response Rates,” Journal of Marketing Research, Vol. 10 (November 1974), 413-17. Computer Simulation and Learning Theory, Volume 3, 1976 317 Cox, William E. “Response Patterns to Mail Surveys,” Journal of Marketing Research 2 (November 1966), 392-7. Ford, Neil M. “Advance Letter in Mail Surveys,” Journal of Marketing Research 3 (Nay 1967), 202-4. Francel, E. G. “Mail Administered Questionnaires: A Success Story,” Journal of Marketing Research 2 (February 1966), 89-92. Frankel, L. R. “Row Incentives and Subsamples Affect the Precision of Mail Surveys,” Journal of Advertising Research 1 (September 1960), 1-5. Green, Paul. E. and Donald S. Tull. Research for Marketing Decisions, New Jersey: Prentice-Hall, Inc., 1975. Heaton, E. E. “Increasing Mail Questionnaire Returns With a Preliminary Letter,” Journal of Advertising Research 5 (December 1965), 36-9. Keane, John G. “Low Cost, High Return Mail Surveys,” Journal of Advertising Research 3 (September 1963), 28-32. Kephart, William and Marvin Bressher. “Increasing the Response to Mail Questionnaire: A Research Study,” Public Opinion Quarterly 22 (Summer 1958), 123-32. Luck, David J., Hugh C. Wales, and Donald A. Taylor. Marketing Research, New Jersey: Prentice-Hall, Inc., 1974. Roscoe, A. Marvin, Dorothy Lang, and Jagdish N. Sheth. “Follow-Up Methods, Questionnaire Length, and Market Differences in Nail Surveys,” Journal of Marketing, Vol. 39, No. 2 (April 1973), 20-7. Simon, Ramond. “Response to Personal and Form Letters in Mail Surveys,” Journal of Advertising Research 7 (March 1967), 28-30. Stafford, James E. “Influence of Preliminary Contact on Mail Returns,” Journal of Marketing Research 3 (November 1966), 410-11. Watson, John J. “Improving the Response Rate in Mail Research,” Journal of Advertising Research 5 (June 1% 5), 48-50. Wentz, Walter B. Marketing Research: Management and Methods, New York: Harper and Row Publishers, 1972. Wiseman, Frederick. “Factor Interaction Effects in Mail Survey Response Rates,” Journal of Marketing Research 10 (August 1973), 330-3. Wotruba, Thomas R. “Monetary Inducements and Mail Questionnaire Response,” Journal of Marketing Research 3 (November 1966), 398-400. Zaltman, Gerald and Philip C. Burger. Marketing Research: Fundamental and Dynamics, Illinois: The Dryden Press, 1975. Computer Simulation and Learning Theory, Volume 3, 1976 318 APPENDIX I REPRESENTATIVE STUDENT HANDOUT DESCRIPTION MAIL SURVEY RESPONSE RATE SIMULATION-EXERCISE INTRODUCTION The greatest drawback to the use of mail surveys in marketing research is low response rates. In fact a return of 25% is generally considered successful. Some researchers, however, have reported very good response rates (above 85 percent) when they broke away from the standard format for mail surveys. These researchers have tried several strategies including: advance letters to alert respondents of the coming questionnaire, personalized cover letters, stamped return envelopes rather than business reply, monetary incentives, and follow-up letters to remind respondents to return the questionnaire. Marketing research operates under a budget constraint; consequently, a conflict arises when the researcher attempts to increase the response rate because each of the above methods adds to the cost of the survey. It also operates under a time constraint and it is some times justifiable to increase the cost of a survey if it will result in a greater amount of information in a shorter time period. The problem which confronts the researcher then, is to select the option or options which affords the greatest number of returns in the shortest time period and still does not exceed the budget limitation. DESCRIPTION OF THE OPTIONS: The researcher has no less than seven options from which to choose. 1. Standard Format. The standard format consists of a business envelope addressed to “Occupant” at the address indicated. The cover letter is addressed to “Dear Consumer:”, and the return envelope has a postal meter return postage stamp designating it as business reply mail. 2. Advance Letter. The advance letter option is one ‘in which an advance letter alerting the respondent to the coming questionnaire Is sent two days ahead of the actual questionnaire letter. 3. Personalized Cover Letter. With the personalized cover letter and address option, the respondent’s name is typed on the envelope and on the cover letter. 4. Stamped Return Envelope. With this option, the self-addressed return envelope has a first-class postage stamp instead of the business reply postage designation. Computer Simulation and Learning Theory, Volume 3, 1976 319 5. Offer of Survey Results. Some respondents request a sumary of the research study results. It is feasible to require respondents to include their names and addresses at the end of the questionnaire so that these summaries can be mailed to them. 6. Monetary Incentive. A monetary incentive may be promised in the cover letter and questionnaire. Presently, the options are: a dime, a quarter, a half-dollar, a dollar bill, or a five-dollar bill. The respondent receives it only after he returns the questionnaire. 7. Follow-Up Postcard. The researcher may opt to send a follow-up postcard reminding respondents who did not return the questionnaires by a prespecified date to fill them out and return them. He must decide ahead of time what day will be the mail-out day for the postcards. They are mailed to all respondents who did not return their questionnaires by that day. DESCRIPTION OF COSTS As related earlier, the immediate problem is to control costs so as not to exceed the survey budget. It is therefore mandatory that the researcher has a complete inventory of the costs associated with the survey and specific costs associated with each option. (We are only concerned with the variable costs in this exercise.) Unfortunately, the costs are a bit complicated. 1. Common Costs. The following costs are common to all options. a. Duplication costs Each page must be duplicated, and the following schedule applies: Three pages must be duplicated: the questionnaire is two pages long and the cover letter is a single page. b. Collating cost. The two pages of the questionnaire must be collated. The rate is $3.00 per 1000 sheets handled or $.006 per questionnaire. c. Stapling cost. The questionnaire sheets must be stapled together. The stipulated rate is a flat $10 for any amount of questionnaires up to 1000. Any other beyond 1000 are charged $.01 each. Computer Simulation and Learning Theory, Volume 3, 1976 320 d. Envelopes. Two envelopes must be purchased for each letter. The price of envelopes is listed as $.02 apiece for any amount up to 1,500. Beyond 1,500, they are $.01 each. e. Stuffing and seal in Q cost. Stuffing and sealing is done by office workers. The wages are $3.00 per hour, and the average stuffing and sealing rate s 50 letters per hour. Therefore, the cost per letter averages $.06. f. Mail out costs. A business postage meter is used for mailing the letters out. In order to qualify for bulk rates, at least 200 letters must be sent out in a bundle. The bulk rate outgoing is $.063 per letter, and the company is charged $.12 for each reply. 2. Cost Specific to Options. Each option carries with it special additional cost. a. Advance letter. Costs are: additional envelopes, duplication for one page, folding, stuffing and sealing, also postage out. b. Personalized letter. Requires additional typing on envelopes and cover letters. Typists are paid $3.00 per hour and average 45 letters per hour; thus, additional typing cost is $.067 per letter. c. Stamped return envelope. When office workers are required to place a stamp on the return envelope, their speed decreases to 40 letters per hour or $.08 per letter. Stamps are $.10 each. d. Offer of survey results. The offer and provision for respondent’s name and address do not incur any additional cost. The results summary is returned with promotional literature; consequently, the research budget does not absorb this cost. e. Monetary incentive. The cost of the incentive is equal to the incentive used (e.g., a dime is $.10). Promotional literature is included with the incentive; hence, the promotional budget absorbs the mail-back costs. OPERATION OF THE COMPUTER SIMULATION The computer program is designed to allow you experiment with the various options in order to determined their effects. Each time you submit your choice of options, it simulates 5 surveys, each one independent of the others. These five outcomes will acquaint you with the variability that often characterizes marketing research studies. Probably the best strategy is to compute some sort of average result for the five outcomes and proceed from there. Different options and different combinations of options will produce different outcomes and it will be up to you to make several runs and perform meaningful analysis. It should be noted that each of the options affects the response in a unique way, and you should attempt to determine this early in your experimentation. Computer Simulation and Learning Theory, Volume 3, 1976 321 Also, some options not only affect the rate of response, but they also affect the timing of the responses. That is, respondents may return their questionnaires sooner than normal. MULTIPLE DECISION CARDS The program accepts up to 25 decision cards on a single run. Each survey is simulated five times. An average percentage returns is computed for the five replications and printed out after each set. PLOT If five or more decision cards are included in a run, the program automatically ends with a plot of the average percentage returns against the cutoff day. Erroneous decision cards are plotted as zero response and zero days. Computer Simulation and Learning Theory, Volume 3, 1976 322 Computer Simulation and Learning Theory, Volume 3, 1976 323