AN EXPERIENTIAL EXERCISE IN BAYESIAN DECISION-MAKING Developments in Business Simulation & Experiential Exercises, Volume 13, 1986 178 AN EXPERIENTIAL EXERCISE IN BAYESIAN DECISION-MAKING John R. Dickinson, University of Windsor ABSTRACT Bayesian decision-making is a fundamental element of most contemporary marketing research texts. The model, in a marketing research context, serves basically to structure the components of a normative decision-making situation and to place the role of marketing research within that structure. In broader contexts, the model has been employed in a variety of marketing strategy decision areas, for examining (consumer) decision-making behavior, and so on. The experiential exercise described herein facilitates the introduction of Bayesian decision-making with the direct involvement of students in an abstract decision- making situation and by serving as a vehicle for discussion and other investigation of the Bayesian model. INTRODUCTION AND BACKGROUND The purpose of this paper is to describe a computer supported experiential exercise in Bayesian decision- making. The content of the exercise (based on a well- known published experiment), general administration of the exercise, the potential “lessons” of the exercise, and the supporting computer programs are discussed. Specific details of administration are presented in an appendum. The exercise has been used by the author on several occasions in undergraduate and graduate introductory marketing research courses. However, Bayesian decision theory has application in any number of marketing areas and the appropriateness of the exercise is not limited to a marketing research context. Decision theory has been an integral part of marketing research, in the classroom and in basic and applied research, for at least decades. The Bayesian model’s popularity in marketing circles blossomed in the early 1960s with the publication of cornerstone articles in major marketing journals by major figures in marketing research [2;3;7;14]. Though publication of Bayesian theoretical and application articles seems to have peaked by the mid-1970s, there exists a healthy body of marketing-related literature on the topic. The Bayesian model has been employed in a variety of marketing contexts including consumer behavior [13], marketing strategy formulation [5; 15; 16; 20; 29; 31), marketing management [8; 21; 23], and forecasting [10; 28] as well as in marketing research [9; 17; 25]. Bayesian decision theory has been employed in at least two experiential teaching applications, audit sampling [11] and sample size determination [12], and it is standard fare in contemporary marketing research texts. Virtually all current introductory marketing research textbooks address statistical decision-making to some degree. In many instances, this is by way of laying a foundation for the introduction to some variation of the scientific method (i.e., the “marketing research process”) as a paradigm for designing research in marketing and, more pointedly, for the purpose of weighing the acquisition of “additional information”, i.e., conducting marketing research. Accordingly, the basic components of a decision- making situation are identified and placed in the common payoff table format. Thereafter, the phases of prior, preposterior, and posterior analyses are presented. The table below provides an overview of the treatment of statistical decision-making by major marketing research texts. The in-class exercise described herein was developed to facilitate the experiential involvement of students in a (clinical) Bayesian decision-making situation. Generally, the objectives of the exercise are to (1) allow for identification of the components of the normative decision-making situation and to consider their interrelationships and Implications, (2) draw parallels between the exercise scenario and real world decision- making situations, and (3) monitor the degree of Bayes- like behavior exhibited by the students. Several related analyses, both empirical and qualitative, may be based on the exercise. EXPERIENTIAL EXERCISE DESCRIPTION The experiential exercise design is an adaptation of an experiment originally carried out by Green, Balbert, and Minas [18]. Two batch process computer programs generate the required administration materials and carry out appropriate analyses, respectively. Two hypothetical populations are defined, one consisting of 70% “X” cards and 30% “0” cards and the other consisting of 30% 0 cards and 70% X cards. Students are given samples of 50 (punched computer) cards drawn Developments in Business Simulation & Experiential Exercises, Volume 13, 1986 179 at random from one of the two populations, each card being punched with either an X or an 0. As well, students are told the prior probability that the sample deck was drawn from either of the populations. Each student is given nine decks of 50 cards with the prior probability of a deck’s origin being the X population varying from .1 to .9. At each of the nine trials, each student decides how many (of the 50 available) sample cards he/she wishes to purchase, records the number (on a tally sheet printed by the generation program), selects that number of cards, records the nature of each card (X or 0), and decides as to the card deck’s origin. Though various parameters may be used, the author (consistent with the original published experiment parameters) has typically specified a $.03 cost per sample card purchased, $1.00 added for a correct origin decision, and $2.00 subtracted for an incorrect decision. (Thus, rewards are based on decision- making success, not on adherence to the Bayesian model.) To enhance student involvement, the author also usually provides a real dollar pool (averaging about $2.00 per student) to be prorated among the students based on final asset position. (Final asset position equals correct decision “winnings” as specified, less incorrect decision “losses”, net of information expenditures.) The analysis program indicates each student’s final asset position and automatically prorates the actual dollar pool. Input to the analysis program consists of sample size purchased, number of X cards in the sample purchased, and origin decision for each student for each trial. (A master deck containing true origin information is provided by the generation program.) BAYESIAN MODEL APPROACH Students make two critical decisions in the course of this exercise: (1) the number of sample cards to purchase and (2), in light of whatever sample information was purchased, the decision as to whether the card deck was originally drawn from the population 0 stack or from the population X stack. Normatively, these two decisions are based on the following described process: 1. For each feasible sample size, 0 to 50 cards in this application, determine all possible sample outcomes, i.e., all possible mixes of X and 0 cards. 2. For each possible sample outcome, tentatively revise the prior probabilities to yield posterior probabilities of a given deck being either an X or an 0 type. 3. For each set of posterior probabilities calculate an expected monetary payoff for each decision alternative, i.e., an X decision and an 0 decision. This step yields a series of conditional expected values. 4. Determine the marginal probability of obtaining each possible sample outcome for each sample size. 5. For each possible sample, multiply its marginal probability times the higher (of either an X decision or an 0 decision) conditional expected monetary value determined in 3. for each possible sample outcome to yield an unconditional expected value. 6. Select the sample size leading to the highest unconditional expected monetary value. The above steps comprise the preposterior phase of Bayesian decision-making. Having, normatively, determined the optimal sample size, one then proceeds to select that number of cards from the 50 available. In light of the actual nature of the cards selected, one need only refer to the applicable contingencies already mapped out in the preposterior phase to make one’s decision as to the deck’s origin. CRITERIA The two decisions, sample size and deck origin, give rise to three (partially redundant) measures of the degree to which students deviate from normative Bayesian behavior. - Sample size: Comparison between optimal and actual number of cards purchased. - Unconditional expected value (UNCEV): Sample size should be determined by the UNCEV associated with each possible sample size. Thus, the UNCEVs of the optimal and actual sample sizes may be compared. - Conditional expected value (CEV): Once sample information (of optimal size or not) has been procured, the deck origin decision should be based on the CEV of the X or 0 choice alternatives in light of that information. Thus, the maximum and actual CEVs may be compared. (These will be equal where the student decides with the Bayesian norm.) The above deviation measures are presented by the analysis program for each student for each of the nine trials and in aggregate over the nine trials. Two additional measures related to actual decision- making success are also presented: correctness of each decision and final asset position (as defined earlier). Due to sample acquisition expenditures, these two measures need not be perfectly related. Correlations between the three deviation measures and these two success measures are provided by the analysis program. (Examples of feedback printouts as well as program listings are available from the author.) PEDAGOGICAL ISSUES Any number of empirical and qualitative issues may be examined within the context of this experiential exercise. It should be borne in mind that the point of such examination is not to demonstrate the validity of the Bayesian model but, rather, to understand its nature. Several of these issues are outlined below. Bayesian Model Per Se - The various components of a normative decision- making situation may be identified within the exercise scenario. - The normative Bayesian procedure may be outlined, including calculation of the prior expected value of perfect information. - Sensitivity analyses of optimal sample size as a function of population make-up (70:30 proportions), Developments in Business Simulation & Experiential Exercises, Volume 13, 1986 180 prior odds, sample information costs, and payoffs may be discussed or, utilizing the analysis program, examined empirically. - For selected combinations of initial parameters, what sample information composition would be necessary for a Bayesian to decide against the prior odds? - The role of sampling error can be illustrated in those instances where the optimal and actual decisions match, but are nevertheless incorrect. - Note that, due to sample information expenditures, the number of correct decisions is not the sole determinant of final asset position. Exercise and Real World Correspondence or Lack Thereof On one hand, this clinical exercise is a considerable simplification of typical real world marketing research scenarios. Having only two decision alternatives and two states of nature, the exercise is of smaller scale. Too, all parameters are much more well-defined. Though this is true for all parameters, the vagary of specifying real-world states of nature and manageably anticipating possible research Outcomes with, say, a multiple question survey, are particularly problematic. (These two factors determine the necessary conditional probabilities. To the author’s mind, arriving at these probabilities is the Achilles’ heel of the Bayesian model.) On the other hand, in a sense, within the exercise the number of “research design alternatives is much greater than would be practically considered in the real world. Research designs in the exercise amount to the possible sample sizes, 51 in all. Seldom are that many possible research projects put before management. (Of course, sample size is also a part of any real world research design and in that sense this point is obviated.) As well, all possible combinations of research outcomes would normally be reduced to a handful of “overall” general interpretations, e.g., favorable, marginal, unfavorable. What the Bayesian model does and does not address: In the typical marketing research textbook context the purpose of the model is to arrive at posterior probabilities and a decision as to whether or not to acquire additional information. However, the additional information within the confines of the model only addresses/affects probabilities of the states of nature. More realistically, marketing research is concerned with other elements of the statistical decision- making situation as well, e.g., developing alternative strategies, defining possible states of nature, and estimating payoffs. Furthermore, the Bayesian model has only a very indirect relationship with many common objectives of marketing research: e.g., advertising effectiveness, segmentation bases, consumer behavior other than choice behavior, and so on. Hardly a criticism of (Bayesian) decision-making, the point is simply that the model may be useful in some circumstances, but not in all. Under what conditions might the Bayesian model be literally applicable in the real world, under what conditions might its principles be subjectively invoked, and under what conditions is the model not applicable? What types of consumers (or marketers) might be likely to behave as Bayesians? What types not? On what specific points would they differ? Student Behavior vis-à-vis Bayesian Model - By inspecting the three measures of deviation, to what degree do students behave as Bayesians? Do they tend to buy too much information or too little and does this vary by prior probability? Are students able to assimilate the sample information they do purchase in a Bayesian manner? - The extent to which it “pays” to behave as a Bayesian can be gleaned from the correlations between the two measures of decision success (number correct decisions and final asset position) and the three deviation measures. - It is feasible to at least calculate the prior expected value of perfect information and thereby establish an upper limit on the sample size to be purchased. How many students did this? - Faced with the virtual impossibility of literally implementing the Bayesian model, what heuristics did students invoke? Do people behave intuitively as Bayesians? - In those instances where sample information seemed contrary to the prior odds, which way did students decide? This seemingly innocuous experiential exercise, straightforward for the instructor to administer and for the student to assimilate, fosters a high level of involvement and bears considerable scrutiny. COMPUTER FUNCTIONS Generation Program Output 1. Specified parameters 2. Master description of sample deck origins and contents, by student, by trial. 3. Coding sheet for preparing summary data for input. into analysis program. 4. Exercise instructions and tally sheets for each student. 5. Punched deck of sample deck composition information. This deck is required input to accompanying analysis program. 6. Punched deck of sample card deck cover cards. These cards serve to cover the sample card information and to identify subject number and each sample card deck as to prior probability. 7. Punched sample card decks, essentially nine decks per student. Analysis Program Feedback For administrative purposes, a single copy of parameters of the exercise is printed and master data bank summary information may be requested. Each student can be provided with a printout consisting of the components described below. The number of 11” x 15.5" standard printout pages per individual copy equals (It students/10) + 3. 1. Glossary of terms labeling the particular printout items. Developments in Business Simulation & Experiential Exercises, Volume 13, 1986 181 2. Descriptive summary of student decision-making behavior vis-à-vis the Bayesian model, by student by trial. Items include actual vs. optimal sample sizes, actual vs. maximum unconditional expected values, actual vs. maximum conditional expected values, and an optimum-actual-origin decision comparison. Deviations from Bayesian behavior are totaled over the nine trials for each subject. 3. Graph of actual versus optimal sample size purchases, averaged over all students, by trial, and related statistical information. 4. Averages and standard deviations (over students) of actual behavior and deviations from Bayesian model, by trial. 5. Averages and standard deviations (over students) of decision making success and deviations from Bayesian model, aggregated over all nine trials. 6. Correlations between decision making performance alleviations from Bayesian model. Printing of the various major components can be selectively suppressed. REFERENCES [1] Aaker, David A. and George S. Day, Marketing Research, Second Edition (New York: John Wiley & Sons, 1983). [2] Bass, Frank M., “Marketing Research Expenditures: A Decision Model," Journal of Business, Vol. 36, January 1963, pp. 77-90. [3] Buzzell, Robert D. and C.C. Slater, “Decision Theory and Marketing Management,” Journal of Marketing, Vol. 26, July 1962, pp. 7-16. [4] Boyd, Harper W., Jr., Ralph Westfall, and Stanley F. Stasch, Marketing Research--Text and Cases, Sixth Edition (Homewood, IL: Richard D. Irwin, Inc., 1985). [5] Calabro, Natalie, “Bayesian Theory in the Evaluation of New Product Ideas,” Journal of the Academy of Marketing Science, Vol. 1, Spring 1973, pp. 12-24. [6] Churchill, Gilbert A. Jr., Marketing Research-- Methodological Foundations, Third Edition (Chicago: The Dryden Press, 1983). [7] Day, Ralph L., “Optimizing Marketing Research through Cost-Benefit Analysis,” Business Horizons, Vol. 9, Fall 1966, pp. 45-54. [8] Enis, Ben, “Bayesian Approach to Ad Budgets,” Journal of Advertising Research, Vol. 12, February 1972, pp. 13-9. [9] Ericson, W.A., “Optimal Sampling Design with Non- Response,” Journal of the American Statistical Association, Vol. 62, March 1967, pp. 63-78. [10] Fourt, L.A. and J.W. Woodlock, “Early Prediction of Market Success for New Grocery Products,” Journal of Marketing, Vol. 25, October 1960, pp. 31-8. [11] Gentry, James W., Mary H. Bonczkowski and Charles W. Caldwell, “The Use of Program Bayaud in the Teaching of Audit Sampling,” in Daniel C. Brenenstuhl and Samuel C. Certo (editors), Exploring Experiential Learning: Simulations and Experiential Exercises (Denver: ABSEL, 1978), pp. 53-59. [12] Gentry, James W., Alvin C. Burns and Edward T. Reutzel, “The Use of Program Bayes in the Teaching of Sample Size Determination in Survey Research,” in Richard H. Buskirk (editor), Simulation Games and Experiential Learning in Action, (Bloomington, IN: ABSEL, 1975), pp. 73-80. [13] Gentry, James W. and John 0. Summers, “Are Laboratory Induced Revisions in Purchase Intentions Meaningful?”, Journal of Business Research, Vol. 4, May 1976, pp. 177-86. [14] Green, Paul E., “Bayesian Classification Procedures in Analyzing Customer Characteristics,” Journal of Marketing Research, Vol. I, May 1964, pp. 44-50. [15] Green, Paul E., “Bayesian Decision Theory in Pricing Strategy,” Journal of Marketing, Vol. 27, January 1963, pp. 5-14. [16] Green, Paul E., “Bayesian Statistics & Product Decisions,” Business Horizons, Vol. 5, Fall 1962, pp. 101-9. [17] Green, Paul E. and Ronald E. Frank, “Bayesian Statistics and Marketing Research,” Applied Statistics, Vol. 15, November 1966, pp. 173-90. [18] Green, Paul E., Michael H. Halbert and J. Sayer Minas, “An Experiment in Information Buying,” Journal of Advertising Research, Vol. 4, September 1964, pp. 17- 23. [19] Green, Paul E., Michael H. Halbert and Patrick J. Robinson, “An Experiment in Probability Estimation,” Journal of Marketing Research, Vol. II, August 1965, pp. 266-73. [20] Harris, J., “New Product Marketing: A Case Study in Decision-Making under Uncertainty,” Applied Statistics, Vol. 16, No. 1, 1967, pp. 39-42. [21] Layton, R.A., “Controlling Risk and Return in the Management of a Sales Team,” Journal of Marketing Research, Vol. IV, August 1968, pp. 277-82. [22] Lehmann, Donald R., Market Research and Analysis, Second Edition (Homewood, IL: Richard D. Irwin, Inc., 1985). [23] Little, J.D. and L.M. Lodish, “A Media Planning Calculus,” Operations Research, Vol. 17, January- February 1969, pp. 1-35. [24] Luck, David J., Hugh G. Wales, Donald A. Taylor and Ronald S. Rubin, Marketing Research, Sixth Edition (Englewood Cliffs, NJ: Prentice- Hall, Inc., 1982). [25] Mayer, Charles S., “Application of Bayesian statistics to Research Design,” Proceedings, Business and Economics Section, American Statistical Association, 1968, pp. 181-7. Developments in Business Simulation & Experiential Exercises, Volume 13, 1986 182 [26] Nelson, James E., The Practice of Marketing Research, (Boston: Kent Publishing Company, 1982). [27] Peterson, Robert A., Marketing Research, (Plano, TX: Business Publications, Inc., 1982). [28] Reinmuth, J.E. and M.D. Geurts, “A Bayesian Approach to Forecasting the Effects of Atypical Situations,” Journal of Marketing Research, Vol. IX, August 1972, pp. 292-7. [29] Roberts, Harry V., “Bayesian Statistics in Marketing” Journal of Marketing, Vol. 27, January 1963, pp. 1-4. [30] Tull, Donald S. and Del 1. Hawkins, Marketing Research--Measurement and Method, Third Edition, (New York: Macmillan Publishing Company, 1984). [31] Villani, Kathryn E. and Donald G. Morrison, “A Method for Analyzing New Formulation Decisions," Journal of Marketing Research, Vol. XIII, August 1976, pp. 284-8. Table of Contents Volume 13, 1986 How to Use Microcomputers in Human Resource Management Courses What's My Line? An Exercise in Job Analysis, Description and Classification Relationships Between Team Cohesion Dimensions and Business Game Performance Four Factors Affecting Group Performance in Business Policy Simulations Enhancing Strategic Management Goal-Setting Skills in the Business Policy Course Using a Business Simulation in the Principles of Management Course - Learning Outcomes and Perceptions Personal Computers: Drexel's Experience Using Psychological Type to Enhance Negotiation Skills The Effects of Conflict Handling Styles on Negotiation Behavior: An Analysis of an Experiential Exercise An Experiential Test of Bargaining Decisions and Representation Self-Grading of Examination by Students, A Viable Alternative to Traditional Procedures Perceived Instructor Enthusiasm and Student Achievement An Experiential Approach to Teaching Subordinate-Oriented Communication Assessing the Effects of a Computerized Study Guide in Macroeconomic Principles: A Statistical Analysis A Sales Management Simulation for the PC an Integrative Tool for the Sales Management Course Teaching Re-evaluation of Salespersons Through the Use of a Simulation Game A Test of Student Performance and Attitudes and Performance Under Varying Game Conditions Developing and Testing Airways: A Marketing Simulation Experiential Learning Revisited: Some Thoughts on Designing More Adaptive Management Education Programs Developing the Competencies of Creativity and Accurate Self-Assessment Self-View and Norms: Some Longitudinal Laboratory Results Strategic Planning with an Experiential Case The ABSEL Research Consortium: Preliminary Steps and Position Papers An Analysis of ABSEL Conference papers (1974-1985) The Creation and Operation of a Data-base System for ABSEL Research Funding for a Research Consortium on Simulation and Experiential Learning Simulation - Indoctrination or Learning The Dilemma In Evaluating Classroom Innovations Personality Variables on Group Cohesion, Team Participation and Total Learning Justifications for and Problems in Developing and Using Computerized Experiential Activities Management Competency Models and the Life-Long Learning Project: What Role for ABSEL External Validity of Business Games Restructuring the University - An Experiential Exercise The Surrogate Class Exercise Images of Effectiveness: A Classroom Exercise The Algebran Industry: A Tutorial Framework Another Chance: A Multi-Leveled, Macro Business Game for Micro-Computers A Water Quality Management Simulation Game Acquisitions? Divestitures? Progress Report on a Conglomerate Game Enterprise: A Multi-Purpose Management Development Simulation Marketer: A Microcomputer Simulation in a High Tech Industry A Pedagogical Analysis of Alternative Tax Elections for Exploration Costs in the Mining Industry Using of Multiple Microcomputers Application Programs to Teach Fundamental Business Concepts and Practices Demonstrating the Role of Simulation in Strategic Operations Planning: A Case Study in Bank Check Processing Location Analysis The Subjective Side of the Decision Support System: A Pitfall for the Panacea A Group-Based Procedure for Revealing Judgmental Heuristics and Biases An Experiential Exercise in Bayesian Decision-Making Values for Selected Parameters in Physical Distribution Simulation and Games Computerized Business Management Simulations for Tyros The Management Decision Laboratory at New York University An Example: The Use of Management Games on Microcomputers by Computer Novices Enhancing Mainframe Simulation Via Microcomputers: Designing Decision Support Systems Integrating Personal Computers Into a Course as a Decision Support Tool A Decision Support System for Capital Funds Forecasting Performance and Attitudinal Affects of a Decision Support Package in a Business Game Using the Real World: Interviews with Woman Mangers The Cooperatively-Authored Student Handbook International Buying: An Experiential Exercise The Research/Teaching Interface: Turning a Pretest into an Experiential Exercise Building Microcomputer Business Simulations Simulation with Discrete and Continuous Mathematical Modeling An Interpolation Approach to Developing mathematical Functions for Business Simulations The Design of Simulation Models Using Thought Organizers The Assessment Center as a Teaching/Learning Device An Approach to Meeting the AACSB Guidelines for Introducing Skills and Personal Characteristics (SACs) Using an Interclass Simulating and Role-Playing Pedagogy Experiential Learning about the World of Work: A Program for Primary and Secondary School Educators Assessment of Relevant Personal Characteristics of Potential Entrepreneurs