TEACHING ABOUT SAMPLING IN A MARKETING RESEARCH CLASS Developments in Business Simulation & Experiential Exercises, Volume 14, 1987 57 TEACHING ABOUT SAMPLING IN A MARKETING RESEARCH CLASS David Finn, Texas Christian University ABSTRACT In the spirit of sharing that is characteristic of ABSEL, this paper describes a classroom exercise that the author uses to convey the meaning of a sampling distribution. Although students enrolled in marketing research courses usually are required to have passed an introductory statistics course, the concept of sampling distributions (so important in understanding sampling) seems to have eluded them. A description of the class exercise, instructions to students, and the source program is included. INTRODUCTION For years I have been struggling with the problem of how to get marketing research students to understand the reasoning behind sampling and sample size theory. Text-book treatments, although usually very thorough, tend to be beyond the comprehension of the average student. Many texts use a standard reference and a similar set of figures to describe how the distribution of the sample mean approaches a normal distribution with very small standard error as the sample size increases (e.g., see [1] and [3] who both cite [2]). This is shown to be true when the statistic being measured is the mean, and with larger sample sizes, when the statistic is the proportion of sample members in a particular category. Feed-back tests reveal that students are able to memorize the rules and formulae associated with such coverage, but when they are asked to actually use or explain the phenomenon, it becomes clear that understanding is often totally absent. This is not surprising, given that fact that many of us did not understand the concept until graduate school! Yet it is not a very complex topic! I have come to believe that the problem is in the methods we use to “teach” the idea. Texts that treat the topic with patience (e.g., [1]) develop the idea of a “derived population” of sample means by showing a listing of all possible sample means that can be computed from a stated population (necessarily small so that it can be listed in the book). Then those means are plotted to show the distribution of means. Generalizations about larger populations follow. To a trained reader, these patient treatments look elegant. Only when we get into the classroom do we realize that our students are either totally confused or have decided to skip “all that explanation”. The elegance is lost on them! As a member of ABSEL, I KNEW that if it were possible to let students generate observations of distributions of means by themselves, they would have a much better learning experience. The problem was that the marketing research course presumes that students already know and understand the topic (since introductory statistics is a prerequisite course) and is usually not designed to devote much time to such topics - - besides (at least at our school), marketing research emphasizes planning, using, and evaluating marketing research as a decision aid and de-emphasizes the statistical methods used in research. That means that any experiential learning of statistical concepts should be a very MINOR part of the course. All of the above led to the development and trial of the exercise that I am about to describe. THE EXERCISE One week before the concept of sampling is introduced, students are given an assignment to complete. They are told to sign on to the main frame (although any system will work) and to execute a program called SAMPLE (reproduced in Figure 1). I explain to them in class and via a conversational introduction to the program that the program samples from a very large population (millions) of beer drinkers whose ages range from 18 to 65 (the “beer drinker” description is purely arbitrary - - in keeping with an established reputation). The only information that the computer will extract (they are told) is the age of each population member. What they have to do is ask the computer to sample from that population and to tell them: 1) the ages of the sample elements; 2) the average of the sample; and 3) the percentage of the sample that is in each of 4 different age categories (18 to 24, 25 to 34, 35 to 49, and 50 to 65 years old). The assignment also instructs them to draw several samples of size 10, of size 20, of size 50, of size 100, and of size 200. The actual number of samples they must draw depends on the class size. Generally, I try to get between 75 and 90 different samples among the entire class (e.g., in a class of 45 students, I would have each student ask for 2 samples of each sample size, for a total of 90 samples of each sample size in the class). Since the computer lists the samples and the associated statistics on the CRT, I supply the worksheet in Figure 2 as an aid in recording the results. The program that I have used (and which is presented in Figure 1) is a simple simulation with an average age of 36.5 years, 45% between 18 and 24 years, 10% between 25 and 34 years, 10% between 35 and 49 years, and 35% between 50 and 65 years. These are the parameters that the class will try to discover. On the assigned day, when the students come to class with their worksheets, the learning begins. To start, I ask one student to look at his/her first sample of size 10 and to tell me and the class what the average age of the sample is. When s/he reports it (perhaps something like “25.76 years old”), I say something like “Right!” or “Very Good! How many got that?” Needless to say, most of the students look at me like I am some kind of idiot and point out that they are all different. Which leads to THEIR teaching ME about sampling error. When everyone agrees that every sample SHOULD be different, I ask “how different should they be?” Since they don’t know, I suggest that we plot all their samples on the board to see if we can make any sense of them. “How many average ages in your sample of size Developments in Business Simulation & Experiential Exercises, Volume 14, 1987 58 Developments in Business Simulation & Experiential Exercises, Volume 14, 1987 59 10 fall into the range 18 to 20 years old?” Count a show of hands and plot it on the board. “Between 21 and 23 years old?” And so on. Figure 3a is a reproduction of a plot of 90 student samples plotted from samples of size 10. Following the same format, we then plot the average ages from samples of size 20 and notice that it is almost a normal curve! Then 50, 100, and 200 (although the students see the pattern by the time we have plotted samples of size 50 and it is efficient to skip to 200). Figure 3 reproduces actual student data for several sample sizes. Notice that the result is very similar to the text book figures that this exercise is meant to reproduce! The concepts that I try to get them to notice are: 1. with small sample sizes, our samples are all very different from each other. 2. as sample size increases, all our samples get more and more similar. 3. even with sample sizes of 20, the distribution we have been drawing on the board looks like a normal curve (then discuss what they know about the standard deviation of a normal distribution, etc. - - this reminds them of confidence intervals). Next, we replicate the exercise with percentages. First is the percentage of people in the age category 18 to 24 years old (recall that the “truth” in the simulation is 45%. This was a deliberate choice, so that this distribution could be compared with a 10% category.) - - then with the other age categories. Actual student data is reproduced in Figures 4 and 5. What I try to get them to notice here is: 1. when the “truth” is around the middle of the scale (e.g., 457.), the range of sample percentages is much wider than when the “truth” is closer to one end of the scale (e.g., 10% for the 25 to 34 year old category). 2. when the sample sizes are 20 to 50, the sample distributions do not look like normal curves (as they did when we plotted sample means). We need much larger sample sizes with percentages than with means before we can rely on any knowledge we have about the normal curve (which reminds them of the Central Limit Theorem). By now the students usually have a pretty good “feel” for why we can have so much confidence in confidence intervals. They also SEE why sample sizes don’t have to be super big and that a lot of knowledge about the population can be retrieved from samples as small as 200. They understand that if the sample size is big enough, every sample they can draw will have almost the same mean as any other sample. And the experience enhances their understanding of the statistical principles that they have memorized in other courses. The exercise takes approximately 40 minutes of class time; considerably less when fewer sample sizes are plotted. One word of caution on that subject: the instructor must move through the exercise rapidly. Students will get bored quickly if too much time is spent counting raised hands, etc. STUDENT RESPONSE The only measure of success of this simulation exercise is from student comments during the semester. A few students have approached me and commented that they never understood what a sampling distribution was before this class and that they feel like they know it now. Obviously, a true test of its worth would be an experiment using one set of students exposed to it and one set taught in a more traditional method. However, the ease with which we get through the material, and the absence of blank faces during the exercise precludes me from changing things in the direction of the old ways. Like many ABSEL members, I believe that each of us must “tune in” to what is best for our idiosyncratic style. The ABSEL conference offers us all an opportunity to share and to choose that which fits our own methods. I hope that this exercise is of value to many. REFERENCES [1] Churchill, Gilbert A., Jr., Marketing Research (3rd edition) (New York: The Dryden Press, 1983). [2] Kurnow, Ernest, Gerald J. Glasser, and Frederick R. Ottman, Statistics for Business Decisions (Homewood, IL: Richard D. Irwin 1959), pp. 182-183 [3] Zikmund, William G., Exploring Marketing Research (2nd edition) (New York: The Developments in Business Simulation & Experiential Exercises, Volume 14, 1987 60 Developments in Business Simulation & Experiential Exercises, Volume 14, 1987 61 Developments in Business Simulation & Experiential Exercises, Volume 14, 1987 62 Table of Contents Volume 14, 1987 Simulation Gaming as An Experimental Context for the Study of Multicriteria Decision Making How to Multiply Your Management Game Grading the Personnel In-Basket Exercise Simulation: The Players™ Perspective The Acute Susceptibility of Nominal Grouping to Negativity The Relative Value of the National ABSEL Meetings: An Analysis of Perceptions by Faculty and Deans A Survey of Behavioral Labs Used by American Business Schools A Structured Framework for Applying Conceptual Models to Problem Analysis: Hardening Up An Expert System Approach for Teaching Marketing Case Analysis From Theory to Practice: A Model for Teaching Beginning Advertising The Use of a Simple Forecasting Technique During an Interactive Computerized Business Game An Investigation of the Relationship Between Formal Planning and Simulation Team Performance and Satisfaction The Exponential Logarithm as an Algorithm for Business Simulations Learning macroeconomic Theory and Policy Analysis Via Microcomputer Simulation Laptop: A Principles of Marketing Simulation Teaching About Sampling in a Marketing Research Class Integrating Decision Support Systems and Business Games Demand Generation in a Service Industry Simulation: An Algorithmic Paradox ABSEL - At a Crossroads? Research on Predicting Performance in the Simulation An Expert System for Financial Planners "Developing Various Student Learning Abilities Via Writing, the Stock market Game, and Modified Marketplace Game in Beginning Macroeconomics" Professional Activity Reports: Getting back to Basics Œ A Preliminary Investigation "A Comparison of Performance, Attitudes, and behaviors of MBA and BBA Students in a Simulation Environment: A Preliminary Investigation" Decision Styles and Student Simulation Performance: A Replication Personal Power Strategies: An Experiential Exercise LTL I: A Trucking Simulation Game Total Enterprise Business Games: An Evaluation Can the Skill of Management be Taught? Developing Critical Thinking and Effective Communication Meaningful Notebooks and Enhanced Learning Using a Totally Menu-Driven Data base and Decision Support (DSS) Program Using a Computerized Menu-Driven Data Entry and fiWhat-Iffl Program as a Pedagogical Enhancement for Management Simulation Using a Joint Project Involving MBA Marketing Management and Undergraduate Marketing Research Students to Teach Marketing Research Testing the PAGE Technique: Results and Further Developments A Comparison of Student Perceptions with Accepted Expectations for Business Simulations "Conflict Management in Action: Verbal Strategies, Nonverbal behaviors and Conflict Styles" Dramatic Monologues as Surrogates for Experiential Learning Influencing One™s Superior: An Assessment and Exercise Personality Traits and Organizational Cultures: Two One-Hour Experiential Learning Exercises Computype: A Strategic Marketing Game Open System Simulations and Simulation Based Research "Teaching Forecasting to the Masses: Little Decision Support Training, But a lot of Answers" Goal Setting and Performance Evaluation with Different Starting Positions Œ The Modeling Dilemma The Gamesmanship of Pricing: Building Pricing Strategy Skills Through Spreadsheet Modeling Teaching MPR Experientially Through the Use of Lotus 1-2-3 Business Simulation Through Activities of a Manufacturing Company Simulated Stock Broker Teaching Employee Counseling Skills to Management Students: A Simulation Airline: A Strategic Management Simulation Developing Simulations and Experiential Exercises on the Personal Computer: Some Critical Issues A Powerful Tool Œ For What? An Integrative Systems Approach to Simulating Free Enterprise in a Small Business Course in a Business School Profits: The False Prophet The Use of Expert Systems to Develop Strategic Scenarios: An Experiment Using a Simulated Market Environment An Interdisciplinary Workshop: Spreadsheet Modeling and Case Analysis Techniques for Beginning Executive MBA Students An Innovative Approach to an O. D. Class Œ The Creative Interaction The Application of Spreadsheet Software Technology to Complete Taxpayer Elections Experiential Learning as the Cornerstone of an Industry Management Course "Identifying, Exercising and Enhancing Right Brain Skills in the Business Classroom" Utilizing Envisionary Technology in the Business Policy Course: An Experiential Pedagogy Business Simulation in the Policy Course: A Survey of American Assembly of Collegiate Schools of Business The Moving Van: Vehicle to Success of Disaster for the Two-Gender Work Force Oil and Gas Well Investment Analysis Using the Lotus 1-2-3 Decision Support System Team Cohesion Effects on Business Game Performance