A COMPARATIVE EVALUATION OF A MARKETING GAME Developments in Business Simulation & Experiential Learning, Volume 11, 1984 26 A COMPARATIVE EVALUATION OF A MARKETING GAME Lester Neidell, The University of Tulsa Joseph Wolfe, The University of Tulsa ABSTRACT A popular marketing game was evaluated within a controlled experimental design. The simulation increased knowledge when tested by both objective and essay questions but knowledge increases were not greater than those obtained through the use of cases. The two teaching formats produced different perceptions of the instructor’s testing methods, impartiality, availability, and classroom procedures. INTRODUCTION Most of business gaming’s uses and evaluations have been in the general management or business policy area [12; 14]. A wide variety of functional and industry-specific games are also available but evaluations of them have been sparse [4; 61. This scarcity, however, has not deterred the adoption of marketing games. Using publishing activity as a proxy for collegiate usage, the number of marketing games has increased by about two per year over the past decade. Undoubtedly, a large number of privately-published games are also being used. As was the case for the adopters of the first policy games, marketing game applications have apparently been made on faith alone. A marketing course, however, appears conducive to gaming because of the practical theories involved and the quantitative interactive outputs obtained. This paper presents a controlled evaluation of a popular marketing game in a marketing principles course application. BACKGROUND A number of studies have employed marketing games but learning effectiveness was usually not central to their interests. Philippatos and Moscato [9; 10; 11] basic- ally Investigated the relationship between information usage and company operations. In total, information availability or unavailability did not alter output performance in any appreciable manner. Rowland and Gardner [13] used MARKSIM [5] in a marketing principles course to determine hierarchical effects on learning results as well as the game’s use as an organizational research laboratory. Looking only at their study’s learning aspects, a high correlation existed between a student’s grade-point-average and game performance although a modest negative correlation existed between game performance and course grades. PROBLEM STATEMENT An experimental design tested the efficacy of the lecture/game teaching format versus lecture/cases in meeting the objectives of a large marketing principles section at The University of Tulsa. Course objectives included the (1) mastery of specific knowledge, and (2) analysis of unstructured problems. The general hypothesis for testing the marketing game’s power was: H1: Students in the lecture/game group will learn more than students in the lecture/case group. Student interest is an additional learning factor of interest. Came adopters have often cited student enthusiasm and the game’s unique learning environment as major simulation benefits. Accordingly, a second hypothesis was generated: H2: Students in the lecture/game group will express more positive course attitudes. METHODOLOGY Students in one large marketing principles section were randomly assigned to teams in two laboratory sections. Teams in one laboratory section (n = 31) analyzed marketing cases while teams in the other laboratory (n = 32) played Day and Ness’ Marketing in Action [3] game. The case group performed oral and written analyses of four cases and the game group prepared four written assignments on the simulation. Each laboratory section met with the instructor nine times in addition to attending weekly common lecture and discussion meetings. The same instructor served all groups. A pre-test post-test design was employed. Specific knowledge was determined through objective test questions while unstructured problem-solving ability was tested through essay questions and short case analyses. The pre-test was administered during the semester’s first week and the post-test served as the course’s final examination. Three intermediate examinations were also administered every fifth week during the semester. Student attitudes were obtained from a standardized university- administered student evaluation questionnaire applied during the semester’s twelfth week. RESULTS Table 1 displays the summary results of overall learning scores. Pre-test scores are identical demonstrating initial group equivalency. Post-test scores reveal statistically significant (p < .01) improvements in overall knowledge levels for both groups but that the improvement was greater for the case group (p < .01). Table 2 reveals that a degree of within-test difference existed as the case group performed better on the examination’s objective questions. An analysis of the three interim tests showed no discernable pattern of change. The game group initially out- performed the case group, the case group performed better on the second test, and scores were equal on the third test. The overall student evaluation questionnaire revealed no differences between the two groups. These evaluations yielded ordinal data thus t-tests could not be performed. An examination of extreme scores yielded the differences presented in Figure 1 DISCUSSION Developments in Business Simulation & Experiential Learning, Volume 11, 1984 27 This study has found that the marketing game studied increased knowledge levels but no more so than did the cases assigned to the comparative group. Although the game group initially outscored the case group, this initial superiority disappeared. The student evaluation questionnaire also revealed that the case group TABLE 1 SUMMARY TEST SCORES Measure Group Before After Game Case 41 67 n.s. p < .01 p < .01 41 70 TABLE 2 POST-TEST COMPONENT SCORES Question Group Objective Essay Game Case 38 p < .01 41 29 n.s. 29 FIGURE 1 SUPERIOR SCORES Game Group 1. Presented historical origins of ideas and concepts. 2. Wrote test questions for which the meanings were clear. 3. Gave tests which allowed students to demonstrate what they had learned. Case Group 1. Utilized class time to achieve course objectives. 2. Encouraged students to ask questions. 3. Encouraged students to express their own ideas. 4. Was fair and impartial in dealing with students. 5. Was available for conferences outside of class. held a more favorable view of the instructor on certain items. Overall he was perceived to be a better utilizer of class time, more supportive, and more available for student counseling. Although the game was successful, it was not superior as hypothesized. Given the complexities and unknowns of the typical experiential learning environment, a number of factors may be involved. Burns and Gentry [1] among others [7; 8] have categorized them as those dealing with what is being taught, the game/task situation, exercise procedures, and participant and instructor attributes. This suggests the following possible explanations for the results found: 1. Marketing in Action may be too complex or too simple for the student population studied. 2. The instructor may be more adept at the case method than acting as an experiential facilitator. 3. The simulation’s workload could have taken time away from studying for the various tests administered during the semester. 4. High game performance may have been perceived as having little impact on the course’s grade or reward system. CONCLUSION It was found that the marketing game helped in increasing student knowledge. The more traditional case approach, however, was perhaps marginally superior to the simulation. Further study should examine the reasons for these results so that the supposed benefits of each approach might be maximized in the specific teaching application being made. REFERENCES [1] Burns, Alvin C. and James W. Gentry, “Moving Toward a ‘Theory’ of the Use of Simulation Games and Experiential Exercises,” in Daniel C. Brenenstuhl and William D. Biggs (editors), Experiential Learning Enters the 80’s, (Tempe: Arizona State University, 1980), pp. 17-20. [2] Cooke, Ernest F. and Thomas A. Maronick, “Simulations Do Increase Learning,” in Barnett A. Greenberg and Danny N. Bellinger (editors), Contemporary Marketing, 1977 Educators’ Proceedings, (Chicago: American Marketing Association, 1977). [3] Day, Ralph F. and Thomas E. Ness, Marketing in Action, (Homewood, Ill.: R.D. Irwin, 1973). [4] Faria, Anthony A., “The Use of Decision Simulations in Management Programs: Current Perspectives,” Simulation & Games, (in review), 1983. [5] Greenlaw, Paul S. and Fred W. Kniffen, MARKSIM: A Marketing Decision Simulation, (Scranton, Pa.: International Textbook, 1964). [6] Horn, Robert E. and Anne Cleaves, The Guide to Simulations/Games for Education and Training, (Beverly Hills: Sage Publications, 1980). [7] Parasuraman, A., “A Framework for Determining the Pedagogical Value of Simulation Gaming: Implications for Future Simulation Gaming Research,” in Daniel C. Brenenstuhl and Samuel C. Certo (editors), Exploring Experiential Learning: Simulations and Experiential Exercises, (Tempe: Arizona State University, 1978), pp. 127-132. [8] Parasuraman, A., “Evaluation of Simulation Games: A Critical Look at Past Efforts and Future Needs,” in Daniel C. Brenenstuhl and William W. Biggs (editors), Experiential Learning Enters the 80’s, (Tempe: Arizona State University, 1980), pp. 192-194. Developments in Business Simulation & Experiential Learning, Volume 11, 1984 28 [9] Philippatos, George C. and Donald M. Moscato, “An Empirical Study of the Learning Aspects of Experimental Business Game Playing,” AIIE Transactions, Vol. 1, No. 4, 1969, pp. 343-348. [10] Phillippatos, George C. and Donald R. Moscato, “Effects of Constrained Information on Player Decisions in Experimental Business Simulation: Some Empirical Evidence,” Journal of the Association for Computing Machines, Vol. 18, No. 1, 1971, pp. 94-104. [11] Phillippatos, George C. and Donald R. Moscato, “Experimental Learning Aspects of Business Game Playing with Incomplete Information about the Rules,” Psychological Reports, Vol. 25, 1969, pp. 479-486. [12] Ramsey, V. Jean and Robert H. Woodhouse, “Toward a Theory of Teaching Business Policy,” in William W. Biggs and David J. Fritzsche (editors), Developments In Business Simulation and Experiential Exercises, (Normal: Illinois State University, 1980), pp. 179-183. [13] Rowland, Kendrith and David M. Gardner, “The Uses of Business Gaming in Education and Laboratory Research,” Decision Sciences, Vol. 4, No. 2, 1973, pp. 268-283. [14] Wolfe, Joseph, “The Effects and Effectiveness of Simulations in Business Policy Teaching Applications,” Academy of Management Review, Vol. 1, No. 2, 1976, pp. 47-56. Table of Contents Volume 11, 1984 Simulation Gaming as a Means of Researching Substantive Issues: Another Look A Further Test of the Group Formation and its Impacts in a Simulated Business Environment Impact of Economic Patterns on Student Performance in Computer Business Simulation Games Majority Fallacy Game with Independent Student Simulation and a Case Introducing the Marketing Channel Laboratory A Comparative Evaluation of a Marketing Game A Study of Comparative Effectiveness of Problem-Solving Technologies The Impact of Hierarchical and Egalitarian Organization Structure on Group Decision Making and Attitudes Risk-Free Decision Making The EX-STRA Export Strategy Game Computer Education for Management Students Developing a Computer Game/Job Simulation to Teach Functional Literacy Skills Experiencing Socialization First Hand: An Experiential Exercise in Organizational Socialization Networking Distributive Versus Integrative Approaches to Negotiation: Experiential learning Through a Negotiation Simulation Managerial Education and the Real World: Foudations for Designing Educational Tools Diagnosing Group Climate to Improve Supervisory Effectiveness Student background as a Factor in Simulation Outcomes: The Collective bargaining Example The Use of Pre-Plays in Management Education Experiencing the Process Debrief: A Workshop ABSEL Megatrend Roots MEGATRENDS for Business Simulation and Experiential Learning The Effects and Consequences of the Megatrends on Simulation Gaming: One View Opportunities for the Future: ABSEL's Role Experiential Learning-Based Discussion vs. Lecture Based Discussion: A Comparative Analysis in a Classroom Setting An Evaluation of the Minitab Package in Teaching Business Statistics Concepts A Path Analytic Study of the Effects of Alternative Pedagogies Developing and Using Weighted Application Blanks: An Experiential Exercise Building Airplanes Individual vs. Group Grade: An Exercise in Decision making A Marketing Plan Exercise: Development of Interteam Cooperation Using a Coordinated Experiential Approach Using Student Experience as the Basis for a Consumer Behavior Learning Exercise Student Evaluations of Instructors: What do Students Believe? A Description of the SOFTCAT Computer Assisted Teaching System Comparisons of Practitioners' and Professors' Perceptions of Business Policy Content and Learning Methods The Perceived Relationship Between Pedagogies and Attaining Course Objectives in the Business Policy Course The Use of Simulation in the Teaching of Business Policy A Research Study on Strategic Decisions in a Business Simulation Strategic Management Decision Making Researched Via Simulation Gaming Using Simulation to Investigate Factors in Competitive Bidding Combining Experiential Learning and management Assistance A Model for Teaching Management Skills Putting Experience Back into Experiential Learning: A Demonstration The Teaching and Behavioral Measurement of Managerial/Organizational Competencies: Developing Experiential Exercises and Simulations A Simulation Game Model for Conglomerates QCLAB - A Microcomputer Laboratory in Quality Control CTSS: A Commodity Trading Simulation System Problem Solving: An Exercise on Learning, Coaching, and Operant Conditioning A Demonstration of the Effects of Feedback as a Category of Reinforcement The Assessment of Feedback and Disclosure in Interpersonal Relations: An Experiential Exercise A Study to Determine Whether the Teaching of Basic Grammar Skills in Business Communication Classes Improves Students' Business Letter Writing Corporate Maladies Through the Eyes of the Memo Writer: A Seldom Used Experiential Tool Executive Bailout at Shake & Spear, Inc. The H.E./L&P Merger Intercultural Nonverbal Communications: An Experiential Exercise The Evolving Business Policies Course - Is Management Gaming the Logical Pedagogy? The Use of Decision Simulations in Management Training Programs: Current Perspectives Humanizing the Business of Medicine: The Use of Simulated Patients to Train medical Students Systematic Integration of Simulation Methods in a Graduate Management Curriculum Modeling Non-Price Factors in the Demand Functions of Computerized Business Using Spacial Relationships to Estimate Demand in Business Simulations Two Algorithms For Redistribution Of Stockouts In Computerized Business Simulations Leadership And Strategic Behavior A Comparison Of Two Business Strategy Simulations For Microcomputers Incorporating Decision Support Systems Into Management Simulation Games: A Model And Methodology Using Micro-Computers To Support The Analysis Of Complex Cases: It's As Easy As 1-2-3 Strategic Formulation Consistent With Pims: A Micro-Computer Application