A POWERFUL TOOL--FOR WHAT? Developments in Business Simulation & Experiential Exercises, Volume 14, 1987 197 A POWERFUL TOOL--FOR WHAT? William R. Soukup, University of San Diego Gary G. Whitney, University of San Diego ABSTRACT Much of the research on simulation has probed the questions of whether or riot simulations should be used, and if so, how they should be administered. This paper reports exploratory research based on the proposition that simulation (and other teaching techniques, both experiential and passive,) may be much more useful for some learning objectives than for others. The simulation, “Cartels and Cut- throats” was implemented in a business policy class, after which the students were asked to reflect on their entire undergraduate program and score six different teaching techniques for each of nine separate learning objectives. The results, while preliminary in nature, strongly suggest that students perceive a significant difference in the effectiveness of teaching methodologies, and further, that the effectiveness of all of the techniques is very much a function of the students learning objectives. INTRODUCTION Research on the use of simulations has focused on answering one of two questions: 1. Whether to use simulations? 2. How to use simulations? Typical of the research that attempts to answer the whether question is Miles, et al, [2] which compared use of simulations with cases in Business Policy Courses and found mild support for the case method (6 of 27 questions on their Skills Acquisition Questionnaire showed significant differences). Typical of the research that attempts to answer the “how” question is that of DiBattista [1, p. 31] who found that simulation teams with structured assignments were “more effective at problem solving and making control effective” than were teams with unstructured assignments. After a thorough review of the simulation literature in which he accepted only research that used a controlled design or other rigorous methodology, Wolfe [3] argued that it is time for a “contingency’ approach to research on the use of simulations. That is, we need to add a third question to the list-- 3. When to use That is, under what conditions and toward what goals are simulations most effective? Based on personal experience, observations, and student feedback, there is ample evidence that simulations are a powerful tool but we are uncertain about how to use this power. Wolfe listed 24 contingency variables in four categories (game design characteristics, administration characteristics, player and group characteristics, and administrator characteristics) that should be considered in researching the effectiveness of simulations. He suggests that simulations may be good for some learning objectives and not effective for others. The contingency approach may offer an explanation for the problem that has plagued researchers up to this point -- many studies found mild support for simulations but “no statistically significant” differences. A likely reason for the inability to find significant differences is that there are too many factors introducing variance into the research design. Similarly in the physical sciences, if a new chemical (independent variable) has a strong growth accelerator on plant XX (dependent variable) and is a strong growth inhibitor on plant YY (dependent variable), a researcher who looks for a correlation between the chemical and its effects on growth of all the vegetation will find none and erroneously conclude that the chemical has no effect. What the researcher needs to do is try the fertilizer on many planes and identify which ones grow better and which do not. This is the exploratory phase of the research. Once he has identified those that are positively influenced then the research can continue in a more structured manner. The current study is an exploration of the impact of six teaching techniques including simulation for their perceived value toward nine learning objectives. METHODOLOGY Cartels and Cutthroats Cartels and Cutthroats (to be released as “The Boardroom” in 1987) is a general management simulation that is appropriate for use in Management or Introduction to Business classes. Using Wolfe’s elements of game construction it is characterized by: * Functional integration. It is a top management game requiring attention to marketing, finance, and production. Developments in Business Simulation & Experiential Exercises, Volume 14, 1987 198 * Low complexity. The internal algorithms are not difficult to understand and it is rather forgiving of mistakes. Only eight decisions per quarter are required. * Moderate face validity. Students accept the small manufacturing company as plausible but the low complexity of the algorithms limits the realism. * Frequent random events. This simulation sends “memos’ requiring decisions to management teams regarding labor threats, machine breakdowns, government actions and new opportunities almost every quarter of the simulation. The Course The Cartels and Cutthroats simulation was administered in an undergraduate business policy (capstone) class of 21 undergraduate students who were in their final (or in a few instances, next-to-final) semester. The simulation accounted for a total of 30% of the students’ course grade; half of that 30% was based on team performance and the other half on written reports concerning the simulation. (e.g., strategic plans and control and evaluation reports). Performance in the simulation was measured by the single variable, ending stockholders’ equity which is equivalent to net profits, since the simulation does not allow dividend payments. Students were permitted to form and organize their own teams. After a brief discussion of the nature of the simulation during two class meetings, the simulation was conducted as an out-of-class exercise - I.e., student teams would meet at times and places of their own choosing, and submit their decisions at the beginning of the class meeting. Two practice decisions were run, after which the simulation was reinitialized and run for a total of eight quarters. For most of the students, Cartels and Cut- throats was the only simulation they had experienced during their undergraduate studies. About a fourth of the class had been involved in a prior simulation experience in a marketing course. The Survey The survey was administered at the end of the course, which for the great majority of the students, meant the end of their undergraduate program. Eighteen of the 21 students completed the survey. The instrument was designed to obtain student evaluations of six different pedagogical techniques separately for each of nine learning objectives. In order to clarify the rating procedure and to facilitate interpretation of the responses, students were asked to assign scores to the teaching methodologies in such a way that the scores would total to 100 for each of the nine learning objectives. Thus each of the students completed the instrument shown in Figure 1. Developments in Business Simulation & Experiential Exercises, Volume 14, 1987 199 RESULTS The first question is whether students can and do discriminate among teaching methods. In this exploratory study, they very clearly did so. The grand means - i.e., the means of technique across all learning objectives ranged from a high of 28.12 for lectures to a low of 2.78 for films, as shown in Table 1. An important related question is whether students perceive teaching techniques differently as a function of their learning objective. Again the students in this exploratory study appeared to discriminate very clearly. The ratio of the highest to lowest score for each technique ranged from slightly under 2 to more than 7 (see Table 2.) The scores assigned to simulation undoubtedly reflected to a great extent the design of Cartels and Cutthroats (which tends to emphasize general management and broad concepts of marketing and finance rather than detailed analysis), since the majority of the students had no other basis for evaluating that technique. Even so, some of the scores appear sufficiently interesting to warrant further investigation. Simulation was scored as being extremely effective for communication skills and group behavior, while simulation scores for general business concepts, analytical techniques, and financial analysis were somewhat lower than expected as indicated in Table 3. In general, students considered the two experiential approaches (simulation and case discussion) most useful in gaining skills such as communication, decision making, and group behavior. On the other hand, the more passive methods (lecture and readings in particular) were regarded as most helpful in acquiring conceptual knowledge (see Tables 4 and 5.) Developments in Business Simulation & Experiential Exercises, Volume 14, 1987 200 Skill development One result that the authors found surprising was the uniformly low scores assigned to films. Not only were films ranked last for every single learning objective, the numerical scores were extremely low - rarely as much as half the score of the next lowest ranking technique. The best score received for films was for learning strategic concepts, but the mean score there was only 4.72, while even problems, understandably the second-lowest scoring technique for this objective, scored 7.94. The authors have no explanation for these remarkably low scores for films, since no follow-up has yet been undertaken to explain these data. CONCLUSIONS It must be emphasized again that this study was strictly exploratory, and no conclusive statements can be made regarding the effectiveness of any particular teaching technique. The results generally do suggest, however, that further research along these lines could contribute greatly to more meaningful use of simulations and other experiential techniques and to the overall effectiveness of business education. More specifically, the study supports the following conclusions: 1. Students do in fact appear to discriminate among the teaching techniques to which they are exposed, and do so in terms of a number of very specific learning objectives. 2. Experiential and passive techniques are both highly valued by students, but for different purposes. Techniques that are regarded by students as very effective for some purposes are viewed as being quite weak for other objectives. 3. Thus within the limits of an exploratory effort, the results of this study strongly support Wolfe’s (1985) contention that a contingency view should be adopted in further research on the use and effectiveness of simulation. The question of when to use simulations - i.e., for which students and with what purposes - appears far more meaningful, albeit more difficult to answer, than the simple question of whether or not to do so. 4. The first three conclusions further imply that considerable attention must also he given to a much broader issue: that effective curriculum and/or course design requires a very careful identification, and perhaps prioritization, of learning objectives for each course. Appropriate teaching techniques can be selected only if the instructor has first developed a rather precise notion of the learning objectives for that course. REFERENCES [1] DIBATTISTA, RON A. (1986) “Using a Business Simulation in the Principles of Management Course-- Learning Outcomes and Perceptions.” Developments in Business Simulation and Experiential Exercises 13, 28-32. [2] MILES, WILFORD, G., BIGGS, WILLIAM D., AND SHUBERT, JAMES, N. (1986) “Student Perceptions of Skill Acquisitions Through Cases and a General Management Simulation.” Simulation and Games. 17:1 7-24. [3] WOLFE, JOSEPH (1985) “The Teaching Effectiveness of Games in Collegiate Business Courses” Simulation and Games 16:3 251-288. Table of Contents Volume 14, 1987 Simulation Gaming as An Experimental Context for the Study of Multicriteria Decision Making How to Multiply Your Management Game Grading the Personnel In-Basket Exercise Simulation: The Players™ Perspective The Acute Susceptibility of Nominal Grouping to Negativity The Relative Value of the National ABSEL Meetings: An Analysis of Perceptions by Faculty and Deans A Survey of Behavioral Labs Used by American Business Schools A Structured Framework for Applying Conceptual Models to Problem Analysis: Hardening Up An Expert System Approach for Teaching Marketing Case Analysis From Theory to Practice: A Model for Teaching Beginning Advertising The Use of a Simple Forecasting Technique During an Interactive Computerized Business Game An Investigation of the Relationship Between Formal Planning and Simulation Team Performance and Satisfaction The Exponential Logarithm as an Algorithm for Business Simulations Learning macroeconomic Theory and Policy Analysis Via Microcomputer Simulation Laptop: A Principles of Marketing Simulation Teaching About Sampling in a Marketing Research Class Integrating Decision Support Systems and Business Games Demand Generation in a Service Industry Simulation: An Algorithmic Paradox ABSEL - At a Crossroads? Research on Predicting Performance in the Simulation An Expert System for Financial Planners "Developing Various Student Learning Abilities Via Writing, the Stock market Game, and Modified Marketplace Game in Beginning Macroeconomics" Professional Activity Reports: Getting back to Basics Œ A Preliminary Investigation "A Comparison of Performance, Attitudes, and behaviors of MBA and BBA Students in a Simulation Environment: A Preliminary Investigation" Decision Styles and Student Simulation Performance: A Replication Personal Power Strategies: An Experiential Exercise LTL I: A Trucking Simulation Game Total Enterprise Business Games: An Evaluation Can the Skill of Management be Taught? Developing Critical Thinking and Effective Communication Meaningful Notebooks and Enhanced Learning Using a Totally Menu-Driven Data base and Decision Support (DSS) Program Using a Computerized Menu-Driven Data Entry and fiWhat-Iffl Program as a Pedagogical Enhancement for Management Simulation Using a Joint Project Involving MBA Marketing Management and Undergraduate Marketing Research Students to Teach Marketing Research Testing the PAGE Technique: Results and Further Developments A Comparison of Student Perceptions with Accepted Expectations for Business Simulations "Conflict Management in Action: Verbal Strategies, Nonverbal behaviors and Conflict Styles" Dramatic Monologues as Surrogates for Experiential Learning Influencing One™s Superior: An Assessment and Exercise Personality Traits and Organizational Cultures: Two One-Hour Experiential Learning Exercises Computype: A Strategic Marketing Game Open System Simulations and Simulation Based Research "Teaching Forecasting to the Masses: Little Decision Support Training, But a lot of Answers" Goal Setting and Performance Evaluation with Different Starting Positions Œ The Modeling Dilemma The Gamesmanship of Pricing: Building Pricing Strategy Skills Through Spreadsheet Modeling Teaching MPR Experientially Through the Use of Lotus 1-2-3 Business Simulation Through Activities of a Manufacturing Company Simulated Stock Broker Teaching Employee Counseling Skills to Management Students: A Simulation Airline: A Strategic Management Simulation Developing Simulations and Experiential Exercises on the Personal Computer: Some Critical Issues A Powerful Tool Œ For What? An Integrative Systems Approach to Simulating Free Enterprise in a Small Business Course in a Business School Profits: The False Prophet The Use of Expert Systems to Develop Strategic Scenarios: An Experiment Using a Simulated Market Environment An Interdisciplinary Workshop: Spreadsheet Modeling and Case Analysis Techniques for Beginning Executive MBA Students An Innovative Approach to an O. D. Class Œ The Creative Interaction The Application of Spreadsheet Software Technology to Complete Taxpayer Elections Experiential Learning as the Cornerstone of an Industry Management Course "Identifying, Exercising and Enhancing Right Brain Skills in the Business Classroom" Utilizing Envisionary Technology in the Business Policy Course: An Experiential Pedagogy Business Simulation in the Policy Course: A Survey of American Assembly of Collegiate Schools of Business The Moving Van: Vehicle to Success of Disaster for the Two-Gender Work Force Oil and Gas Well Investment Analysis Using the Lotus 1-2-3 Decision Support System Team Cohesion Effects on Business Game Performance