A COMPARISON OF A STAND ALONE VERSION OF A SIMULATION WITH THE TRADITIONAL COMPETITIVE VERSION Developments In Business Simulation & Experiential Exercises, Volume 22, 1995 63 A COMPARISON OF A STAND ALONE VERSION OF A SIMULATION WITH THE TRADITIONAL COMPETITIVE VERSION Jerry Gosenpud and John Wash bush, University of Wisconsin-Whitewater Timothy Scott, Mankato State University ABSTRACT This study compared the traditional, human competitor version of a total enterprise simulation with a version where players compete against programmed competition. Dependent variables were learning and satisfaction. Those who played against programmed competitors learned as much as those playing the traditional version but were less satisfied. INTRODUCTION A game is something very general. There are all kinds, some played &one, some in large groups, some simple, some complex, some requiring a competitive atmosphere, others not. Games have attributes, such as only two people playing chess, originally built into them by choice. With time, these attributes become so ingrained that they are assumed inherent and it is forgotten that their existence is in fact chosen. In total enterprise business games, one such attribute is competitiveness. These games assume that teams play against one another (Keys and Biggs, 1990), and the game makes comparisons across teams regarding performance factors such as profits, revenues, and returns. Total enterprise games usually further assume that these performance comparisons yield game winners and losers, and often the winners make higher grades than the losers. These assumptions are not necessarily true. Golf, for example can be played either by golfers competing against each other for the best score or by a lone golfer playing against the par for the course or against the player’s own past record. Similarly, while business games are often played by players competing against one another, one can program a business game so that, say, a 10% return on investment was good performance, and a 4% return on investment was poor performance. Game designers can also introduce the competitors into the game itself, so that a human player competes against computer-generated competitors whose decision logic is programmed into the computer. The purpose of this paper is to compare (a) players playing against human competitors using the competitive version of Micromatic versus (b) players playing against computer-programmed competitors using the Solo version of Micromatic in terms of learning and satisfaction. Micromatic Micromatic (Scott, Strickland, Hofmeister, and Thompson, 1992) is a competitive total enterprise simulation game that allows 2 to 15 companies, run by individuals or teams, to manage small manufacturing businesses with the same product in the same market. Participants normally play against one another while an instructor administers the problem. The game reports two sets of result’s. First, Micromatic generates individual company operations, marketing, financial, and competitive information reports. From these reports, players make their future decisions. Second, Micromatic generates reports that compare the performance of all companies in the industry using sales, net income, returns, stock price, and earnings per share. Micromatic generates an overall index of performance that instructors can easily translate into a grade. The play-alone Solo version of Micromatic is similar yet different. With Micromatic Solo, a team or individual runs a single company playing against 14 companies that follow computer-programmed strategies. Solo is similar to competitive Micromatic in that human players perform the same actions as they would in the competitive version, making decisions, analyzing reports, and making further decisions based on their analyses. Solo also provides the same competitive performance reports and competitive information reports so human players can see how well they are doing against the programmed competitors. However, Solo Micromatic has a number of important differences: 1. Every participant can be a winner. In the competitive version, even if all players do well, someone will finish in last place. Playing Solo, theoretically, all student teams could finish first in their respective industries. 2. Players can work at their own speed, at convenient times not tied to other teams’ or administrators’ schedules. 3. It is possible to run Solo so that players can redo decisions if they don’t like the results produced by a particular set of decisions. 4. Players can analyze Solos competitors’ computer programmed strategies and figure out how to overcome them. Player results produced by the programmable strategies are less random and more predictable than human behavior. Thus a smart player or team can devise a counter-strategy and damage the overall computer strategy because a humans can react more quickly to changes in the game than the less-flexible computer logic. METHOD Subjects, Research Design, and Procedure In this study, the authors used the two versions of Micromatic as “the game.” In one class, student teams played against other student teams using the competitive version. In the other class, student teams played against computer-generated teams using the Solo version. The subjects for this study were 57 students enrolled of the required undergraduate Administrative Policy course at the University of Wisconsin-Whitewater during the Fall 1993 semester. Section 1 consisted of twenty-five students formed into eight teams who played in eight Solo industries (seven teams with three students, one team with four students). Section 2 consisted of thirty-three students in 11 teams who played in a single competitive industry (nine teams of three students, and one team each of two students and four students). In both sections, 25% the course grade was based on the simulation. Performance in Micromatic was worth 20% of the course grade, and an additional 5% of the course grade was based on peer ratings of team contribution. The grading scheme used to measure performance in Micromatic was the one built into the simulation. All Micromatic industries in both the competitive and Solo conditions were identical with respect to decision factor weights and evaluative criteria. An author of Micromatic suggested that the instructor run the Solo game for 16 quarters (decision periods). There was no practice round. Solo has its own processing program, so a team could run the game and generate all reports including competitive reports by themselves. Teams were told that there was no limit on how many Developments In Business Simulation & Experiential Exercises, Volume 22, 1995 64 times they could rerun a given quarter. Solo students could see their competitive results each time they ran a decision. Therefore students knew how well they did before they turned in their reports, and they could rerun decisions until they became ‘successful” (in their own eyes). Each team was to turn in all reports for four quarters of play every three weeks. Market and competitive conditions varied from industry to industry for Solo students because each time the student or instructor initializes a new industry, the program randomly determines the market and competitive behavior for the industry. The eight teams that used Micromatic Solo faced one of three different programmed sets of conditions. In the competitive condition, students played the simulation for 11 quarters, preceded by a practice round. Every team had to turn in a quarter's decision each week (as opposed to the Solo condition where only reports were turned in). The administrator ran the game and returned competitive reports to the student after each quarters decisions. Learning To measure learning, the researchers developed two forms of a m multiple-choice and short-essay examination. The examinations were constructed using questions and situations routinely confronted by participants in the Micromatic experience. For example, these questions included the determination of costs of goods sold, understanding the consequences of performing or not performing ratio analysis or cash flow projections, and understanding the relationship between plant capacity and marketing expenses. They were extracted from dimensions proposed by simulation administrators, reflecting important simulation learning goats (Gosenpud and Washbush, 1994). The test developers used a common scoring key for an questions to ensure uniformity of measurement. Statistical reliability estimates for the instruments range from .65 to .7. The questions measured analytical, syntheses and application skills of the Bloom Taxonomy (Bloom, 1956). Examples of items in these instruments are contained in the Appendix. Form 1 was administered as a pre-test at the beginning of the semester. Form 2 was administered during finals week, which was a week after the conclusion of simulation play. Learning over the period of play was defined as the difference in percentage score for Form 2 minus the percentage score for Form 1. Neither test counted towards a student’s grade. Satisfaction The authors measured student satisfaction for this study by analyzing student responses to an open-ended question concerning their feelings about the simulation. The question was part of the course evaluation given during the last class period of the semester. Because Solo was being used for the first time at the institution, the researchers asked the Solo students to specifically comment on the lack of structure employed in administering the simulation. Since the question tapping satisfaction was slightly different for each version of the game, statistical comparisons were in appropriate. RESULTS Learning Table 1 shows no significant differences in learning improvement scores (the percentage difference scores) between playing the Solo and competitive versions of Micromatic. These results indicate that neither the Solo nor the competitive game was superior in helping students learn principles associated with Micromatic. Satisfaction The responses to the open-ended simulation evaluation question indicated that Solo students were not satisfied with their experience. Of the 25 students who played Micromatic Solo, 19 students answered the open-ended question (the instructor often asks simulation students to comment on the game during course evaluation, and a 76% response rate to this question is high). Of the 19 responses, 10 respondents were critical, and 6 of these were extremely critical (an unusually high proportion). Three respondents complained specifically about the lack of structure, and two students wanted more explanation (a common complaint from Micromatic players). Most of the respondents with complaints said they learned lithe and spent too much of their time copying disks and backing them up. Apparently, these students ran their decisions over and over again until they got good results and were not aware of what they were learning in the process. On the other hand, 5 students said the game was “fine’ and 2 said they preferred less structure. The researchers also obtained comments from students who played the competitive version of Micromatic. In this situation, the instructor asked for comments but did not prompt for comments about the lack of structure as in the Solo class section. Of the 34 students in the competitive condition, only 13 students responded (a fairly low proportion). Three of those were critical and only one very critical (average proportions). In addition, three students criticized the lack of instructive lectures (a criticism of the instructor, not the game). The rest of the comments were either descriptive or positive, with students saying the game was fun’ or that they “wish they had more quarters to catch up,’ or that “they adapted with time.’ DISCUSSION The fact that Micromatic Solo students learned as much from the simulation as students in the traditional competitive condition suggests first that students can learn what there is to learn from a simulation without a competitive environment of human peers. The data show that students can administer their own simulation, and that they do not need to activity involve instructors in running the simulation and keeping track of industry information. On the other hand, conclusions drawn from these results are only pre Developments In Business Simulation & Experiential Exercises, Volume 22, 1995 65 liminary. Result’s here are based on data from only two sections taught by only one instructor. These results also challenge the idea of comparative grading, which bases team grades on the performance of the winning team (Biggs, 1990). In the Solo condition of this study, comparative grading was not used. The grade distribution for Solo teams was: 4 high “A”s (95 or higher), 1 low “B” (80), 2 ‘Cs (75 and 77), and an F. In the competitive condition, simulation performance grades were more or less curved based on the distribution of computer generated index scores. The grades for students playing competitive Micromatic ranged from 68 to 100, with only two grades of 95 or over. Therefore, learning scores were essentially the same in both conditions despite the fact that half the students playing Solo received simulation performance grades of 95 or over while only two of 11 teams in the competitive condition received such grades. On the surface, the critical comments from Solo students may discourage some instructors who may want to use Micromatic Solo in their classrooms. It is likely that the critical comments in the present study may apply only to the administrator of this studys classes. He performed very little monitoring and teaching about the action-consequences of Micromatic and gave very little help or feedback unless requested. Many of the Solo students felt lost. They were told to work on their own unless they needed help. Many students waited to do their work until very near the deadline, and thus had no time to get assistance. Some students, who were able to stay in third or fourth place for the first half of the exercise without working as hard as they could, were lulled into complacency (in spite of being warned) and became angry and desperate during the more challenging last half of the exercise. The way many of the Solo students approached the game fed fuel to their anger. Solo enabled them to see the results of their decisions instantly, and they could also use multiple disks to make as many trial decisions as necessary to attain good results. Many Micromatic Solo teams ran their problems performing minimal analysis and planning between trials. Probably these students learned as they made their decisions, and the statistical results of the study support the assumption that learning took place. But because some Solo students performed little analysis after each decision, it appears they were not aware that they were learning, and they became angry as a result. On the other hand, in the competitive condition, students could not see the results of their decisions before turning them in, and therefore probably ran fewer forecasts of decisions and performed much more analysis between decisions. These students were probably more aware of their learning and thus less angry about the game. Another reason for the critical feedback in the Solo condition was that criticism was specifically asked for. The instructor made it clear that he was using Solo for the first time and stated that comments on its administration were especially welcome. In addition, the instructor knew that students were angry, so the explicit invitation for comments was an invitation to vent their frustrations. On the other hand, the instructor gave no explicit invitation for comments regarding game administration to the students in the competitive condition. He just asked for comments on the simulation. The ability to make multiple decision sets for a decision period could be an advantage for Solo. In the competitive condition it is easy for a team to give up if they fall too far behind. But in Solo, if a team does poorly in a given quarter, they can always rerun their decisions to improve. Having this opportunity is an incentive to not give up and might facilitate learning for teams who initially perform poorly. CONCLUSION Micromatic Solo provides a desirable alternative method of playing a simulation. Solo allows simulation play without some teams performing competently and still feeling like failures because of their poor competitive position. Solo players are not dependent on the administrator or competitors schedules to obtain the results of playing. Solo allows an instructor to measure the individuals learning without the team effect if students perform poorly against the computer competitors, they can stop playing a problem and start a new problem quickly and easily. One can’t easily start and stop the competitive version of Micromatic. Solo generates 3000 possible different problems, or an instructor can set up a unique individual problem. An instructor can also use Solo for an introductory trial, for practice, or for exams. According to our results, students learned as much using Solo as they did using the more traditional competitive version of Micromatic. This means that students can utilize the advantages of playing Micromatic Solo without the loss of learning. However, there are also potential disadvantages for Solo. Many of the Solo students were unhappy with their experiences and were critical of the game. Many students felt they learned very little by playing. As indicated in this paper’s discussion, the way Solo was administered probably fostered these criticisms. This anger could have been lessened by more instruction, scheduled help sessions, and by asking students to turn in written analyses of their decisions, resulting consequences, and plans for the future along with their decisions. A possible course of action for an instructor would be to employ both versions of Micromatic so that students might play against both human and computer-generated competitors. The two versions would provide complimentary ‘earning experiences. Solo would provide a self-paced problem while the competitive version would provide the fun of beating one’s friends. Both versions of the game provide a challenging environment to practice and to learn management skills. APPENDIX The following are examples of questions in the learning assessment instruments: Multiple-choice item Which of the following will most likely cause the interest rate on new long-term debt to be high? a) Decreasing profitability b) Decreasing leverage c) Decreasing short-term debt d) Decreasing product demand Short essay item An important contributor to profits in a manufacturing firm is keeping cost of goods sold as low as possible. How can a firm do that? Developments In Business Simulation & Experiential Exercises, Volume 22, 1995 66 REFERENCES Biggs, W. D. (1990) Introduction to computerized business management simulations, in J. Gentry (ed.), A Guide to Business Gaming and Experiential Learning. East Brunswick: Nichols/GP Publishing Bloom. B. (ed.) (1956). The Taxonomy of Educational Objectives, Handbook I: The Cognitive Domain. New York: David McKay Gosenpud J. and Wash bush, J. (1994) What simulation users think players should be learning from the simulation. Developments in Business Simulation & Experiential Exercises, 21, 96-99 Keys, J. B. and Biggs, W. D. (1990). A Review of Business Games, in J. Gentry (ed.), A Guide to Business Gaming and Experiential Learning East Brunswick: Nichols/GP Publishing. Scott, T. W., Strickland, A. J, Hofmeister, D. L., and Thompson M. D. (1992), Micromatic: A Strategic Management Simulation Boston: Houghton Mifflin. Table of Contents Volume 22, 1995 Simulation Performance, Learning and Struggle Are Good Simulation Performers Consistently Good? Cognitive and Behavioral Consistency in a Computer-Based Marketing-Simulation-Game Environment: An Empirical Investigation of the Decision-Making Process Chalk & Cheese: Executive Short-Course vs. Academic Simulations Revisiting Personality Bias in Total Enterprise Simulations Are Good Strategies Consistently Good? Investigating the Use of a Computer Simulation as an Effective Pedagogical Tool for the Application of a Strategic Model The Problem of determining an Individualized Simulation's Validity as an Assessment Tool A Simulation Based Analysis of the Value of Information in the Hrebiniak Joyce Typology of Adaptation Relative to Porter's Generic Strategies The Impact of Sales and Income Growth on Profitability and Market Measures in Actual and Simulated Industries A Comparison of a Stand Alone Version of a Simulation with the Traditional Competitive Version Computer-Assisted Gaming of International Business Analyzing Simulations with Computer-Based Programs and Applying the Experience to a Real-World Business A Preliminary Investigation of the Use of a Bankruptcy Indicator in a Simulation Environment Graduates' Views on the Use of Computer Simulation Games Versus Cases as Pedagogical Tools An Analytical Advertising Model Approach to the Determination of Market Demand Dealing with the Complexity Paradox in Business Simulation Games A Prototyping Approach for Incorporating Large Data Bases into Media Planning Simulations: An Example Using Magazine Media Through the Looking Glass, Inc: Superior-Subordinate Personality Type and the Leniency effect Evaluating the Effectiveness of Role Playing Simulation and Other Methods in Teaching Managerial Skills Student and Teacher Perceptions of a Management Simulation Course Performance Evaluation: The Effect on the Propensity to Create Budgetary Slack Management Team Formation for Large Scale Simulations Comparative Static Analysis with the Complete PPA Package: A Strategic Market Planning Tool Consistency in Intent: Learning Objectives at the 1994 Intercollegiate Business Policy Competition A New Twist on an Old Game: The Business Strategy Game: A Global Industry Simulation 3ed Evaluation of Performance in Management Simulation: A Management Coefficients Model Building SimuWorlds: Strategic Management Games of the Future Bulls and Bears: A Stock Market Simulation A Systems Thinking Paradigm and Think Computer Simulation Model of Broadcast and Cable Television Industry Competition Jacket Factory The Sales Management Simulation The Marketing Management Simulation A Demonstration of Promodel Demonstrating A New, Cross-Functional Business Simulation: Vision+ A Cost Chain For The Business Strategy Game Simulation Special Session On Experiential Teaching Compensation Dilemmas: An Exercise In Ethical Decision-Making Organizational Storytelling: Telling Tales In The Business Classroom Evaluating Experiential Training: Case Study And Recommendations The Internship Portfolio: An Innovative Tool For Experiential Learning, Critical Thinking, And Communication An Ethnographic Analysis Of The Pedagogical Impact Of Cooperative Communicating Consumer Behavior: A Long-Term Integrated Exercise Using Personal Consumption Journals And Consumer Analysis Papers An Experiential Paradigm For Teaching Business Problem Solving Developing Leadership Skills The Spss® Student Assistant: The Integration of A Statistical Analysis Program Into A Marketing Research Textbook Negotiating With Your Students Using TQM Principles To Transform Accounting Systems Into An Experiential Exercise Enhancing The Effectiveness Of Outdoor-Based Experiential Training Using Virtual Reality Concepts Case Writing In A Developing Country: An Indonesian Example Experiential Learning Using Focus Groups How Real Should Experiential Pedagogy Be? A Viewpoint From Our Students Reengineering The Internship: A New Approach To Experiential Learning Utilizing The Cosmopolitan/Local And Marginal Man Constructs To Measure Students' Propensity For Creativity Developing Experiential Processes For Teaching Quantitative Techniques For Business Team Learning Roles: A Cooperative Learning Technique Creating the Ultimate Small Business Student Experience: Melding Score/Ace with SBI The Development of Trust in Work Teams: The Impact of Touch Some Outcomes of Experiential Learning: How the Cultural Dynamics of Different Countries are reflected in Workplace Norms & Values Incorporation of Job Analysis Results in Various Forms of Selection Interviews Chudesno, Inc.: An Evaluation of an Experiential Training and Development Simulation An Exercise for Exploring the Relationship between Jungian Psychological Types and Organizational Dynamics Partnership: A Radical Approach to Experiential Learning Partnership: A Nice Idea, But How Do I Get Started? Recognizing Discrimination at Work Using Critical Incident Skills Questions to Help Students Become More Successful at Job Interviewing The Video Project Introduction to Psychological Type Theory Come On Down Understanding Facilitation for Development and Continuous Learning: A Micro-Workshop Leadership and Empowerment: An Experiential Exercise in Decision Making Experiential Exercises and Pedagogy Track Workshop: Selecting a Manager for Maquiladora, Inc. Experiential Training In Multi-Cultural Corporate Settings The Role of Facilitation as an Aid to Complete Learning An Experiential Exercise to Illustrate Difference in Information Processing Behaviors and Styles How to Deliver Accessible Survey Results Age Diversity in the Workplace- Family Feud Style Nafta Standoff: A Cross-Cultural negotiating Role-Play Three Strikes and You're Out!: A Downsizing Experiential Exercise