DECISION EFFICIENCY AND EFFECTIVENESS IN A BUSINESS SIMULATION Developments in Business Simulation & Experiential Exercises, Volume 8, 1981 263 DECISION EFFICIENCY AND EFFECTIVENESS IN A BUSINESS SIMULATION Kenneth E. Newgren, Florida State University Ralph M. Stair, Florida State University Ralph R. Kuehn, Florida State University ABSTRACT Using a simulation game in an undergraduate Policy class, decision efficiency; as measured by time, was compared to the decisions effectiveness, as measured by a comprehensive objective function. Mo significant correlation between decision efficiency arid decision effectiveness was found, thus supporting the general decision making literature. INTRODUCTION During the late l950's and early 1960’s, computerized business simulations started to be used in business schools. Today, most business schools use one or more games in various courses, and many schools are using business simulation games in the ‘Policy” course. A large amount of data has been collected about the use of business simulation games in the “Policy’ course, but this data has not clearly demonstrated that the business simulation is the best way to allow students to integrate material acquired in previous course work in playing the game. Theoretically, students should be able to use material obtained from previous courses in helping them make better decisions to obtain a better game performance. Generally, the literature presents two kinds of results from the use of such games. First, there are results which confirm or deny the game as a method for improving learning in the policy course. Secondly, there are results which identify individual and group process variables associated with the use of the game. See Wolfe [7] for a detailed history of games and policy courses. The former results reflect a continuing controversy over the game’s effect upon learning due to a lack of conclusive research. For instance, Raia [3] found that the game enhanced learning and heightened students interest in the policy course. Conversely, Rowland and Gardner [6] found no relationship between game performance and the final course grade. Finally, Wolfe [7] found that while neither the game nor the case method produced superior results alone, they did produce superior learning outcomes when combined. The group process results indicate a growing use of games in a laboratory setting. Thus, Wolfe [8] reports that effective policy teams employed a flexible and wide range of responses to the problems presented. While content variables (rigorous analysis) were of minor importance to the successful firm, the decision making atmosphere was found to be the key. Lucas [2] reports that game players were unable to forecast the next play using subjective probability distributions. Finally, Remus [4] and later Remus and Jenner [5] report that while students put a lot of time and effort into the game, there was no significant correlation between self- reported time and effort and final rankings (in the game). The overall purpose of the exploratory study described below was to investigate the relationship between ‘decision efficiency” and ‘decision effectiveness’ vis-à-vis a business simulation. Specific null and alternative hypotheses were: Decision efficiency refers to the amount of resources a group or individual consumes in the process of making a decision. Money, materials, space, computer time, etc. all exemplify resources which need to be consumed efficiently. However, the major resource consumed during the decision process of an academic business simulation is time. The less time required to complete a particular decision, the more efficient the group’s decision process, ceteris parabis. As described below, the decision times for all teams were carefully monitored during each decision. Decision time refers to the amount of time, in minutes, between the distribution of the results (output) of one set of decisions and the team’s submission of its next set of decisions (input). The resulting total decision time served as a measure of each teams decision efficiency. Efficiency is only one measure of a group’s decision performance, the decisions made must also be effective, Le. , do the decisions made produce the desired result? Decision effectiveness can be measured in terms of overall game performance. Other researchers have found return on investment and accuracy of sales forecasts to be accurate measures of a group’s competitive performance (see 8:9). In this study, however, a quantitative objective function was developed to measure overall decision effectiveness. The objective function included measurement of the firm’s profitability, market share, pollution, and production efficiency. Opportunity cost, as reflected by decision time, was included as a minor factor to encourage the use of decision tools outside of class. Furthermore, a strict growth constraint was imposed requiring each firm to double its ending capacity, from a beginning point of 324,000 units to an ending capacity of 650,000-700,000 units. The complete objective function, with the weightings of each of the five areas, is expressed as follows: Performance(l00%) Profits (30%) + Market Share (20%) + Production Efficiency (20%) + Pollution Performance (20%) + Decision Time (10%) A performance score on the basis of 100 points was thus calculated for each firm for each decision. The firm with the largest market share, for example, received the full 20 points. If its market share was 25%, a firm with 20% market share (80% of 25%) received 16 points (80% of 20 points). The cumulative, Developments in Business Simulation & Experiential Exercises, Volume 8, 1981 264 average performance score for the 10 decisions the game ran was used to grade each firm’s competitive performance. THE SIMULATION ENVIRONMENT Data were collected during an undergraduate Business Policy course utilizing the ENSIM* (Environmental Simulation) game. ENSIM is a highly interactive, general management simulation game with environmental constraints. The game was originally developed by Carl Gooding [1], currently at Eastern Carolina University, and later modified by Dan Voich of The Florida State University. In brief, the company being managed is a two-product manufacturing firm for which a total of 27 decisions must be made each month, or run. Fundamental decisions (price, dealer’s margin, marketing expenditures, R&D expenditures, production volume, and raw materials) must be completed for both products as well as financial decisions (bank loans, stock, bonds, notes, dividends, etc.) for the company as a whole. Productivity is also affected by expenditures on labor relations and management training and development as well as the real threat of being fined and/or shutdown for excessive air or water pollution. Unlike most undergraduate Policy classes, students consisted entirely of management majors. Five teams were formed through self-selection and each consisted of four members. This particular class was held in the early evening and thus may have had a greater diversity of backgrounds among students than normal. However, the majority of students were regular, full-time students. After approximately two weeks of ENSIM orientation, a series of three practice decisions was held. During the orientation period, possible decision support tools were introduced as well as familiarization with the general decision process. Starting with the third practice decision, and continued throughout the five weeks of playing the game, decisions were ‘made” in and during class. Printouts with the results of the firm’s previous decision were distributed at the beginning of class. Each firms next set of 27 decisions was submitted to the game’s administrator (the professor) before the group could leave class. When the team had submitted its decision form, the members were free to leave. This format of administering the game provided accurate records of the time spent by each team in its decision process. However, time spent outside of class by team members, individually or collectively, was not included. Outside applications of decision tools, (regressions, break- even graphical analyses, etc.) in preparing for class should help produce a quicker, more effective decision. This aspect of the decision support system will be returned to later in the paper. ANALYSIS, FINDINGS, AND IMPLICATIONS Table 1 provides the performance scores and decision time for each team for each decision throughout the simulation. The null hypothesis presented earlier, that there is no significant correlation between a group’s (firm’s) decision efficiency and the effectiveness of its decision with regards to its competitive performance, was tested by utilizing the correlation coefficient, r, and the t test. Statistical analysis supported the null hypothesis and thus rejected the alternate hypothesis. As reported in Table 2, the r value of -. 5484 and resulting t=-1.135 is not significant at the .05 level. The lack of a significant correlation between decision efficiency and decision effectiveness is consistent with the literature dealing with decision making in general. Specifically, the trade-offs between efficiency and effectiveness apply to group decisions in a business simulation as well as they do in other laboratory and field studies. Support for the null hypothesis, thus reinforces the validity of using business simulations as a research vehicle for exploring various aspects of the decision-making process in general. Additional research is needed in both the efficiency and effectiveness dimensions of group decision making in a gaming environment. No matter how efficient or sophisticated the process in reading a decision, e decision does not necessarily equal a decision. As indicated earlier, no control was made for the amount of time a group spent outside of class in analyzing trends and figures. The inclusion or perhaps standardization of decision techniques is one area where additional research would be helpful. Similarly, control for individuals’ experience and backgrounds, both professionally and in gaming situations, might be matched with analytical/intuitive skills and personal motivations. The last category is particularly important if trying to generalize one’s findings of a student sample to practitioner population. Developments in Business Simulation & Experiential Exercises, Volume 8, 1981 265 A final implication for future research lies in the area of evaluating a firms performance during and at the end of the game. While single measures such as returns on investment and sales forecast error are accurate measures of a firm’s performance, a comprehensive measurement may be more conducive to the learning objectives of a policy course. The objective function utilized in this study encouraged learning through positive reinforcement during the simulation experience, rather than learning through negative reinforcement after the simulation. This informal observation requires formal analysis but may be helpful in meeting policy course objectives at the undergraduate level. Time as a resource and decision making as a skill are two critical aspects of any manager’s life. Because of its dynamic nature, an interactive business simulation is an excellent research tool for investigating these and other factors contributing to optimal decisions. REFERENCES [1] Gooding, C., Decision Making Under Environmental Constraints: A Management Simulation Game fun- published Ph.D. dissertation, University of Georgia, 1976). [2] Lucas, H. C., “Performance In A Complex Management Game,’ Simulation and Games, Vol. 10, No. 1, 1979, pp. 61-74. [3] Raia, A. P., “A Study of the Educational Value of Management Games," Journal of Business, Vol. 39, No. 3, 1966, pp. 339-352. [4] Remus, W. , “Who Likes Business Games?” Simulation and Games, Vol. 8, No. 4, 1977, pp. 469- 480. [5] Remus, W. arid S. Jenner, “Playing Business Games,’ Simulation and Games, Vol. 10, No. 1, 1979, pp. 75- 86. [6] Rowland, K. arid D. Gardner, ‘The Uses of Business Games in Education and Laboratory Research,” Decision Sciences, Vol. It, No. 2, 1973, pp. 268-283. [7] Wolfe, J., “The Effects and Effectiveness of Simulations in Business Policy Teaching Applications,” Academy of Management Review, Vol. 1, No. 2, 1976, pp. 47-56. [8] Wolfe, J. , “Effective Performance Behavior in a Simulated Policy and Decision-Making Environment,” Management Science, Vol. 21, No. 8, 1975, pp. 872-882. Table of Contents Volume 8, 1981 The Promotion: Human Sexuality in Organization The Simulation of Chaos Leadership Development in a Simulated Urban/Suburban (U/S) Environment Tomed: A Computer Game Emphasizing Social Responsibility/or/why the Pop-Top Can? Integrated Brain Activity and the Manager's Job: Utilizing the Troika Model What does R2 Have to do with a Product Management Course? An Analysis of the Effects of Jungian Problem-Solving Style Dimensions on Marketing Decisions Bargaining Behavior in Personal Selling and Buying Exchanges Extending the Simulation Product Life Cycle A Generalized Algorithm for Designing and Developing Business Simulations Operationalizing a Test of a Model of the Use of Simulation Games and Experiential Exercises Pygmalion and Perception: An Experiential Exercise Behavioral Consequences of Reward Regarding Employee Absenteeism in an Industrial Setting: An Operant Conditioning Approach The Simlab Program: The Use of Experimental Simulation and Process Analysis for the Development of Management and Organizations Designs for Research on Simulation-Games, Cases, and Other Experiential Exercises The Role of Students in The Case Method Weaknesses in Research Design Critical Variables in Research on the Educational Value of Management Games Research Questions for Cases Research on the Learning Effectiveness of Business Simulation Games - A Review of the State of the Science The Effects of Valuation Techniques on Holding Cost During Inflationary Periods: A Simulation Exercise The Operations Simulation - A Study in Game Development Applying Guided Design to the Production/Operations Management Course: A Progress Report and Evaluation Terminal Data Entry and Retrieval Systems Simulations and Microprocessors Microcomputers and Related Technology for Simulation Gaming Microcomputers - A New Technology for Innovations in Business Simulations Microprocessor Controlled Interactive Video Simulation Business Game Design: From Theory to Practice The Success of a Computerized Simulation in Microeconomic Pedagogy The Test Preview Game: Applying the Game Show Format Providing a Real World View of the Personal Function: A Simulation Finding an Effective Means of Teaching Managerial Behavioral Skills: Two Different Experiential Teaching Methods Compared A Management Development Program Based on the Experiential Learning Model Participant Type Differences in Response to Experiential Methods: An Informal Look Preparing Student Groups to Participate in Experiential Group Projects: An Organizational Development Approach An Instrument for the Assessment of Learning Dimensions: A Progress Report of the Learning Dimension Scale (LDS) An Empirical Analysis Relating the Learning Style Inventory to Memory and Logical Ability Teaching Styles in Simulation Experiential Learning Versus Traditional Teaching Styles Student Perceptions of Effective Teaching Behaviors Problems in Evaluation of Experiential Learning in Management Education Students' Perceptions of Learning by Simulation A Relative Evaluation of Experiential and Simulation Learning in Terms of Perceptions of Effected Changes in Students Overview of Computer Based Business Games in Business Policy Classes Behavioral Decision Theory and Business Policy Giving Accounting Students Writing Experience as Job Preparation Getting to First Base with MBO: An exercise for Writing and Evaluating Objectives Dimensions of Conflict in Experiential Learning Consumer Alienation and Perceived Relevance of the Business Simulation Using the Self-Reference Criterion to Simulate Culture in Internationalized Business Course Experiential Learning in a Cross Cultural Setting- The Practice of Simulation Approach to Business Education in Japanese Universities Student' Perceptions of the Use of a Computerized Simulation in Teaching Management Information Systems Using the Case Study Approach to Develop a Microcomputer Bases, Fully Integrated, Data Base Driven, Management Information System The Case Study as a Tool for Organizational Change: Applying the Steel Ax to the Designers of Management Information Systems Toward a Theory of Teaching Business Policy A Case Study in the Use of Experiential Learning (A Management Game Simulation) to Enhance Student Understanding of Strategy Evaluation and Policy Formulation Suggestions for Integration of the Business Administration Core Publishing Opportunities and Requirements for Business Simulation and Experiential Learning Materials How do we Apply Experiential Learning Intercollegiate Case Competitions for M.B.A. Students: Initiation and Implementation Teaching Business Policy and Strategy Using the Incident Process The Learning Co-Op Approach to the Core Policy Course Meeting the Managerial Skill Shortage - Is Academia Up to It? An Empirical Analysis of Experiential Learning Reinforcement International Experiential Learning: Experience is the Best Teacher Encouraging Student Participation During International Academic Programs European Summer Study Program: Can you, Should you, What Does it Take? Travel Seminars in Europe: How can I Direct One? International Experiential Learning: Student Evaluation ABSEL: Empirical Findings on the State of the Association Sensitivity of Performance Scores in Business Simulations Improving the Learning of a Business Simulation Game by Increasing the Process Content Student Participation in Deciding Performance Criteria for Grading in an OB Course: An Exercise and a Case Study Markup for Profit: A Simulated Self-Administered Experience in Retail Pricing The Investment Decision Game: An Experiential Learning Approach to Stock Market Decisions Through Gaming CHIPS: A Marketing Channels management Game External Validation: An Experimental Approach to Determining the Worth of Simulation Games Teaching Performance Appraisal Skills: An Experiential Approach In Support of Experiential learning: Results of a Follow-Up Survey The Introductory Management Course: Taking Theory Application One Step Further Decision Efficiency and Effectiveness in a Business Simulation The Implications of Cognitive Processing Variables and the Complex Decision Simulating the Simulation for Enhanced Player Rationality Simulation/Experiential Learning Audit