STIMULATING RIGOROUS ANALYSIS IN SIMULATION GAMING Developments in Business Simulation & Experiential Exercises, Volume 12, 1985 26 STIMULATING RIGOROUS ANALYSIS IN SIMULATION GAMING Richard F. Barton, Texas Tech University ABSTRACT A method for incorporating analytical thinking by simulation game players into the model is described. By evaluating the rigor of “external” analysis and then allowing these evaluations to influence consequences output by the model, game administrators can stimulate players to improve their analyses before each decision. INTRODUCTION Over the years many attempts have been reported to Induce simulation game players to participate in the manner the game designer and game user had in mind. Many simulation games, especially those with models processed by computers, can reduce to number games and hence to mere guessing games if additional aspects of game play are not provided. For most computerized simulation games, a certain amount of numbers is required for each decision and players can provide these numbers very quickly by skipping the individual thought processes or the group interaction desired for learning what the simulation experience offers. Obviously, such behavior is not achieving the purposes of game play. Most game administrators have observed this behavior and many take steps to prevent it. Some means for encouraging players to participate well beyond number guessing are: lengthening the time between decisions to allow more deliberation and discussion, requiring statements of goals and objectives before play begins, requiring a formal review of progress at intermediate points during play, requiring teams to keep a “game book” during play, and providing a computer-based decision support system (DSS). DSS’s for simulation game play have taken two forms. One is a DSS that allows players to construct a management information system and write analytical routines using game histories to the current decision [5,6]. The other is a DSS that allows players to “simulate the simulation” by using the game model itself to generate hypothetical performance results [1,2]. The “simulating the simulation” system mentioned above (which was reported at the 1981 Orlando ABSEL meeting) provided players the opportunity to try out their decisions before making them official. Players could hypothesize proposed actions for their own firms and also hypothesize certain competitive actions by other firms in the industry. The system then allowed them to obtain a “simulated” report of what their official game reports might look like if all the hypothesized actions were in fact the ones on which the official simulation was run. Use of this system has produced some interesting player behavior. Sortie player teams merely used the simulation mode to check their analysis. These teams used only a few simulations for each official decision. At the other extreme were teams that did little or no analysis and used the simulation mode as a trial and error machine, depending on the simulations entirely to generate their official decisions. Basing simulation game play on selecting the best result of a large number of trial-and-error simulations clearly was not accomplishing the original purpose of adding rationality to game play. (It did however introduce players to the rational idea of sensitivity analysis.) Instead of reasoning through the features of the gaming model in a way that used the relationships of the model as intended by the game designer and the game user, these players and player teams rather ‘reasoned not’ and substituted fingering the keyboard and inspecting the terminal screen for rational analysis of the situation represented by the current state of the model. Of course, this indictment of certain players is extreme. They must of necessity had to do some thinking about the model and what it represents in order to choose those variables on which to perform their numerous sensitivity simulations prior to recording their official decision. However, performing a large number of trial-and-error runs through the simulation model is a big difference from what most game users intend. Let us take as example of the prediction of the next end-of- period cash balance in the usual business policy type simulation game. This is perhaps something most game users would want players to learn to do. Rational players would first study the income and balance sheet relationships, then predict future levels for assets, liabilities, income and dividends and finally do a source and application of cash analysis that used the players’ understanding of the cash flows in the modeled enterprise. Then they might go to the decision support terminal and check their analysis against an actual running of the model, given the assumptions upon which their cash flow analysis was based. In contrast, the sensitivity analysis players would go immediately to the terminal and start trying various decisions to see what happened to cash levels. Some wouldn’t even think about cash but would merely attend to cash if the balance fell below zero, in which case the model alerted them with an emergency short- term loan and associated extra interest expense. One possibility to stimulate genuine analysis was to remove the DSS. However, for extreme guessers, this would merely reduce the number of trial-and-error decisions to one, i.e. the official decision. Therefore, other means were sought to induce vigor and rationality. USE OF EXTERNAL INPUT TO THE MODEL Fortunately, the computer model used for implementation of the simulate-the-simulation DSS also contained an external input feature intended to extend the simulation experience beyond a numbers-game level by incorporating qualitative input [3,4]. Qualitative judgments by game administrators (or by other judges) during a gaming session may take many forms. What is observed and judged may range all the was from general playing behavior to numerous small special assignments. When placed on a scale, these qualitative judgments are indexes. Indexes may be scaled against an arbitrary standard for noninteracting simulation designs or they may represent relative performance among competing players or teams for interacting designs. The indexing scheme should not distort the verisimilitude of the model. To achieve educational objectives, the behavior indexed should, if possible, also be observed by all participants and the indexes for that behavior publicly posted. Developments in Business Simulation & Experiential Exercises, Volume 12, 1985 27 Flexibility Incorporating qualitative judgments as indexes in a simulation model designed for discrete decision cycles can be independent of the source language used to program the model . Without this indexing method, the administrator may manipulate the data base from decision to decision. However, in this case, the administrator needs to know the operations of the model well enough to achieve the effect desired. The indexing method requires additional variables-- the indexes--as input to the simulation model . The index variables are then programmed as multipliers of one or more input or generated variables to produce the desired alteration of output as a consequence of the value assigned to the index. The participant behavior that is observed and indexed is external to the operation of the model and can be designed and manipulated by the administrator as he wishes without reconstructing the model . However, once a convention is made for a particular model for which the computer reads indexes, this convention restricts the qualitative’ assignments that can be made because indexes will have specific effects. Administrative Procedures Simulation gaming is usually run in discrete decision cycles. Players make decisions that are read by the computer to produce printouts. The printouts are returned to the players, who again make decisions, and so on. If qualitative indexes are to be incorporated at a specific cycle, the behavior to be judged must be concluded soon enough for completion of the judgment process before the next computer run. The administrator may announce indexes before player decisions are finalized, in which case players may adapt their decisions, or he may announce indexes later so players must make decisions without knowing the influence of the indexes. In either case, players should know the consequences of good and bad indexes when the assignment is made. Index Revisions Indexes for a specific decision cycle may apply only to that cycle or, once given, may hold constant for the remainder of play. Another alternative is to allow opportunities for assignment revision and hence improved indexes. This opportunity encourages participants to interact competitively in the behavior that is indexed as well as through the numbers that are their official decisions. If revisions are permitted, administrators should insist that players or teams live with the initially earned index and its consequences for at least one decision cycle before a change is made. For interactive simulation designs, all indexes may be revised if one or more players or teams submits an assignment revision. Conceptually, unlimited revisions of indexed assignments would permit the quality of these assignments to converge to a point where all indexes were equal. The effect of indexes would then cancel out. This never happens because players usually have too much to do during gaming sessions to try more than a few revisions. Experience with allowing revisions of indexed external assignments suggests that player performance cannot be critiqued or publicly revealed. If the assignments are publicly shown players either adapt to any criticism marked on assignment papers or they copy the paper receiving the highest index. How then can player teams receive feedback so they can “harden up’ their analysis for a revision? One way is for the game administrator to develop a check list of content items he thinks should be included in the assignment. Another, is to make a list of content items player teams actually included in their assignments. When such a list (of either type) is revealed immediately after initial index grades for an assignment are announced, players whose teams receive lower grades can then conclude how they might improve their rigor in a revision. Without such feedback, revisions tend to be a repeat of the original approach but with added effort rather than shifts to more rigorous approaches. Preventing Distortions Due to Indexing The effects of indexes should be well understood by administrators so that extreme indexes will not distort the legitimacy of the gaming sessions or the verisimilitude of the model. Preventing distortion by setting upper and lower bounds for indexes depends on the effects built into the computer model. Acceptable ranges may differ from index to index. For interactive simulations, all low indexes may depress the representation of the object system simulated, e.g., an industry in management gaming. On the other hand, all high indexes may accelerate representation of the object system beyond reasonable player expectations. To prevent this, indexes should average out over interactive players or teams to a neutral quantity. One technique for doing this is to program the model to always expect indexes, then have the indexes be uniform and neutral at the first simulation session. The computer model reported here [3] begins simulation cycles with the computer reading indexes of one. Indexes are then given among competing teams so they always average one. However, allowing indexes to average other than one allows additional flexibility. For example, if certain indexes were to average less than one, depressing effects would occur for the entire simulated industry; allowing these indexes to average more than one would bring industry prosperity and assured labor peace. USING QUALITATIVE INDEXES TO STIMULATE RATIONALITY In the particular application of qualitative indexes reported here, there were five indexes. Each affected different simulation model output variables, as shown In Table 1. Assignments to be graded in index form by the game administrator should substantively relate to these effects. For the purpose of stimulating rigorous analysis, assignments were limited to specific game decision variables. These are also shown in Table 1. In order to “grow’ rigorous analysis as assignments progress, assignments were ordered from simplest first to most complex last. This is the order shown in Table 1. TABLE 1 Policy/Strategy Assignments Decision Variables to be Analyzed Output Variables Affected 1. Administrative Operations research Overheads Operations research 2. Employee Fringe benefits per hour Probability of a strike 3. Financial Dividends Short-tern Loans Bonds Capital stock Interest expense Bond issue constraint Stock issue constraint 4. Marketing Price Salesmen Advertising Material inputs Product & D Salesmen Advertising 5. Production Production levels Materials orders Factory capacity Total labor hours Product quality Developments in Business Simulation & Experiential Exercises, Volume 12, 1985 Assignments were done by teams and the index grades affected team decision results. No instructions were given on internal team organization. In order to provide time for revisions, the first assignment was received prior to the first official decision and its index grades entered into the model for the first decision. Thereafter, an index-graded assignment was due before and then entered into each subsequent decision for four more decisions. There remained two more decisions in the simulation run when revisions could be made. The convention was adopted that index grade changes would always be known by players before the grades took effect. Therefore, very shortly into play, the administrator was announcing before each decision index grades on one new assignment and revised grades on one or more previous assignments. Also, new index grades were not entered into the model until after the official decision deadline so that players could not simulate on the DSS the effects of new indexes. This forced players to do a mental analysis of how official results would differ from simulation trials. Effects of Indexes For the affected output variables in Table 1, the specific effects of the indexes were as follows: Administrative index. High administrative policy grades reduced factory and administrative overhead, and vice versa. For example, a grade of 1.50 would help lower total labor hours if operations research had been purchased and also cut overheads in half, while a grade of .50 would increase overheads by 50% and reduce the effectiveness of operations research. Employee index. High employee policy assignment grades lowered probability of a strike, low grades increased it. For example, a low grade of .90 added ten percentage points to a team’s probability of a strike. Financial index. A high grade on financial policy and strategy would relax constraints on bond and capital stock issues and reduce interest expense. For example, a grade of 1.30 would allow 30% more dollar value of bonds and stock shares to be issued and would reduce interest expenses by 30%, while a grade of .70 would have opposite effects. Marketing index. A high grade on marketing strategy increases competitive effectiveness of advertising and salesmen. For example, a grade of 1.40 would make each dollar spent on advertising and salesmen Interact as if it were $1.40, while a grade of .60 would make dollars interact as if they were $.60. Production index. A high grade on production policy and strategy directly reduced total labor hours and also increased product quality. For example, a grade of 1.05 would reduce total labor hours by 5% and increase product quality by 5%, while a grade of .95 would have opposite effects. As can be surmised from the above description of the importance of these indexes, player teams receiving low indexes were motivated to quickly submit revisions to overcome their obvious competitive disadvantage. Occasionally, a team earning a high index would revise to protect its advantage. MAKING QUALITATIVE ASSIGNMENTS RIGOROUS The original purpose of these qualitative indexes in the mind of the game designer was to allow expansion of game details outside the formalities of the game model and also to let the administrator enter into the model nuances of human behavior. To stimulate rigorous analysis this purpose was changed. Player teams were told to limit their assignments (which were two written pages plus exhibits) to ‘hard-core’ analysis of the decision variables shown for each assign-rent in Table 1 and not to do the assignments as given in the book, which asked for imaginative descriptions of policies and strategies. Players were also told not to bring in any concern outside the game rules other than to apply past course learning to the specific game situation. The instruction to do “hard-core’ analysis was often repeated. Special Use of Indexes to Stimulate Revisions In the past, player teams tended to accept the initial grading for indexed assignments without making the effort to improve their grade. This was overcome in two ways. One way was to give a large spread to the grades so that some were very low and some were very high, giving dramatic effects in both directions. In one such case, a team with a very low grade had its overheads almost doubled. (That team revised quickly!) Another way is to announce an automatic reduction in grades for all teams period by period until revisions start to come in. (This is particularly effective for assignments such as employee policy where blanket grade reductions increase each teams probability of a strike several percentage points per period). Of course, blanket reductions distort the industry as discussed earlier, so when revisions started coming in grades were then given to again average one. RESULTS The pedagogical results of using indexes with “hard-core” assignments were dramatically different for all aspects of game play from game runs not using the indexes and from those using the indexes to grade assignments that were descriptive and qualitative in nature. Illustrative indexes and revisions are shown in Table 2 for several assignments. These are drawn from experience with three different groups of MBA students in a Business Policy course. Of interest in Table 2 are the number of revisions and the convergence of the grades. A benefit to players of this procedure is that the substantive “hard-core’ analyses of these assignments are directly relevant to total game play. Hence by stimulating rigorous analysis for specific assignments early in play, the results of these analyses can be used by player teams in making their official decisions and total play becomes more rigorous. For those teams that were initially not so ‘hard-core’ , the revision opportunities stimulated them to “harden up’ their analyses for improved index grades which then not only improved official consequences but improved the quality of their decisions as play progressed. This was especially evident as revisions were submitted based on the current stage of play and not on conditions when the assignment was first given. Use of the DSS changed somewhat due to these index- graded assignments. Instead of haphazard trial -and- error “simulations of the simulation,’ player teams tended to systematically use the DSS to build sensitivity and decreasing returns curves on which they then based their decision analysis. Developments in Business Simulation & Experiential Exercises, Volume 12, 1985 * Adjustments by the game administrator to stimulate revision R Revisions and others dug out old textbooks to refresh themselves on past course work. A feature of this simulation experience is an end-of-game presentation. How teams fared on the index-graded assignments was frequently given as a cause of the results reported during the presentation, with some teams showing how high grades partially caused their high performance and other teams expressing regret about low grades and their consequences or over not having revised low grades. Some concluded that a team could not rank first or second in performance with grades less than one. Others reported team strategies to get high index grades along with strategies for guiding their simulated companies. Among positive comments on an end-of-course questionnaire regarding the index-graded assignments were: ‘Helped rethink how to play the game. “Helped apply past course material to use in an overall company strategy.’ “Caused you to evaluate your thinking on the game.’ “Able to verbalize specifics of strategies.” ‘How to work with numbers or, a report.’ Learned hard-core analysis.” Of course, not all comments were positive. Some negative comments were ‘busy work or “just more case analysis.’ REFERENCES [1] Barton, Richard F. A Decision Support System to Enhance Rationality of Simulation Game Play Journal of Experiential Learning and Simulation [2] ____________. “Simulating the Simulation for Enhanced Player Rationality.’ Association for Business Simulation and Experiential Learning (ABSEL) Orlando, February, 1981: 269-272. [3] . The IMAGINIT Management Game, (Active Learning Lubbock Texas, 1978). [4] ______. “Incorporating Qualitative Judgements into Man-Computer Simulations.” Simulation & Games, Vol. III, No. 1 (March, 1972): 79-78 [5] Courtney, Jr., James F. and Jensen, Ronald L. SLIM: system Laboratory for Information Management. (Dallas: Business Publications Inc., 1981). [6] Sugges, Jr., Peter R. ‘An Evaluation of SLIM (A System, Laboratory for Information Management).” Proceedings of the Ninth Annual Conference of the Association for Business Simulation and Experiential Learning, Phoenix, February, 1982, pp. 38-39. Table of Contents Volume 12, 1985 One Step Beyond Naiveté: Laboratory Simulation of Strategic Management Predicting Performance Over the Course of the Simulation Congruency of Critical Performance Factors in Simulated and Real World Environments: An Exploratory Study Stimulating Learning and Creating Course Involvement Through the Use of Debates A Novel Approach to Writing Texts ABSEL Revisited: In-Class Student Involvement Increases Learning Bus-Sim Micro: An Introductory Business Simulation Demonstration Solutions Diversified: And Experiential Introduction to Management Information Systems Simulating Rigorous Analysis in Stimulation Gaming The Use of Decision Support Systems (DSS) and Operations Research/Management Science (OR/MS) Techniques to Enhance the Learning Experience of Students Participating in Computerized Simulation Problems Associated with the Administrative Use of Student Ratings Enthusiasm Awareness in the Experiential Classroom Participation Expectations of Students in Experiential Settings Using the Computer as a Planning Aid in an Applied Marketing Course Integrating Microcomputers in the Marketing Curriculum Through the Use of Marketing COMPUPROBS Giving Students Experience in Team Report Writing Description and Results of an Experiential Approach in the Teaching of Business Policy Using Real World Experiences as an Experiential Activity: the Page Technique A Guide to Writing Microcomputer Simulations The Development of a Microcomputer Distributed Processing Business Game A Monte-Carlo Approach to Interactive Gaming Single Case Experimental Design as an Alternative to the Conventional Approach Learning Theory and Research Design: How Has ABSEL Fared? Criticisms of the Use of Simulations in Economics: A Rebuttal Using the General Management Game in the Business Policy Course General Management Policy Simulations: Which Games are Popular, and Why do Professors Stop Using Games? Business Policy Content and Teaching Methods: Comparisons of the Hegarty and Summers and Boyd Findings Let's Talk to the Experiential Authors A Computerized Logistics Game for Micros Effects of Contextual Factors on Bargaining Decisions of Experiential Group Representatives The Linking of a Collective Bargaining Simulation and a Business Simulation Game - A Case Study Students are Surrogates in Arbitration Studies AIRWAYS: A Microcomputer Simulation of a Service Industry A Proposed Interactive Inventory Control Simulation Microcomputer Demonstration Exercises for a Course on Creativity and Problem Solving Processing Skills for Teachers, Trainers and Managers Developing the Competencies of Resistance to Stress and Accurate Self-Assessment Raising Elizabeth: Socializing Girls to Aspire to Careers as Managers, Secretaries or Corporate Wives Factors Influencing Perception: Gaining Understanding Through Experience Painless Computer Simulation The Effects of Differential Grade Weights on Business Game Learning levels An Investigation of the Validity of a Recommendation for Experiential Exercise Debriefing Simulation and Experiential Practices of Faculty: A Data-Based Workshop The Live Case Study: Filling the Gap Between the Case Study and the Experiential Exercise A Reactive Microcase Pedagogy Advantages of a Multigame Simulation Course Using the Logistic Function to Score Computer-Supported Games A Simulation Model to Calculate Session Time for Running Simulation Models in a Shared Resource Environment On the Core Size of Small Task Groups and the Cause for the Core Size to Emerge