IMPACT OF ECONOMIC PATTERNS ON STUDENT PERFORMANCE IN COMPUTER BUSINESS SIMULATION GAMES Developments in Business Simulation & Experiential Exercises, Volume 11, 1984 10 IMPACT OF ECONOMIC PATTERNS ON STUDENT PERFORMANCE IN COMPUTER BUSINESS SIMULATION GAMES William S. Sekely, University of Dayton ABSTRACT This paper reports on a study done to determine if the timing of economic conditions in a computer business simulation game would influence the performance of the student teams. A relationship was found between the pattern of economic cycles and the ability of the student teams to cope with these changes. Further, it was found that the pattern influenced the performance beyond that expected by examining the level of economic activity alone. INTRODUCTION Increasing interest and use of computer business simulation games as supplements to other more traditional approaches in teaching various business courses have led to consideration of the many implications that use of such games have. There have been studies on the learning that students achieve through playing these games N], increased motivation that accrues to students playing such games [6], criteria that can be used to measure student performance in such games [1;5], and many aspects of the validity of such games, both as learning tools and as representation of the “real” business world. [7;1O] This paper presents the results of a study done to measure the impact of the economic environment on the performance of participants in a simulation game. That the economic level could impact the performance of simulation participants has been generally recognized and given explicit consideration both in the construction of games and the evaluation of performance. For example, Biggs points out in discussing the evaluation procedures to use in simulation games: “Game users must be careful to identify the type of environmental conditions which teams face when evaluating performance. If the industry is ever-expanding it may be possible for all teams to perform well, while a recessionary environment may result in only a few teams doing well. These types of conditions can greatly distort results.” [1, p. 196] In addition to the level of economic activity, the actual timing of patterns of economic changes or levels could also have an impact on the performance of student teams. This situation might come about, for example, if an instructor has large classes and must use multiple industries in a single class to avoid too many firms in a single industry. In order to reduce interaction and collusion between industries, different economic patterns may be established which would allow for the same final level of potential sales to occur in all industries, but the timing of these sales to vary. Thus, the instructor would have some basis for evaluating performance between industries, at least to the degree of how much of potential was achieved. The hypotheses that were tested then dealt with the relationships between the level of economic activity and the performance levels of the teams in the computer simulation. More specifically, they were: Hypothesis 1 - There is no relationship between the level of economic activity and the performance level of simulation teams as measured by (a) sales or (b) profits. Hypothesis 2 - There is no relationship between the order of economic level change and the performance level of simulation teams as measured by (a) sales or (b) profits. METHODOLOGY Two sections of an upper division marketing elective course were used. The classes were composed of senior business students and had about thirty students each. A simulation game which had the students acting the part of top level marketing executives was used as a supplementary part of the course. Students were placed in teams of three on a random basis. Each class had two industries A and B, with five teams each. Student teams were randomly assigned to the two industries. All students groups were given ten quarters of history on the simulated industries. All the histories were identical. The game used a decomposition model for primary determination of the industry base demand (i.e. the student decisions impacted their market share and financial performance, but affected base industry demand only slightly). The ten quarters of history had fairly stable economic conditions. To reduce the impact of the product life cycle, the products used were mature ones having relatively flat trend lines. The teams then had ten quarters of decisions to make during the game play. The first two decisions made by all teams were made under identical economic conditions. (These corresponded to the third and fourth quarters of the third year of history.) This was done to allow the teams to achieve some degree of experience in making the game decisions. Those teams in the two industries indicated by “A” then went into a year of economic expansion followed by a year of severe recession. Those teams in “B” industries first went into a year of economic recession followed by a year of economic expansion. The economic indicators were chosen to allow identical total market demand for all industries. RESULTS Students game performance in the simulation was evaluated using multiple criteria including aggregate profits, market share, and aggregate sales. The two major groupings of criteria dealt with profits and sales. Each team was evaluated for their performance only against teams in their own five team industry. However, since the market conditions except for economic activity was the same for all teams in all industries, various rank analyses were Developments in Business Simulation & Experiential Exercises, Volume 11, 1984 11 performed on all twenty teams. This allowed the determination of the impact of both the level of economic activity and the pattern of it. Table 1 shows the results of the two years of simulation play using the different economic patterns. The first two decisions with similar economic levels were filtered out. The two industries are indicated by “A” (good economic conditions followed by poor ones) and “B” (poor economic conditions followed by good ones). The teams from the two classes are indicated by the firms’ numbers. Teams 1 through 5 are from one class and 6 through 10 from the second class. TABLE 1 PERFORMANCE RESULTS OF BUSINESS COMPUTER SIMULATION TEAMS Team Rank Year 1 Sales Year 1 Profits Year 2 Sales Year 2 Profits Cumulative Sales Cumulative Profits 1 A 2 A 7 B 9 B 4 A 7 B 4 2 A 7 A 1 B 4 B 2 A 2 B 9 3 A 1 A 8 B 7 B 9 B 4 B 7 4 A 3 B 4 B 2 B 7 B 9 B 2 5 A 8 B 3 B 3 B 3 A 1 B 3 6 A 4 A 2 A 7 B 1 A 3 B 1 7 B 4 B 7 B 10 B 6 A 8 A 1 8 A 10 B 9 B 1 B 10 B 2 B 6 9 A 6 A 4 B 5 A 3 B 7 B 10 10 B 9 A 3 A 2 B 8 A 4 A 8 11 A 5 B 10 B 6 A 8 B 3 A 7 12 B 2 A 10 A 1 A 1 B 6 B 8 13 A 9 B 6 A 8 B 5 A 6 A 2 14 B 7 B 1 B 8 A 6 B 10 B 5 15 B 6 A 5 A 3 A 2 B 1 A 3 16 B 3 B 2 A 9 A 7 A 10 A 4 17 B 10 A 6 A 6 A 10 A 5 A 10 18 B 1 B 5 A 4 A 9 A 9 A 6 19 B 5 A 9 A 10 A 4 B 5 A 5 20 B 8 B 8 A 5 A 5 B 8 A 9 Differences Between Classes The first areas examined dealt with differences between the two sections of the marketing class. The Kruskal-Wallis Test was used to test for similarity of the rankings of the two classes. This test sums the ranks of the two samples and compares these sums with the following test statistic: T = (12 / N(N+1)) x Σk Ri2 / n - 3(N+l) (1) i = 1 where: N = Σk n i = 1 Ri = The sum of the ranks of sample i. The test statistic is then compared to the Chi-square distribution for significance. In this case with two samples, k- 1, the number of degrees of freedom is 1. If T is greater than the respective Chi-square values, the differences of the ranks are significant at that level. The required values for different levels of significance for all the following tables are: 2.706 significant at the .1 level 3.8141 significant at the .05 level 6.635 significant at the .01 level 7.897 significant at the .005 level 10.83 significant at the .001 level Table 2 shows the results of analysis of the differences between the two classes for the various measures of performance. While class 1 appeared to perform very slightly better in a couple of the measures, there were no areas of where the differences were significant. From this comparison, it can be seen that the two classes were equivalent in their abilities to perform in the simulation game. Developments in Business Simulation & Experiential Exercises, Volume 11, 1984 12 TABLE 2 DIFFERENCES IN SIMULATION PERFORMANCE BETWEEN CLASSES MEASURE OF PERFORMANCE Σ OF RANKS CLASS 1 Σ OF RANKS CLASS 2 T STATISTIC LEVEL OF SIGNIFICANCE Year 1 Sale3 97 113 .363 n.s. Year 1 Profits 99 111 .206 n.s. Year 2 Sales 103 107 .022 n.s. Year 2 Profits 102 108 .051 n.s. Cumulative Sales 96 114 .463 n.s. Cumulative Profits 100 110 .l43 n.s. Differences Between Industries The Kruskal-Wallis Test was also performed on the rank differences between the two industries. These industries were designated by A and B, and differed only in the pattern of economic activity used in the game. In this comparison there were several very significant differences between the ranking of the performances of the teams in the two industries. Table 3 shows the analysis done on the ranking of the various performance measures of the two industries. TABLE 3 DIFFERENCES IN SIMULATION PERFORMANCE BETWEEN INDUSTRIES MEASURE OF PERFORMANCE Σ OF RANKS INDUSTRY A Σ OF RANKS INDUSTRY B T STATISTIC LEVEL OF SIGNIFICANCE Year 1 Sales 62 148 7.686 .01 Year 1 Profits 94 116 .7114 n.s. Year 2 Sales 146 64 9.606 .005 Year 2 Profits 151 59 12.09 .001 Cumulative Sales 95 115 .57 n.s. Cumulative Profits 149 61 11.06 .001 DISCUSSION As might be expected, there was a relationship between the level of economic activity and the performance of the simulation teams. However, the level of economic activity only influenced some aspects of the game play. Sales by firms in the industry having the economic expansion were significantly higher than sales by firms in the recessionary economy. In the first year, eight of the top ten teams in sales came from industry A, the expansionary economy. In the second year, eight of the top ten teams came from industry B, which was the expansionary industry in that year. There was no significant difference between the industries in terms of aggregate level of sales for teams in the two years. This indicates that the total potentials for the two industries were very similar using the complementary economic cycles. The figures supporting these results can be found in Table 2. Even though there was a strong relationship between sales of the firms arid the level of economic activity, this relationship did not extend to the more important measure of a firm’s success, its profitability. Although all firms were profitable in expansionary economies, industry A, which had the initial good economy had only slightly more highly profitable firms than did industry B, which had the initial recession. Six of the firms from industry A were among the top ten firms in terms of profitability during the first year. This did not prove to be a significant difference from industry B. During the second year of play and in total, there was a significant difference in profitability performance between the two industries. Industry B had nine of the most profitable ten firms in year two and eight of the top ten firms overall. Both of these results were significant at the .001 level. What caused the differences in profitability performance between the two Industries? There appear to be several related reasons for the differences between the two industries. First, was the response to turns in economic levels. The teams in industry B, on the average, responded to the downturn at the beginning of year one and the upturn in year two in slightly over one quarter. This means that by quarter two in year one, most of the teams were undertaking cost saving measures and by quarter two of year two were aggressively expanding. On the other hand, firms in industry A reacted almost one quarter slower. It took the average firm almost to quarter three in year one to really expand and to quarter three in year two to retrench. Thus it appears as if firms who had initial problems were much more attuned to the environment and changes In it while the firms who were well off at the beginning tended to be more complacent. The second reason for the differences has to do with the competitive reactions of the firms in the two industries. In industry A there was much more concern with expanding sales and market share, very often with disproportionate expenditures and price cuts. There appeared to be a self-feeding cycle that teams could not break. Teams in industry B, because they had a Developments in Business Simulation & Experiential Exercises, Volume 11, 1984 13 recession so close to the beginning of play, tended to be more conservative and profit oriented, even in the expansionary period. The third reason dealt with differences in interpersonal relations in the teams in the two industries. Through discussion with members of the class, there appeared to be more frustrations and discord among the teams in industry A than those in industry B. It seemed to be easier to accept poor showings and try to work together when the problem3 occurred at the beginning. When things had been going good for a period, there was a greater tendency to try and place the “blame” when results were not up to expectations. SUMMARY The results of the present study confirm the anticipated relationship between level of economic activity and level of game performance by student teams. The null hypothesis with respect to sales level was definitely rejected. Teams do tend to perform better under positive economic conditions. However, the 3ituation was not as clear’ with respect to profits. The null hypothesis could only be rejected in the case of industry B and not A. High level of economic activity did not “hurt” a team’s chances to make higher profits, but did not make it a certainty. The null hypotheses with respect to the impact of pattern of economic activity was rejected for profits and not for sales in this study. Teams in the two industries were essentially the same level of overall sales. The profits of Industry B were significantly higher than that of A, causing the null hypothesis to be rejected with respect to profits. It appears, from this study, that there are several relationships which can occur between economic activity and the results of simulation teams. Thus if cross-. industry analyses are to be made, either for student evaluation purposes or for some other purpose, It is necessary that not only should the level of economic activity be explicitly considered, but also the pattern of the changes. This study has shown, at least for the game examined, that the changes in the pattern may actually be more influential In some areas of measurement than the absolute level. REFERENCES [1] Biggs, W. D., “A Comparison of Ranking and Relation- al Grading Procedures in a General Management Simulation,” Simulation and Games, Vol. 9, June, 1978, pp. 185-199. [2] Biggs, W. D. and Greenlaw, “Role of Information,” Simulation and Games, Vol. 7, March, 1976, pp. 53- 614. [3] Edwards, Keith, Students’ Evaluations of a Business Simulation Game as a Learning Experience, (Office of Education, Report No. R-121 , December, 1971). [14] Greenlaw, P.S. and F. Wyman, “Teaching Effectiveness of Games,” Simulation and Games, Vol. 14, September, 1973, pp. 274-293. [5] Lucas, “Performance in a Complex Game,” Simulation and Games, Vol. 10, March, 1979, pp. 63-74. [6] Orbach, “Motivation for Learning,” Simulation and Games, Vol. 10, March 1979, pp. 29-37. [7] Schellenberger, R. E. and John Keyt, “A Methodology for Assessing the Internal Validity of Business Simulations,” in Lee Graf and David Currie, ed., Developments in Business Simulation & Experiential Exercises, Vol. 10, (Normal: Illinois State University, 1982), pp. 6-8. [8] VanSickle, R. ‘Designing Simulation Games to Teach Decision-Making Skills,” Simulation and Games, Vol. 9, December, 1978, pp. ‘M3-1428. [9] Yantis, Betty and John Nixon, “Interpersonal Compatibility: Effect on Simulation Game Outcomes,” Simulation and Games, Vol. 13, Sept., 1982, pp. 337-3149. [10] Wolfe, Joseph and Richard Roberts, “A Longitudinal Study of the External Validity of a Business Management Game,” In Lee Graf and David Currie, ed., Developments in Business Simulation & Experiential Exercises, Vol. 10, (Normal: Illinois State University, 1982), pp. 6-8. Table of Contents Volume 11, 1984 Simulation Gaming as a Means of Researching Substantive Issues: Another Look A Further Test of the Group Formation and its Impacts in a Simulated Business Environment Impact of Economic Patterns on Student Performance in Computer Business Simulation Games Majority Fallacy Game with Independent Student Simulation and a Case Introducing the Marketing Channel Laboratory A Comparative Evaluation of a Marketing Game A Study of Comparative Effectiveness of Problem-Solving Technologies The Impact of Hierarchical and Egalitarian Organization Structure on Group Decision Making and Attitudes Risk-Free Decision Making The EX-STRA Export Strategy Game Computer Education for Management Students Developing a Computer Game/Job Simulation to Teach Functional Literacy Skills Experiencing Socialization First Hand: An Experiential Exercise in Organizational Socialization Networking Distributive Versus Integrative Approaches to Negotiation: Experiential learning Through a Negotiation Simulation Managerial Education and the Real World: Foudations for Designing Educational Tools Diagnosing Group Climate to Improve Supervisory Effectiveness Student background as a Factor in Simulation Outcomes: The Collective bargaining Example The Use of Pre-Plays in Management Education Experiencing the Process Debrief: A Workshop ABSEL Megatrend Roots MEGATRENDS for Business Simulation and Experiential Learning The Effects and Consequences of the Megatrends on Simulation Gaming: One View Opportunities for the Future: ABSEL's Role Experiential Learning-Based Discussion vs. Lecture Based Discussion: A Comparative Analysis in a Classroom Setting An Evaluation of the Minitab Package in Teaching Business Statistics Concepts A Path Analytic Study of the Effects of Alternative Pedagogies Developing and Using Weighted Application Blanks: An Experiential Exercise Building Airplanes Individual vs. Group Grade: An Exercise in Decision making A Marketing Plan Exercise: Development of Interteam Cooperation Using a Coordinated Experiential Approach Using Student Experience as the Basis for a Consumer Behavior Learning Exercise Student Evaluations of Instructors: What do Students Believe? A Description of the SOFTCAT Computer Assisted Teaching System Comparisons of Practitioners' and Professors' Perceptions of Business Policy Content and Learning Methods The Perceived Relationship Between Pedagogies and Attaining Course Objectives in the Business Policy Course The Use of Simulation in the Teaching of Business Policy A Research Study on Strategic Decisions in a Business Simulation Strategic Management Decision Making Researched Via Simulation Gaming Using Simulation to Investigate Factors in Competitive Bidding Combining Experiential Learning and management Assistance A Model for Teaching Management Skills Putting Experience Back into Experiential Learning: A Demonstration The Teaching and Behavioral Measurement of Managerial/Organizational Competencies: Developing Experiential Exercises and Simulations A Simulation Game Model for Conglomerates QCLAB - A Microcomputer Laboratory in Quality Control CTSS: A Commodity Trading Simulation System Problem Solving: An Exercise on Learning, Coaching, and Operant Conditioning A Demonstration of the Effects of Feedback as a Category of Reinforcement The Assessment of Feedback and Disclosure in Interpersonal Relations: An Experiential Exercise A Study to Determine Whether the Teaching of Basic Grammar Skills in Business Communication Classes Improves Students' Business Letter Writing Corporate Maladies Through the Eyes of the Memo Writer: A Seldom Used Experiential Tool Executive Bailout at Shake & Spear, Inc. The H.E./L&P Merger Intercultural Nonverbal Communications: An Experiential Exercise The Evolving Business Policies Course - Is Management Gaming the Logical Pedagogy? The Use of Decision Simulations in Management Training Programs: Current Perspectives Humanizing the Business of Medicine: The Use of Simulated Patients to Train medical Students Systematic Integration of Simulation Methods in a Graduate Management Curriculum Modeling Non-Price Factors in the Demand Functions of Computerized Business Using Spacial Relationships to Estimate Demand in Business Simulations Two Algorithms For Redistribution Of Stockouts In Computerized Business Simulations Leadership And Strategic Behavior A Comparison Of Two Business Strategy Simulations For Microcomputers Incorporating Decision Support Systems Into Management Simulation Games: A Model And Methodology Using Micro-Computers To Support The Analysis Of Complex Cases: It's As Easy As 1-2-3 Strategic Formulation Consistent With Pims: A Micro-Computer Application