CAN BUSINESS GAMES EFFECTIVELY TEACH BUSINESS CONCEPTS? Experiential Learning Enters the Eighties, Volume 7, 1980 128 CAN BUSINESS GAMES EFFECTIVELY TEACH BUSINESS CONCEPTS? Ronald D. Klein, Columbus College ABSTRACT A single business policy class of 44 students was divided into two groups. During the class period, the groups met together analyzing cases, all of which were based on domestic firms, and listening to lectures, three of which were on international business issues. Outside of class, the control group played a domestic business game (TEMPOMATIC IV) and the experimental group played the Bowling Green University International Management Game (MMG). An international business oriented pre- and post-test was given to both groups. I t was hypothesized that both groups would improve on the posttest due to the lectures, but that the experimental group would perform significantly better due to their experience with the MNC. The results confirmed the hypothesis: the experiential method was an effective learning medium for international business concepts. BACKGROUND This study was motivated by the existence of two problems. The first revolves around the pedagogic value of simulation games. The second is a common problem faced by many business schools, that of adding more international business to the core program to satisfy AACSB accreditation requirements. Exactly what business games teach, if anything, has been the subject of much controversy. They have been accepted for their ability to generate student interest and enthusiasm and for their seeming relevance to the decision making task. However, much less is known about their ability to convey business theory to students. The contribution to the knowledge gained from a particular course is confounded by knowledge acquired from other sources: lectures, texts, and cases. Isolating contributions by pedagogic sources has been a difficult task. The problem of integrating international business concepts into existing programs is also a very real problem and adds a degree of relevance to this study. The addition of a large body of new material to the core program is a complex undertaking. Should it be added as a separate course? If so, which course does it replace and what are its prerequisites? It would be difficult to lecture to students on international finance without their having a principles course first. The same is true for law, marketing and management. Such a course would require as many prerequisites as a policy capstone course. Another approach is to add chapters to each functional area across the core program. This would require the purging of individual course material to make room. Further, the policy or capstone course would need its international flavor as well. Integration at the capstone level could be accomplished via the simulation game especially if it can be shown that the simulation is a valid learning experience for new concepts. Hence, this study attempts to quantify the ability of a simulation game to transmit new concepts to the student with a minimum impact on lecture content. THE VALIDITY CAP The possible need for further research on learning effectiveness can be demonstrated by a review of the three most recent annual ABSEL Proceedings and latest NASACA Proceedings. In these publications, there were a total of 105 articles covering some aspect of business simulation games. It can be seen from the Simulation Publication Pie, Figure 1, that the largest category of the articles, over thirty percent, were descriptions of newly developed games, none of which, incidentally, were international business games. The second highest category of game publications was that of game philosophy, articles by learned educators describing subjectively the manner in which games should be used, their shortcomings and advantages, and personal experiences with games as used in seminars and classrooms. The next largest was that in which games were used as an evaluator of pedagogic aspects such as the impact of the administrator in gaming effectiveness, group size, and configuration and optimum number of play periods. Most of these studies were objective in nature as were the remainder of the categories. Comparative studies ( case versus game, lecture and case versus lecture and game, etc.), measuring student interest and motivation make up the next largest category with just over twelve percent of the game oriented articles. Also included in this group were empirical comparative studies of teaching effectiveness (game versus case, etc.) using perceived learning as a dependent variable, that is, how much did students feel they learned. In each case the research instrument administered to the players was in the form of a questionnaire which asked players their degree of interest on a Likert type scale. FIGURE 1 THE SIMULATION PUBLICATION PIE Experiential Learning Enters the Eighties, Volume 7, 1980 129 Table 3 examines several critical issues in international business. Of those examined, the experimental group did comparatively better on questions concerning direct foreign investment and product life cycle. This is consistent with their strong showing in the finance functional area. The experimental group did moderately better on comparative advantage and tariffs. The groups were about even on product standardization. The domestic game permitted product improvements for the purpose of differentiation while the MNG did not. Consequently, these results are fairly consistent with what the games stress. The experimental group did outperform the control group in all areas. Table 4 shows some taxonomic comparisons: long versus short questions and the first twenty versus the last twenty. The performance difference on the long versus the short questions can be attributed to at least two possible causes. The experimental group may have been more familiar with the language and hence could understand the questions more easily. Or, because they were more familiar with the subject matter, were more highly motivated to answer the questions. This problem is discussed more fully in the next section. In comparing questions by their position on the test, the experimental group performance remained effectively constant in terms of percent of correct answers when comparing performance on the first twenty versus the last twenty questions. The control group performance dropped during the last twenty questions. This again may be a kind of respondent fatigue due to the relative lack of interest, especially since the performance drop was not present on the pretest. DISCUSSION There are three major points that should be highlighted. During the review of other effectiveness studies which attempted to compare simulation games and other pedagogic media, it became apparent that a possible problem exists in the development of the test instrument used in the cross comparison. For instance, when comparing the text/lecture method to simulation games, questions are normally taken from the textbook instructor’s guide or from review questions at the end of various chapters. The questions are slanted toward the text material. The simulation game, on the other hand, is a very different type of learning experience, one that is not slanted toward questions requiring relatively precise answers. Hence, the test instrument may be biased in favor of the text/lecture method. The development of a fair test instrument may be an impossible task because of the very different learning environments. The design used in this study circumvents this potential problem since no cross comparison was made. Even though the questionnaire was text oriented, both groups were subjected to the same conditions. Hence, performance may have been affected but not in favor of one group. Consequently, the results of this study should be valid from this standpoint. The second problem is also related to the test instrument. It is extremely difficult to make fair comparisons by subject or functional area to isolate where learning takes place. Each business game is unique and each stresses and ignores specific areas of Interest. Although it is possible to select similar games, it is not possible to select games which are identical in all aspects except some specific area of interest. Hence, the results or outcome of the study cannot be Experiential Learning Enters the Eighties, Volume 7, 1980 130 Almost five percent of the cases examined the use of mathematical decision-making support programs and their relationship to the game being supported. Games provide a format for the use of support programs which in turn sharpen the decision-making skills of the players and give the student a chance to utilize the skills learned in the quantitative courses. The Georgia State Business Game, SIM-Q, is specifically designed for use with support programs.(l) In the final category, learning or teaching effectiveness, there were four studies. Brenenstuhl found a positive but not significant difference in learning by an experiential learning group using a combination of lecture and business simulation compared to the control group which used a lecture only. (2) In a comparison of the lecture-case versus lecture-simulation methodologies, Mancuso found no significant differences on the objective tests used to measure learning, although student performance and interest favored the simulation group.(3) In a third study, Catalanello and Brenenstuhl found no significant difference among three groups (lecture and discussion, lecture and other experiential methods, and lecture and simulation) with respect to perceived learning and satisfaction with the course. (4) The final study in this survey was based on a high school student sample in which Pascale found that a simulation group outperformed two other groups. The findings were based on multiple choice, essay questions, and fact recall questions. (5) These mixed findings underscore the need to improve experimental control. Because of the myriad of factors that influence learning in an experiential context, isolation of cause and effect has been a serious problem in validating games as learning tools. RESEARCH DESIGN A capstone business policy course was divided into two groups of students on a matched pair basis using grade point average. The control group listened to the lectures and discussion, reviewed cases, and played Tempomatic IV, (6) a business game which focuses on the task of producing a single product and marketing it in three areas. The experimental group heard the same lectures and worked on the same cases side by side with the control group in the same classroom. However, they played the Bowling Green University Multinational Management Game which also utilizes a single product and is marketed in three countries. (7) In other words, the test subjects were a single class studying business policy. Outside of the class, there were to separate competitions taking place, one in the domestic business arena and the other in the international environment. The games were selected from a group of ten domestic and five international business games because of their similarity in both game configuration and complexity. Very few of the participants had previous international business experience or classes. Any knowledge gained or sensitization to international business concepts and issues would be as a result of experiential learning either directly through class (three lectures devoted to international business during the quarter) and game participation or indirectly by being motivated to increase their knowledge from other sources. To assess the impact of the learning experience, a 65 question multiple choice test covering a broad selection of international business concepts was given simultaneously to both groups during the first and last class sessions. It was hypothesized that both groups would show improvement due to the lectures, but the experimental groups would score significantly higher on the post- test due to their added learning experience. Null hypothesis - Ho: No significant difference between groups on pretest or posttest. Alternative Hypothesis - H1: Experimental groups would perform significantly better on the posttest. RESULTS Because of the small sample size of 28 students, a contingency table analysis was used to analyze the data. The mean test scores are presented in Table 1. There were a total of 65 questions on the data collection in- s t rumen t. TABLE 1 PRE-AND POSTTEST MEAN SCORES BY GROUP Pretest Posttest Experimental 31.1 N = 14 38.8 N = 13 Control 29.6 N = 14 28.5 N = 13 These results show that the experimental group improved their scores by an average of eight points while the control group experienced a drop in mean score of about 1 point. On the posttest the difference between the two groups was significant at the .90 level of confidence. The T-statistic was 3.36 (T> 2.71 at .90 level of confidence). Three additional comparisons are shown below. The first (Table 2) shows that, of the major functional areas examined, students learned comparatively more about international finance, followed in turn by law, marketing and management. The control group out-performed the experimental group on questions concerning international management, though neither group did well in this area. These data were developed by combining the raw score responses of several questions related to each functional area. The raw scores were then translated to the percent of correct answers for each group. Experiential Learning Enters the Eighties, Volume 7, 1980 131 independent of the subject matter. This problem is inherent in experiential research and the experimenter can only minimize the problem by careful comparisons and instrument design. The final problem revolves around the Hawthorne effect. The test instrument was oriented toward international business. The control group’s experiential learning situation focused on the domestic business environment. Consequently, their motivation and interest level may have been affected, particularly on the post-test resulting in a lower performance. This study did show that the experimental group’s performance improved considerably as a result of the experiential learning experience, but the degree of improvement over the control group is in doubt. Future studies should take this problem under consideration. CONCLUSION The primary contribution of this limited study has been to show that simulations provide the participant with a valid learning experience in the area of international business at the capstone level. Administrators can with less intrepedation comply to AACSB guidelines via the simulations game. Secondary contributions include the isolation of some potential problem areas in conducting experiential research perhaps leading to improved experimental designs and test instruments. REFERENCES (1) Nichols, Arthur C. and Brian Schott. SIM-Q: A Business Simulation Game for Decision Science Students. (Georgia State University, Atlanta). (2) Brenenstuhl, Daniel C. “An Experiential Study of Performance in a Basic Management Game.” ABSEL Proceedings: Second Annual Conference. April 1975. (3) Mancuso, Louis C. ‘A Comparison of Lecture-Case Study and Lecture-Computer Simulation Teaching Methodologies in Teaching Minority Students Marketing.” ABSEL Proceedings: Second Annual Conference. April 1975. (4) Catalanello, Ralph S. and Daniel C. Brenenstuhl. “An Assessment of the Effect of Experiential Simulation and Discussion Methodologies Used in Laboratory Sections of An Introductory Management Course.” ABSEL Proceedings; Fourth Annual Conference. April 1977. (5) Pascale, Victor. “Effect of Simulation-Gaming and Lecture-Question on Critical Thinking.” NASAGA Proceedings: 14th Annual Conference. October 1975. (6) Scott, Charles R. Jr. and Alonzo J. Strickland. TEMPOMATIC IV. (Houghton Mifflin Company: Boston). 1974. (7) Hoskins, Wilham R. and Steven Mandell. Multinational Management Game. (Bowling Green State University: Ohio). 1979. Table of Contents Volume 7, 1980 Symbol Recognition and Correlation for Evaluating Decision Making in Computer Aided Business Simulations Polanal: An Experiential Approach to Decision Support The Use of Time Contracts in Formal Education Indexing Simulation Model Response for Gaming Flexibility Moving Toward A Theory of the Use of Simulation Games and Experiential Exercises Use of Simulation Administration to Achieve Pedagogical Objectives Experiential Learning on the Job - A Business Internship Program Toward the Ultimate Experiential Exercise A Learning Through Managing Program Workshop Using Experiential Materials in Industry Training Sponsored Experiential Learning - An Opportunity Problems and Pitfalls of Externally Sponsored Field Research Projects Viewed form an Experiential Learning Perspective A Modular Approach to Experiential Learning: Classroom & Consulting Using Simulation & Experiential Learning In Industrial Settings Terminations: An Experiential Review Agenda Items -- Board of Supervisors' Meeting - Town of Jori The Objective-Setting Interview in MBO: An Experiential Approach The All-Star World Series Team Exercise: an Experiential Learning Exercise Dealing with Various Organizational Behavior Issues Progress Report on global, A Rich Multinational Gaming Environment Computer Simulation: A Tool to Teach Queuing Theory New Technology for Business Games Technological Frontiers in Computer Simulations for Business Education Simulating the Product Life Cycle on Interactive Terminals On Compensatory Demand Functions in Marketing Simulations The Use of Games at Different Levels in a Single Marketing Course to Increase Game Participation Sun Airlines: A Heterogeneous Consumer demand exercise Incorporating a Group Selection Test An Organization Development Approach to Teaching Organization Behavior CBID: Cognitive, Behavioral, and Interpersonal Development - A Skill Development/Social Learning Approach to Management Development Using a Live Case Via Video Tape A Town and gown Approach Development of Student Generated Cases Using Computerized Text Editing and Database Technology Interdisciplinary Approached to Problems in Utilizing Experiential Techniques To Use or Not To Use Experiential Techniques, That's is the Question Forming Participant Teams in Simulation Gaming The Problems of Motivating Students and Clients in Live-Case Projects Problems of Teaching Leadership Skills Through Experiential Techniques Conflict Style Measurement: Antecedent to Change - A Proposal for an Experiential Exercise Demonstration Fundamentals of Simulation for Newcomers An M.B.A. Orientation Simulation for Managing Time and the Areas of One's Life SimNet Workshop: The International Simulation Network Demonstrates Three New Business Games The Use of a Simulation Model in the Planning and Evaluation of Commercial Bank Operations Probability Assessment and Performance in Business Game Simulations WageSim: A Wage and Salary Administration Simulation Grading as a Teaching and Feedback Mechanism: Modifying Student's Self-Perceptions of Performance Can Business Games Effectively Teach Business Concepts? Development of Multiple Value Orientations in Conflict Resolution Behavior: An Experiential Teaching Paradigm in Labor-Management Relations Course Designing a Competency-Based Peer Assessment Scale for the Evaluation fo Teaching in Higher Education A Method for Evaluating Information for the Equipment Replacement Decision: An Application of Monte Carlo Simulation Simulation: A Method of Appraising Communication Networks in Managerial Decision Making An Evaluation of In-Class Student Involvement Evaluation of the SBI Program from an Experiential Viewpoint: Focus on the Student Differential Predictors of Academic Performance for White and Non-White Samples The Manager's Dilemma: An Unobtrusive Measure of the managerial Sex-Role Stereotype Are Computer Simulations Sexist? The Effect of Group Size on Attitudes Toward the Simulation Associations Between Individual Cognitive Processing Variables and Business Game Performance and Play Students' Perceptions on Managerial Functions After Exposure to Either the Case Method or a Simulation What Business Students Learn from Finance Simulations Attitude Toward Experiential Exercises, The Student-Teacher Relationship, Student Psychological Types, and Performance An Example of How to Design a Research-Based and Classroom-Effective Organizational behavior Exercise Some Issues in Game Design Untested Hypotheses: An Approach to Experiential Learning Evaluation of Simulation Games: A Critical Look at Past Efforts and Future Needs Is Self-Perception Predictable? - Some Laboratory Results An Exercise in Conflict-Handling Behavior The Relationship Between Group Size and the Learning Curve Effect in a Gaming Environment Learning About Organizational Management Through Organizational Management: Closing the Gap Strategies, Managerial Approaches, and Decision Making in a General Management Simulation A Comparison and Evaluation of Similar and Dissimilar Group Scenarios Generated Using Manual Simulation Games Weaknesses of Research Methods in Experiential and Simulation Studies