A FRAMEWORK FOR DETERMINING THE PEDAGOGICAL VALUE OF SIMULATION GAMING: IMPLICATIONS FOR FUTURE SIMULATION GAMING RESEARCH Exploring Experiential Learning: Simulations and Experiential Exercises, Volume 5, 1978 127 A FRAMEWORK FOR DETERMINING THE PEDAGOGICAL VALUE OF SIMULATION GAMING: IMPLICATIONS FOR FUTURE SIMULATION GAMING RESEARCH A. Parasuraman, University of Northern Iowa INTRODUCTION Simulation gaming is becoming part of the curricula of an increasing number of business schools. However, only a limited amount of effort has been made to research the effectiveness of simulation games. According to Greenlaw and Wyman: “In summary, the effort and expenditures which have thus far been invested in developing business games have not been justified by the knowledge of specifically what games teach, if anything.” [2, p. 292] Even the limited number of studies researching the effectiveness of simulation games have only led to ambiguous findings. A study by Boseman and Schellenberger [1] found that participation in a management game does not provide increased motivation, interest and learning for students. A review by Pierfy [31 of 22 studies that researched the effectiveness of simulation vis-à-vis other teaching tools, concludes that simulation games are no better than conventional teaching methods in fostering student learning. However, this review also found that simulation games are better in terms of increasing student interest and their retention of learned information. The inability of research studies to conclusively establish the worth of simulation gaming might, at least in part, be due to the nature of the choice and administration of the simulation games that were researched, rather than the games per se. The novelty of the simulation gaming approach to instruction will not, by itself, ensure its effectiveness when used in a course. For instance, Greenlaw and Wyman [2] have suggested that the success or failure of a business game depends on a complex set of factors, including those related to the game itself, the game participants, the game administrator, and the game administration procedure. If simulation gaming is to be as effective as it is popular, there is a great need for a set of guidelines: (a) to help an instructor determine whether or not simulation gaming would be beneficial in a certain course; and (b) to help him/her make the most useful game-related choices so as to minimize the effectiveness of the simulation game in a course suited for its use. This paper will propose, and discuss, a conceptual framework that will lead to a broad set of such Exploring Experiential Learning: Simulations and Experiential Exercises, Volume 5, 1978 128 guidelines, and will provide much needed impetus and direction to future simulation gaming research. A CONCEPTUAL FRAMEWORK RELATED TO THE EFFECTIVENESS OF SIMULATION GAMING The framework to be discussed in this section is intended to be exploratory in nature, aimed at highlighting certain simulation gaming issues that seem to have received scant attention until now. Hopefully, it will help focus the attention of academicians, and the future efforts of researchers, on critical areas related to the choice and use of simulation games. Conceptual Development of the Framework This framework is first of all based on the contention that certain course situations may be more suitable for the use of simulation gaming than others. The appropriateness of simulation gaming for a course may depend on a number of course- related characteristics, such as certain student and instructor characteristics. An instructor should be aware of, and should evaluate, a set of such “independent” characteristics prior to the start of a course, to see if it is a suitable candidate for the use of simulation gaming. Conceptually then, one major part of the proposed framework is an “independent module”, that contains all relevant independent characteristics on which depend the suitability of a course for simulation gaming. If an instructor does determine that it may be worthwhile to use simulation gaming, he or she will still have to make a number of decisions related to choosing an appropriate simulation game and its administration. These decisions--such as the choice of a game with an appropriate level of complexity, the time duration of the game, the grading weight to be assigned to the game portion of the course, etc.-- may be critical to the effectiveness of the game in the course. Thus, the second major part of the framework is a “dependent module” that contains the various game-related decisions. The choices to be made by an instructor for any specific course (regarding the various decisions in the dependent module) will not only depend on the structure of the independent module as applied to the course, but also on the need to make game-related choices that are consistent with one another so as to maximize the game’s effectiveness. Given the structure of the components that make up the independent and dependent modules, the conceptual framework can broadly be depicted as a sequence of steps that a potential game user would have to follow, in order to systematically evaluate the appropriateness of simulation gaming for a course, and to derive maximum benefit from the use of gaming in a course that is considered to be suitable for that purpose. The framework is summarized in Figure 1 as a simple sequence of steps. Exploring Experiential Learning: Simulations and Experiential Exercises, Volume 5, 1978 129 Illustrative Application of the Framework Consider a hypothetical case in which the independent module, referred to earlier, only consists of three relevant components, each of which is assumed to exist at one of two different levels (for any course situation) as shown below: Exploring Experiential Learning: Simulations and Experiential Exercises, Volume 5, 1978 130 The independent module as illustrated above is not very realistic because: (a) in a practical situation a number of additional relevant components must perhaps be identified and included in it; and (b) certain components (such as the instructor’s familiarity with simulation gaming) can exist at more levels than the simple “1 or 0” type configuration shown above. However, the purpose of this section is merely to illustrate a possible methodology for operationalizing and applying the conceptual framework, after it is more fully developed through future research efforts. Let us further assume for the sake of illustration that, in this hypothetical situation, the dependent module only consists of two game-related decisions, each of which is assumed to have only two alternatives as shown below: In order to operationalize the conceptual framework one should also uncover the following types of relationships, based on research: Type I: The generalized relationships that exist between each component in the independent module, and the extent of appropriateness of simulation gaming for a course. Type II: The generalized relationships that exist between each component in the independent module, and each decision in the dependent module, so as to maximize the effectiveness of simulation gaming in a course. Type III: The generalized relationships that exist among various decisions within the dependent module, and could have a bearing on the effectiveness of simulation gaming in a course. Specifically, in this illustrative application, hypotheses of the following kind must be formulated and verified, for each of the three types of relationships just described: Illustrative Hypotheses for Type I Relationships H1: Simulation gaming would be more appropriate for an elective course than for a required course. H2: Simulation gaming would be more appropriate for a course in 2 which students have had at least some prior business background than for a course in which students have had no prior business background. Exploring Experiential Learning: Simulations and Experiential Exercises, Volume 5, 1978 131 H3: Simulation gaming would be more appropriate for a course whose instructor has a high degree of familiarity with simulation gaming than for a course whose instructor has a low degree of familiarity with simulation gaming. Illustrative Hypotheses for Type II Relationships H4: A complex simulation game would be more effective in an elective course than in a required course. H5: A simple simulation game would be more effective in a required course than in an elective course. Illustrative Hypotheses for Type III Relationships H6: When a complex game is used, a relatively low grading weight for it would prove to be more effective than a relatively high grading weight. H7: When a simple game is used, a relatively high grading weight for it would prove to be more effective than a relatively low grading weight. After the formulation and verification of all such relevant hypotheses, the framework will be fully ready for providing guidance to any potential game user. For illustrative purposes, let us assume that the hypotheses H1 through H7 have been verified to be true. Then, a course for which all three components in our illustrative independent module are at level “1” would be most appropriate for the use of gaming. Similarly, a course which is characterized by level “0” for all three independent module components would not benefit much from the use of gaming. Courses characterized by level “1” for one or more, but not all, independent module components would possess some intermediate degree of appropriateness for the use of gaming. Furthermore, if an instructor decides to use simulation gaming, and chooses a complex game (based as an analysis of the independent module as applied to the course), he or she must make sure that too much weight is not assigned to the course. IMPLICATIONS OF THE FRAMEWORK FOR SIMULATION GAMING USERS AND RESEARCHERS Making the suggested conceptual framework fully operational poses a number of challenges for scholars interested in the use of simulation gaming. First of all, there is a need for a more exhaustive identification of the components of the independent module--i.e., the factors that have a bearing on the initial decision to use, or not to use, simulation gaming in a certain course. Secondly, there is a similar need to more fully identify the various decisions that should be included in the dependent module--i.e., all game-related decisions that may be critical to the effectiveness of simulation gaming in a course. Next, there Exploring Experiential Learning: Simulations and Experiential Exercises, Volume 5, 1978 132 is a need to specify the nature of the relationships that may exist between the components of the independent module and various game-related decisions, as well as possible relationships among components within the dependent module. Finally, these hypothetical relationships must be verified through appropriate research studies. Identification of the various components and relationships in the framework will not be easy, and perhaps cannot be accomplished by any one researcher in a short period of time. Efficient development of such a framework in operational form would call for a lot of systematic research focused on various aspects of the framework--through the cooperative effort of a number of researchers, each striving to corroborate, and add to, the research findings of others, so as to gradually (but surely) give shape to the final form of the framework. CONCLUSION Though the simulation gaming approach is gaining popularity as a novel pedagogical tool, there is a great need to develop ways of evaluating its effectiveness when used under different circumstances. There is also a great need to develop sound guidelines for the proper choice and administration of simulation games under different circumstances. The conceptual framework discussed in this paper is an initial step in striving towards filling these critical needs. Eventually, systematic analysis of a course situation based on such a framework can aid potential simulation game users in two ways: (a) it can warn an instructor against using simulation gaming in an inappropriate course, thus avoiding potential waste of time, energy and resources, as well as potential frustration; and (b) it can lead to increased effectiveness of simulation gaming in courses that do appear to be suitable for its use. REFERENCES 1. Boseman, F. Glenn and Robert E. Schellenberger, “Business Gaming--An Empirical Appraisal,” Simulation and Games, Vol. 5, No. 4 (December 1974), pp. 383-402. 2. Greenlaw, Paul S. and F. Paul Wyman, “The Teaching Effectiveness of Games in Collegiate Business Courses,” Simulation and Games, Vol. 4, No. 3 (September 1973), pp. 259-293. 3. Pierfy, David A., “Comparative Simulation Game Research-- Stumbling Blocks and Stepping Stones,” Simulation and Games, Vol. 8, No. 2 (June 1977), pp. 255-268. Table of Contents Volume 5, 1978 Debriefing: The Key to Effective Experiential Learning A Kiss Before Debriefing Debriefing with Serialized Theory Development for TaskTeam Learning Insights Into Debriefing Experiential Learning Exercises The Two-Step Flow of Experiential Learning: A Preliminary Investigation Programmatic Experienced-Based Learning in an MBA Program The New Research Focus: An Analysis of the Simulation Game User An Analysis of Performance in Simulation Games Compared to Performance on Structured Course Criteria: A Case Study Le's Simplify the Administrative Requirements of Computerized Educational Simulations Assessment of Sex Stereotypes Within Task Group Simulation The Use of Program BAYAUD in the Teaching of Audit Sampling The Effectiveness of Progressively Complex Accounting Simulations in Increasing Decision-Making Performance The Natural Learning Project The Application of a US-USSR Trade Simulation to the Reaching of Business Russian The Use of Semi-Autonomous Work Group in Graduate Management Education: An Australian Experience Evaluation of an Experimental Course in Organizational Behavior for Managers of Japanese Banks Berkshire II: An Experiential Decision Making Exercise: An Exercise in Hegelian Inquiry Creating a Responsible Managerial Experience A Framework for Determining the Pedagogical Value of Simulation Gaming: Implications for Future Simulation Gamin Research Emergent Simulations in Administration Courses Measurement of Administrator Role for Feedback on Structure and Goals An Experiment in Assessing the Theoretical Recognition and Application Leaning Skills of Student Case Writers Some Game Information Systems Experience Using Job Control Language (JCL) for Group and Individual Simulation Considerations for the Use of Computerized Business Simulations Assessing the Effectiveness of Learning Styles as Predictors of Performance within Three Distinct Pedogogic Methodologies Learning Style and Performance A Psychometric Analysis of Kolb's Learning Styles Inventory Learning to Identify and Satisfy consumer Wants: A Classroom Game An Experiential Exercise in Product Benefit Segmentation Sales and Sales Management: A Case/Simulation Approach The SBI Case as Experiential Learning The Use of a Live Case in Teaching Organization Theory and management Principles to Graduate Students The Residency in Hospital and health Administration as Experiential Learning The Extended Live Case Approach to Teaching Marketing Research Experiential Learning in Marketing: Student Consultants The Brazil-Columbia Coffee Marketing Negotiations: An Experiential Exercise Aimed at Highlighting the Dynamics Involved in the International Business Negotiation Process Educational Values of Simulation Gaming Business Simulation Re-Revisited Simulation and Experiential Learning as Human Subject Research An Experiential Evaluation of a Didactic - Experiential Apporach for Teaching personnel Management Experiential Learning in Wage and Salary Administration Integrating Structured Experiences into Personnel Simulation Locus of Control and Performance in a Management Simulation Correlations Between Academic Achievement, Aptitude, and Business Game Performance Suggestions for Student Input of Decisions in General Management Simulations An Exploratory Investigation of Student Perceptions of Computer Simulation as an Educational Tool Time to Sell?: An Experiential Learning Approach to Stock Market Decisions through Interactive Gaming