RESEARCH ON THE LEARNING EFFECTIVENESS OF BUSINESS SIMULATION GAMES-- A REVIEW OF THE STATE OF THE SCIENCE Developments in Business Simulation & Experiential Exercises, Volume 8, 1981 72 RESEARCH ON THE LEARNING EFFECTIVENESS OF BUSINESS SIMULATION GAMES-- A REVIEW OF THE STATE OF THE SCIENCE Joseph Wolfe, University of Tulsa In their classic chapter on research designs in educational settings, Campbell and Stanley [l, p. 2] held that an experiment was “…the only means for settling disputes retarding educational practice, ... the only way of verifying educational improvements, and the only way of establishing a cumulative tradition in which improvements can be introduced without the danger of a faddish discard of old wisdom in favor of inferior novelties. Given the controversy which still hovers around business gaming and the cult quality which many opponents attach to the experiential movement, game proponents should be especially rigorous in the conduct and use of gaming validation studies. A review of ABSEL’s Proceedings issues for the years 1976-1980, however, finds a proliferation of what Campbell and Stanley have termed pre-experimental designs and an almost total absence of true experimental studies. Although pre-experimental studies are necessary and appropriate in the early stages of a developing science, business gaming practice has advanced far past that stage since the late 1950’s. All the studies appearing in the ABSEL Proceedings failed to meet the Criteria For external validity, and more importantly for classroom pedagogical purposes, vet-v few of the studies net the criteria for Internal validity. The use of true experimental designs would control for the invalidating effects of history, maturation, testing, instrumentation, repression, selection, and mortality and would eliminate rival hypotheses for the results obtained. Without the necessary controls, researchers are adding to the confusion regarding the effectiveness of business games while simultaneously losing credibility with those outside the gaming field who review the discipline’s work. The conduct of non-cumulative and uncontrolled research also keeps the gaming field from addressing unresolved questions and from clarifying old truths and wisdoms. My symposium discussion will describe the strengths and weaknesses found in the designs frequently employed by ABSEL members. It will then generate a research agenda for business gaming with the belief that imaginative hypotheses are the prime ingredient for meaningful research and that the research’s design is merely the implementation of an experimental ideal. Problems inherent in educational research will also be discussed as will the specific problems faced by business game researchers. REFERENCES [1] Campbell, Donald T. and Julian C. Stanley, Experimental and Quasi-Experimental Designs for Research (Chicago: Rand McNally & Company, 1963). Table of Contents Volume 8, 1981 The Promotion: Human Sexuality in Organization The Simulation of Chaos Leadership Development in a Simulated Urban/Suburban (U/S) Environment Tomed: A Computer Game Emphasizing Social Responsibility/or/why the Pop-Top Can? Integrated Brain Activity and the Manager's Job: Utilizing the Troika Model What does R2 Have to do with a Product Management Course? An Analysis of the Effects of Jungian Problem-Solving Style Dimensions on Marketing Decisions Bargaining Behavior in Personal Selling and Buying Exchanges Extending the Simulation Product Life Cycle A Generalized Algorithm for Designing and Developing Business Simulations Operationalizing a Test of a Model of the Use of Simulation Games and Experiential Exercises Pygmalion and Perception: An Experiential Exercise Behavioral Consequences of Reward Regarding Employee Absenteeism in an Industrial Setting: An Operant Conditioning Approach The Simlab Program: The Use of Experimental Simulation and Process Analysis for the Development of Management and Organizations Designs for Research on Simulation-Games, Cases, and Other Experiential Exercises The Role of Students in The Case Method Weaknesses in Research Design Critical Variables in Research on the Educational Value of Management Games Research Questions for Cases Research on the Learning Effectiveness of Business Simulation Games - A Review of the State of the Science The Effects of Valuation Techniques on Holding Cost During Inflationary Periods: A Simulation Exercise The Operations Simulation - A Study in Game Development Applying Guided Design to the Production/Operations Management Course: A Progress Report and Evaluation Terminal Data Entry and Retrieval Systems Simulations and Microprocessors Microcomputers and Related Technology for Simulation Gaming Microcomputers - A New Technology for Innovations in Business Simulations Microprocessor Controlled Interactive Video Simulation Business Game Design: From Theory to Practice The Success of a Computerized Simulation in Microeconomic Pedagogy The Test Preview Game: Applying the Game Show Format Providing a Real World View of the Personal Function: A Simulation Finding an Effective Means of Teaching Managerial Behavioral Skills: Two Different Experiential Teaching Methods Compared A Management Development Program Based on the Experiential Learning Model Participant Type Differences in Response to Experiential Methods: An Informal Look Preparing Student Groups to Participate in Experiential Group Projects: An Organizational Development Approach An Instrument for the Assessment of Learning Dimensions: A Progress Report of the Learning Dimension Scale (LDS) An Empirical Analysis Relating the Learning Style Inventory to Memory and Logical Ability Teaching Styles in Simulation Experiential Learning Versus Traditional Teaching Styles Student Perceptions of Effective Teaching Behaviors Problems in Evaluation of Experiential Learning in Management Education Students' Perceptions of Learning by Simulation A Relative Evaluation of Experiential and Simulation Learning in Terms of Perceptions of Effected Changes in Students Overview of Computer Based Business Games in Business Policy Classes Behavioral Decision Theory and Business Policy Giving Accounting Students Writing Experience as Job Preparation Getting to First Base with MBO: An exercise for Writing and Evaluating Objectives Dimensions of Conflict in Experiential Learning Consumer Alienation and Perceived Relevance of the Business Simulation Using the Self-Reference Criterion to Simulate Culture in Internationalized Business Course Experiential Learning in a Cross Cultural Setting- The Practice of Simulation Approach to Business Education in Japanese Universities Student' Perceptions of the Use of a Computerized Simulation in Teaching Management Information Systems Using the Case Study Approach to Develop a Microcomputer Bases, Fully Integrated, Data Base Driven, Management Information System The Case Study as a Tool for Organizational Change: Applying the Steel Ax to the Designers of Management Information Systems Toward a Theory of Teaching Business Policy A Case Study in the Use of Experiential Learning (A Management Game Simulation) to Enhance Student Understanding of Strategy Evaluation and Policy Formulation Suggestions for Integration of the Business Administration Core Publishing Opportunities and Requirements for Business Simulation and Experiential Learning Materials How do we Apply Experiential Learning Intercollegiate Case Competitions for M.B.A. Students: Initiation and Implementation Teaching Business Policy and Strategy Using the Incident Process The Learning Co-Op Approach to the Core Policy Course Meeting the Managerial Skill Shortage - Is Academia Up to It? An Empirical Analysis of Experiential Learning Reinforcement International Experiential Learning: Experience is the Best Teacher Encouraging Student Participation During International Academic Programs European Summer Study Program: Can you, Should you, What Does it Take? Travel Seminars in Europe: How can I Direct One? International Experiential Learning: Student Evaluation ABSEL: Empirical Findings on the State of the Association Sensitivity of Performance Scores in Business Simulations Improving the Learning of a Business Simulation Game by Increasing the Process Content Student Participation in Deciding Performance Criteria for Grading in an OB Course: An Exercise and a Case Study Markup for Profit: A Simulated Self-Administered Experience in Retail Pricing The Investment Decision Game: An Experiential Learning Approach to Stock Market Decisions Through Gaming CHIPS: A Marketing Channels management Game External Validation: An Experimental Approach to Determining the Worth of Simulation Games Teaching Performance Appraisal Skills: An Experiential Approach In Support of Experiential learning: Results of a Follow-Up Survey The Introductory Management Course: Taking Theory Application One Step Further Decision Efficiency and Effectiveness in a Business Simulation The Implications of Cognitive Processing Variables and the Complex Decision Simulating the Simulation for Enhanced Player Rationality Simulation/Experiential Learning Audit