GRADUATES’ VIEWS ON THE USE OF COMPUTER SIMULATION GAMES VERSUS CASES AS PEDAGOGICAL TOOLS Developments In Business Simulation & Experiential Exercises, Volume 22, 1995 83 GRADUATES’ VIEWS ON THE USE OF COMPUTER SIMULATION GAMES VERSUS CASES AS PEDAGOGICAL TOOLS Thomas L. Leonard, Bryant College Nancy H. Leonard, Providence College ABSTRACT In this study, recent marketing graduates were surveyed to determine their perception of whether simulations or case analyses better prepared them for their current job. Results indicated that graduates felt that simulations or simulations together with case analyses better prepared them for their job than case analyses alone. Graduates also rated the simulation experience over case analyses in terms of decision making involvement and providing a learning framework. INTRODUCTION There has been a long-standing debate among professors as to the future of computer simulations as an effective and efficient pedagogical tool for concepts disseminated in strategy courses (Keeffe, Dyson & Edwards, 1993). Proponents argue that simulations reflect reality and enable students to apply the strategic concepts taught in the course. Opponents argue that simulations are tactical and not strategic, and they do not reflect the reality of the strategic management process due to the lack of soft decision- making inputs. Many of these opponents utilize casework as an alternative tool for applying strategic concepts. Our preference in utilizing simulations rather than cases in the marketing strategy capstone course at two eastern universities has led us to wonder what recent graduates think about the relative educational value of simulations versus cases. Recent graduates are in a unique position to judge the educational value of simulations versus cases. The marketing strategy course is still salient in their minds and they are currently employed in the “real world”. This enables these individuals to evaluate that recent course experience in light of its applicability to their current job. In the following section, we will briefly review research which has examined simulations and casework as pedagogical tools and also review literature which addresses the external validity of simulations. LITERATURE REVIEW The two teaching methods most commonly used in senior-level capstone strategy courses are cases and simulations. The case method was introduced to the classroom at Harvard Business School in the early twenties. A case “typically is a record of a business issue which actually has been faced by business executives, together with surrounding facts, opinions and prejudices upon which executive decisions had to depend. These real and particularized cases are presented to students for considered analysis, open discussions, and final decision as to the type of action which should be taken” (Gragg, 1951, p. 1). Simulation games are experiential exercises (usually computer- based) which enable students to compete with each other as members of management teams of simulated companies producing and marketing consumer or industrial goods. In playing the simulation, students acting as management teams make a variety of decisions that have an impact on their company’s operations. The model underlying simulations is interactive so that these decisions must be made in response to and in light of, likely actions and reactions of competitors. Participants set objectives, develop strategy to realize these objectives and create operating policies to ensure that operating decisions support the strategy. Participants make periodical operating decisions for one or more functional areas such as finance marketing and production and integrate the decisions for the purpose of meeting the firm’s overall goals and aspirations. In both casework and simulations, students may be asked to participate individually or as a member of a group or management team. The goal of both teaching methods is to provide students with “real life” experience in individual decision making, as well as group decision making and social interaction if groups are used. A number of internal validity studies were conducted in the early years of business gaming, comparing learning from games with learning from the case method within strategic management-type courses. The studies by Kaufman (1976), McKenney (1962; 1963), Raia (1966), and Wolfe & Guth (1975) found superior results from game-based groups versus case groups either in course grades, performance on concept examinations, or goal setting exercises. Estes and Smith (1979) found superior game-related results in basic management and management ethics courses. Negative findings were reported by Boseman & Schellenberger (1974) and Pearce (1979), but the former study’s general course method and reward system was oriented toward casework and the latter study employed case-oriented scoring protocols. Developments In Business Simulation & Experiential Exercises, Volume 22, 1995 84 Another area which has been studied is the external validity of gaming. In an alumni survey of opinions about the value of their gaming experience, Kaufman (1976) found that alumni had positive feelings toward the gaming experience, but he did not address learning or comparative career success. Wolfe (1976) found strong associative relationships between successful executive game playing and previous career success and Wolfe & Roberts (1986) found in a 5-year follow-up study that an individual student’s game performance was related to that student’s subsequent career mobility and career satisfaction. In another 5-year longitudinal study, however, Norris & Snyder (1980) found no association between a student company’s game results and subsequent career success when measured by number of promotions, proximity to CEO’s of real world companies, or percentage of salary increases obtained. Though the above-cited studies examined the external validity of either simulations or casework, none of these studies compared simulations to casework with respect to external validity. Another issue of external validity which has seldom been addressed is the question of whether or not simulations actually reflect “real life” settings and whether the types of decisions which are made in the simulation are similar to those made in the job setting. In a recent study, Hemmasi and Graf (1992) assessed the educational value of simulation gaming in business policy courses, by comparing students’ and recent graduates’ (termed ‘practitioners’) perceptions of the effectiveness of simulations in enhancing participants’ appreciation of, and skills in, such areas as interpersonal relationships, theory application to real-world problems, planning, decision making/problem solving, conceptualization/integration of business functional issues, and environmental analysis. They also examined whether students and practitioners felt that the skills and concepts learned were or would be helpful in their performance on the job. This is in essence asking whether simulations reflect the real world. These authors found that practitioners, as compared to students: 1) attributed significantly greater overall educational utility to business simulation gaming as a teaching pedagogy, and b) viewed business simulations as particularly more effective in teaching such essential managerial skills as teamwork, planning, and problem solving/decision making. While these authors examined students’ and practitioners’ perceptions of the effectiveness of simulations as a teaching pedagogy, they did not explore how these individuals perceived simulations versus casework. In this study, we will specifically address this question. The research hypotheses to be tested are presented below. Hypothesis 1: There is a significant difference between graduates perceptions regarding whether business simulation gaming versus casework better prepared them for the type of decisions they are or will be making in their job setting. Hypothesis 2: Most students will rate computer simulation games higher than case analyses. METHODOLOGY Sample and Data Collection The sample for the survey was composed of recent marketing graduates (1993 and 1994) from two eastern universities. A requirement for marketing majors at both schools is a senior-level capstone course in marketing. This course typically revolves around case analyses and! or a semester-long computer simulation game. Four different instructors taught this course at school A during the Fall 1992 to Spring 1994 period, two different instructors taught the sections at school B. The specific cases employed varied by professor and semester. The only computer simulation used was BRANDS: A Marketing Game (Chapman, 1992). Teams with three to five members played the game for 12 weekly decision rounds, simulating three years of decision making. In August 1994, 320 questionnaires were mailed to marketing graduates according to the following schedule: 153 questionnaires were returned a response rate of 48%. The questionnaire included a number of single-item classification questions: respondents were queried about their gender, school, date graduated, employment status, and whether they had used cases and! or a computer simulation game in their marketing capstone course. 135 out of 151 (2-non responses) of the respondents were employed. Breakdowns of school, year graduated, and gender show no indication that the sample was not representative: Developments In Business Simulation & Experiential Exercises, Volume 22, 1995 85 128 of 146 graduates reported that they had used case analyses in their marketing capstone course, 88 said that their class played a computer simulation game, and 71 reported using both (One reported using neither cases nor a simulation!?): Variables and Measurement Occupational Preparation. A six-item, seven-point Likert type scale was used to assess respondents’ evaluations of how well their capstone course prepared them for their present careers. Exploratory factor analysis using principal components with a varimax rotation indicated that the six course evaluation items were unidimensional, a single factor emerged: Simulation/Cases Evaluation. Respondents’ evaluations of the computer simulation game, if applicable, and the cases they used, if any, were measured using identical fourteen-item, seven-point Likert-type scales. The fourteen items assessing evaluations of the simulation yielded three dimensions: Developments In Business Simulation & Experiential Exercises, Volume 22, 1995 86 Respectively, the three dimensions seem to represent: 1) a decision making involvement, 2) a learning component, and 3) a time consumption factor. Discarding the multiply loading items, three evaluative indices were created by summing subjects’ scores on items loading on common factors. That is, each subject’s scores for items one, two, three, four, and six were summed to yield his/her decision making involvement evaluation, items nine, ten and eleven yield the learning component evaluation and items thirteen and fourteen, the time consumption evaluation. The multiply loading items (5, 7, 8, and 12) were discarded. Case Analyses Evaluation. To facilitate comparability, items evaluating cases were grouped in a manner identical to that used for simulation evaluations. Distribution summaries of the variables and estimates of their internal consistency are shown in the Table 3. Data Analysis Hypothesis One. To explore the first hypothesis, which method offered the best occupational preparation, the sample was split into three groups according to whether respondents had used cases and/or a simulation in their marketing capstone course (see Table 4). The mean for the ‘cases only’ group is considerably lower than the means for the other groups: An ANOVA indicates that the difference between the means of the ‘cases only’ group and the other two is significant at the p <0.05 level. Hypothesis Two. Our second hypothesis suggests that most graduates would rate simulations higher than case analyses. We investigated the ratings of the 71 students who had indicated that they had used both cases and a simulation in their capstone- marketing course. We compared their ratings of cases and simulation on each of the three evaluative scales: decision-making involvement, learning framework, and time required. As can be seen from the table an overwhelming majority rated the computer simulation higher than case analyses for illustrating an active decision making environment and for providing a learning framework. Testing against null hypotheses of equal numbers rating each teaching method as better, we can conclude that most of our respondents rated the simulation higher than cases at a significance level of p < 0.005 (x2 = 32.3 and x2 = 11.5 with 1 df, respectively). Though more respondents rated cases higher than simulation for time requirements, the ratio was not significantly different from 50/50 (x2 = 0.67). RESULTS AND DISCUSSION Our results indicate that using a computer simulation was well received by former marketing students. Those who participated in a simulation in their capstone course rated the course higher in terms of occupational preparation. Further, those respondents who used both case analyses and a simulation rated the simulation higher for providing an active decision making environment as well as for providing a framework for learning course concepts. Developments In Business Simulation & Experiential Exercises, Volume 22, 1995 87 A final observation worth noting regards the factor patterns observed for the evaluations of case analyses and simulation. As explained above, the factor analysis of the fourteen items evaluating the simulation yielded three dimensions. Based on these dimensions multi-item scales were generated for both the simulation and cases. This procedure was used to ensure comparability of evaluations of cases versus simulation. Interestingly, an exploratory factor analysis of respondents’ evaluations of cases yields only two distinct dimensions. One factor is a ‘time required’ dimension identical to the one for the simulation. The other twelve items load strongly together (see Table 6). Thus, respondents appear to have viewed the simulation experience differently from the case analyses. The underlying dimensionality of their perceptions was different. The simulation experience may have had an extra ‘depth’ to it. Limitations of the Study Response Bias. An obvious confounding factor was bias caused by identifying the investigators in the cover letter to the survey respondents. Several former students wrote personal greetings on the returned questionnaires. Since the investigators’ students participated in simulations in their courses, this general good will may have been reflected in their ratings of simulations. Investigator Bias. Finally, since both investigators are dedicated users of computer simulations, there may be a general bias designed into the present study. REFERENCES Boseman, F.G. & Schellenberger, R.E. (1974) Business gaming: An empirical appraisal. Simulation & Gaming, 5 (4): 383-402. Chapman, Randy (1992) BRANDS: A Marketing Game. Englewood Cliffs, NJ: Prentice Hall. Estes, J.E. & Smith, J.R. (1979) Research on the effectiveness of using a computerized simulation in the basic management course. In S.C. Certo & D.C. Brenenstuhl (Eds.), Insights into experiential pedagogy: 25-28. Tempe, AZ: Arizona State University. Gragg, C.I. (1951) Because wisdom can’t be told (Ref. No. 9-45 1 - 005). Boston: Harvard Business School, HBS Case Services. Hemmasi, Masoud & Graf, Lee A. (1992) Managerial Skills Acquisition: A Case for Using Business Policy Simulations. Simulation and Gaming, 23(3), 298-310 Kaufman, F.L. (1976) An empirical study of the usefulness of a computer-based business game. Journal of Educational Data Processing, 13 (1): 13-22. Keeffe, Michael J., Dyson, David A., Edwards, Robert FR. (1993) Strategic Management Simulations: A Current Assessment. Simulation and Gaming, 24(3), 363-368. McKenney, J.L. (1962) An evaluation of business game in an MBA curriculum. Journal of Business, 35: 278-286. McKenney, J.L. (1963) An evaluation of a decision simulation as a learning environment. Management Technology, 3 (1): 56-67. Norris, D.R. & Snyder, C.A. (1980) External validation of simulation games. Simulation and Gaming, 13, 73-85. Pearce, J.E. (1979) Developing business policy skills A report on alternatives Journal of Educational Technology, 7(4): 361-371. Raia, A.P. (1966) A study of the educational value of management games. Journal of Business, 39: 339-352. Wolfe, J. (1976) The effects and effectiveness of simulations in business policy-making teaching applications. Academy of Management Review, 1 (2): 47-56. Wolfe, J. & Guth, G. (1975) The case approach vs. gaming in evaluation. Journal of Business, 48(3): 349-364. Wolfe, J. & Roberts, R.C. (1986) The external validity of a business management game: A five-year long longitudinal study. Simulation & Gaming, 17 (1): 45-59. Table of Contents Volume 22, 1995 Simulation Performance, Learning and Struggle Are Good Simulation Performers Consistently Good? Cognitive and Behavioral Consistency in a Computer-Based Marketing-Simulation-Game Environment: An Empirical Investigation of the Decision-Making Process Chalk & Cheese: Executive Short-Course vs. Academic Simulations Revisiting Personality Bias in Total Enterprise Simulations Are Good Strategies Consistently Good? Investigating the Use of a Computer Simulation as an Effective Pedagogical Tool for the Application of a Strategic Model The Problem of determining an Individualized Simulation's Validity as an Assessment Tool A Simulation Based Analysis of the Value of Information in the Hrebiniak Joyce Typology of Adaptation Relative to Porter's Generic Strategies The Impact of Sales and Income Growth on Profitability and Market Measures in Actual and Simulated Industries A Comparison of a Stand Alone Version of a Simulation with the Traditional Competitive Version Computer-Assisted Gaming of International Business Analyzing Simulations with Computer-Based Programs and Applying the Experience to a Real-World Business A Preliminary Investigation of the Use of a Bankruptcy Indicator in a Simulation Environment Graduates' Views on the Use of Computer Simulation Games Versus Cases as Pedagogical Tools An Analytical Advertising Model Approach to the Determination of Market Demand Dealing with the Complexity Paradox in Business Simulation Games A Prototyping Approach for Incorporating Large Data Bases into Media Planning Simulations: An Example Using Magazine Media Through the Looking Glass, Inc: Superior-Subordinate Personality Type and the Leniency effect Evaluating the Effectiveness of Role Playing Simulation and Other Methods in Teaching Managerial Skills Student and Teacher Perceptions of a Management Simulation Course Performance Evaluation: The Effect on the Propensity to Create Budgetary Slack Management Team Formation for Large Scale Simulations Comparative Static Analysis with the Complete PPA Package: A Strategic Market Planning Tool Consistency in Intent: Learning Objectives at the 1994 Intercollegiate Business Policy Competition A New Twist on an Old Game: The Business Strategy Game: A Global Industry Simulation 3ed Evaluation of Performance in Management Simulation: A Management Coefficients Model Building SimuWorlds: Strategic Management Games of the Future Bulls and Bears: A Stock Market Simulation A Systems Thinking Paradigm and Think Computer Simulation Model of Broadcast and Cable Television Industry Competition Jacket Factory The Sales Management Simulation The Marketing Management Simulation A Demonstration of Promodel Demonstrating A New, Cross-Functional Business Simulation: Vision+ A Cost Chain For The Business Strategy Game Simulation Special Session On Experiential Teaching Compensation Dilemmas: An Exercise In Ethical Decision-Making Organizational Storytelling: Telling Tales In The Business Classroom Evaluating Experiential Training: Case Study And Recommendations The Internship Portfolio: An Innovative Tool For Experiential Learning, Critical Thinking, And Communication An Ethnographic Analysis Of The Pedagogical Impact Of Cooperative Communicating Consumer Behavior: A Long-Term Integrated Exercise Using Personal Consumption Journals And Consumer Analysis Papers An Experiential Paradigm For Teaching Business Problem Solving Developing Leadership Skills The Spss® Student Assistant: The Integration of A Statistical Analysis Program Into A Marketing Research Textbook Negotiating With Your Students Using TQM Principles To Transform Accounting Systems Into An Experiential Exercise Enhancing The Effectiveness Of Outdoor-Based Experiential Training Using Virtual Reality Concepts Case Writing In A Developing Country: An Indonesian Example Experiential Learning Using Focus Groups How Real Should Experiential Pedagogy Be? A Viewpoint From Our Students Reengineering The Internship: A New Approach To Experiential Learning Utilizing The Cosmopolitan/Local And Marginal Man Constructs To Measure Students' Propensity For Creativity Developing Experiential Processes For Teaching Quantitative Techniques For Business Team Learning Roles: A Cooperative Learning Technique Creating the Ultimate Small Business Student Experience: Melding Score/Ace with SBI The Development of Trust in Work Teams: The Impact of Touch Some Outcomes of Experiential Learning: How the Cultural Dynamics of Different Countries are reflected in Workplace Norms & Values Incorporation of Job Analysis Results in Various Forms of Selection Interviews Chudesno, Inc.: An Evaluation of an Experiential Training and Development Simulation An Exercise for Exploring the Relationship between Jungian Psychological Types and Organizational Dynamics Partnership: A Radical Approach to Experiential Learning Partnership: A Nice Idea, But How Do I Get Started? Recognizing Discrimination at Work Using Critical Incident Skills Questions to Help Students Become More Successful at Job Interviewing The Video Project Introduction to Psychological Type Theory Come On Down Understanding Facilitation for Development and Continuous Learning: A Micro-Workshop Leadership and Empowerment: An Experiential Exercise in Decision Making Experiential Exercises and Pedagogy Track Workshop: Selecting a Manager for Maquiladora, Inc. Experiential Training In Multi-Cultural Corporate Settings The Role of Facilitation as an Aid to Complete Learning An Experiential Exercise to Illustrate Difference in Information Processing Behaviors and Styles How to Deliver Accessible Survey Results Age Diversity in the Workplace- Family Feud Style Nafta Standoff: A Cross-Cultural negotiating Role-Play Three Strikes and You're Out!: A Downsizing Experiential Exercise