USING STUDENT OPINIONS IN EVALUATING RESULTS WITH A BUSINESS GAME Simulation Games and Experiential Exercises in Action, Volume 2, 1975 92 USING STUDENT OPINIONS IN EVALUATING RESULTS WITH A BUSINESS GAME Ralph M. Roberts, University of West Florida Steven E. Field, University of West Florida The use of business games has increased greatly in the last 15 years. Instructors using games are convinced that gaming generates greater motivation and greater learning than traditional teaching methods. Studies attempting to demonstrate the advantages of gaming as a learning device have been generally favorable with some mixed results. The studies suggest that greater student motivation, using games, contributes to the learning process. However, no studies have examined student motivation and learning relative to traditional teaching methods - as reported by students. This paper reports such a study in a teaching situation consisting entirely of playing The Management Game [14]. Students in the study reported greater motivation and learning in the gaming course than in traditional courses. LITERATURE REVIEW The first generally publicized business game originated with the American Management Association in 1956 [19]. Gaming was further popularized in 1958 by “Business Games - Play One” [1] by G. R. Andlinger and gaming spread rapidly into college education. By 1965 Date and Klassen [6] reported that sixty-four of the ninety leading schools of business utilized games. With the rapid adoption of simulations came discussions and studies about the efficiency of gaming as an educational’ device. In 1958 Andlinger [2] stated that for some objectives management simulations have a unique advantage over traditional methods of instruction. Cohen and Rhenman [5] in 1961, while cautioning the reader about the ‘lack of objective evidence, stated a number of educational concepts possibly obtainable through business gaming. In 1962 Stanley [2L1], also noting the lack of objective measurement, stated that management games were interesting and perhaps also educational. Kibbee, Craft and Nanus [13] indicated that games provide a powerful tool for business education and McKenney [15] found that gaming students learned more in 2 of 3 test areas than non- gaming students. Dill and Doppelt [8], in 1963, also reported learning from games. Raia [18], in 1968, tested three hypotheses about gaming and found significantly increased learning and motivation but not significantly better student attitudes. In 1968 Dolbear et al. (9) ported that a macroeconomics game economically covered a number of topics and Shubik [23, p. 634], referring to other literature, stated in summary, gaming has been used with success in training and in teaching skills; often there is no other alternative Perritt [17, p. 61], in 1971, generally concludes “Measuring the tangible Simulation Games and Experiential Exercises in Action, Volume 2, 1975 93 effects of an educational exercise is at best very difficult; nevertheless, those instructors who have used simulations are quite satisfied with the results they obtain.” Rowland and Gardner [21], in 1973, took an extremely pessimistic view of the value of gaming for education. Referring to the specifically negative comments contained in earlier papers and to their study, they concluded that favorable student reception is the only justification for computer-based business games. Decoster and Prater [7] support them, and other negative findings can be found in Schriesheim [22]. However, at the 1974 meetings of the Association for Business Simulation and Experiential Learning Byrne and Wolfe [31 reported greater learning in a simulation class. At the same meetings, Fritzsche (10] reported that a game centered course committed more information to student memory, and Goosen [11] reported his subjective feelings about increased student knowledge through gaming. The majority of the writers appear to be generally favorable towards gaming as a teaching device. Many of them also commented upon a related topic, student interest, motivation, and attitude toward gaming classes. Even in 1954, before gaming became widespread, Mood and Specht [16] commented that a game can easily be made fascinating enough to put over the dullest facts. Other representative comments are as follows [5, p. 1661: “Because of the high degree of emotional and psychological involvement of the players in such a training game they can become highly receptive to learning new ideas.”; [6, p. 4], “...and it is precisely the extremely high degree of interest and involvement on the part of the participants that makes them such a valuable training tool”; [7, p. 16], ‘Perhaps a key to why they are popular is that they generate a great deal’ of emotional involvement.. [12, p. 458], ‘. . .a game situation can be expected to be relatively successful in capturing the students’ attention”; [14, p. 59], “The computer has brought a magical means to motivate students and to help them to learn simulations; [12, p. 35], “The value of the gaming approach is that it demands active participation, kindles enthusiasm, and increases student motivation.” Raia [18] found that a gaming group had significantly higher levels of interest and motivation than a non-gaming group, and Goosen [11] reported that his game increased student interest and that participation was fun and exciting. Rowland and Gardner Dill and Decoster and Prater [7], however, found no better participant attitude in their gaming classes. Certo and Dougherty [11] generally analyze positive student comments about the value of simulation. Most of the writers mentioned above indicated their favorable but subjective beliefs concerning student opinion and motivation in gaming. All indicated that high levels of these variables contribute to the learning process. However, no study asked students to directly and comparatively comment upon their perceived learning and motivation in a gaming course versus other non-gaming courses. Simulation Games and Experiential Exercises in Action, Volume 2, 1975 94 THE STUDY This paper reports student opinion about the value of gaming and team playing relative to traditional teaching methods, the motivation and enjoyment of gaming relative to traditional teaching methods, and the amount of learning relative to other teaching methods. It is recognized, particularly in the area of learning, that students may not be completely accurate in their assessments. However, in the areas of motivation and interest, the student’s relative judgment should be an accurate source of data. The Management Game is offered each quarter at The University of West Florida as an integrative capstone course. A full discussion of the game can be found in McKenney, McFarlan, and Seller (14) and a discussion of the game operation for this study can be found in Roberts (20). Briefly, simulation is the entire activity for the course and student teams must make comprehensive and competitive marketing, production, and financial decisions for sixteen quarters of play. At the end of the quarter a questionnaire is given to each student to be completed anonymously. Students are encouraged to be open and candid in their answers. The questions of interest in this study are as follows: 1. Comment generally on the value, if any, of the course for your general educational’ growth and development. 2. Frankly state your opinions of the value of the team playing and extended simulation method of teaching this course as contrasted to traditional teaching methods. 3. Did you learn more or less in this course than others? Why? 4. Were you more motivated toward efforts in this course than in traditional lecture courses? Why? 5. Did you enjoy this course more than others? Why? In order to devise some numerical “sense” from these questions, the two instructors carefully rated the overall tone of the response to each question. The responses were rated A to F; with A strongly positive, B positive, C neutral, D negative, and F strongly negative. The instructors preferred to “grade” free responses rather than ask the students to assign a grade to assure more accuracy of grading. Table I is a summary of student responses. The matrix shows student response to be primarily in the A and B category, or highly favorable to the gaming experience. Question one did not request a relative response. Students were asked only to comment on the general value of the course. Responses were 21% strongly positive, 63% positive, and 5% negative. A large majority of students felt that they obtained considerable growth and development in the class. Responses to question two concerning the relative value of simulation and team playing were 48% strongly and 40% positive with 1% negative. This indicates that an overwhelming majority of students found the team played simulation of Simulation Games and Experiential Exercises in Action, Volume 2, 1975 95 more value than traditional teaching approaches. Question three requested a relative evaluation of the learning derived from the course. Three percent of the students responded strongly positive and 69% responded positive. This indicates that 72% of the students believed that they learned more in the gaming course than in traditional courses. Twenty-six percent of the students responded with neutral comments and only 1% made negative comment. Questions four and five asked for responses about relative motivation and enjoyment in the course. A total of 84% of the students indicated more motivation in the course with six percent indicating less motivation than in traditional courses. Also 81% of the students indicated greater enjoyment in the gaming course while six percent indicated less enjoyment. Student response proportions for each question were tested statistically to determine if the very high proportion of positive comment and very low proportion of negative comment could have occurred by chance. In each case it was determined that the proportions could not have occurred by chance, at the .01 level of significance. Thus student opinion was significantly more favorable toward the gaming class than toward traditional classes. TABLE 1 RESPONSES TO QUESTIONNAIRE Percentage of Responses Question A B C D F Total Responses 1 21% 63% 11% 4% 1% 75 2 48% 40% 10% 1% 0% 67 3 3% 69% 26% 1% 0% 69 4 15% 69% 9% 5% 1% 78 5 11% 70% 12% 5% 1% 74 Average (2-5) 9.6% 2.2% 3.6% .2% .6% In summary, an average of 19.6% of the students answered strongly positive to the four relative questions, 62.2% answered positively, 13.6% responded neutrally, 3.2% responded negatively, and .6% responded strongly negative. Thus, an average of 81.8% of the students indicated greater relative value for the gaming course than traditional courses. A 1% rounding error exists for each line. To provide a qualitative measure of the responses, some representative comments are listed below: Simulation Games and Experiential Exercises in Action, Volume 2, 1975 96 1. As much human value as pure management value; this course seemed to “get it together” for me; I mainly found out how complicated and difficult it is to be a top level manager; tremendous value; the capstone effect is terrific. 2. The most enjoyable team experience I’ve had; mistaken viewpoints become apparent much quicker; happy to be out of the traditional classroom; I feel the team method of working in this class was good, realistic although sometimes very frustrating; the game was very informative in the life like way in which it is taught; we put in many hours in making decisions, but the amount of knowledge gained from the game is terrific; we learned how to work with each other and in learning for ourselves; we learned and retained more than if we sat through a quarter of lecture; a real world situation, extremely valuable. 3. It gave me practical experience in utilizing what I have ‘earned in school; I learned a lot about how to fit many things together which I had been confronted with in previous courses; more, learned by trial and error, had immediate feedback; I ‘earned to apply what I had ‘earned. 4. Very much so. I’m not sure of the total reason. I’m a very competitive person by nature, so trying to win the game was important to me. Also, I would rather suffer a personal letdown than fee’ responsible for a team failure. This was a highly motivating factor to me; yes, because there was no choice to escape from the responsibility of my grade; the constant feedback from the computer runs was stimulating; yes, I always try to win at anything, here competition is accentuated. 5. Yes, it was different; more application than theory; no, a lot of work, no time for fun; no, because I couldn’t get along with other team members; working with a team was fun but it wasn’t without anxiety. CONCLUSION The literature contains many studies and references about the educational benefits and motivational values of business gaming. The studies do not, however, examine student opinion about business gaming. This paper examined the opinions of 84 students who had just completed a course consisting solely of an extended play of The Management Game. Student opinion was found to be overwhelmingly positive toward the simulation in relation to other, traditional methods of education. It is obvious from a students point of view that gaming provides an extremely effective educational experience. Simulation Games and Experiential Exercises in Action, Volume 2, 1975 97 REFERENCES 1. Andlinger, G. R., “Business Games - Play One,” Harvard Business Review, Vol. XXXVI, No. 2 (March-April 1958), pp. 115-125. 2. Andlinger, G. R., “Looking Around: What Can Business Games Do?” Harvard Business Review, Vol. XXXVI, No. 4 (July August 1958), pp. 147-160. 3. Byrne, Eugene T. and Douglas E. Wolfe, “The Design, Conduct and Evaluation of a Computerized Management Game as a Form of Experiential Learning,” Proceedings of the 1974 Association for Business Simulation and Experiential Learning, (Norman, Oklahoma: The Center for Economic and Management Research, The University of Oklahoma, 197k), pp. 22-30. 4. Certo, Samuel C. and Robert H. Dougherty, “The Teaching Potential of a Structured Experience,” Proceedings of the 1974 Association for Business Simulation and Experiential Learning, (Norman, Oklahoma: The Center for Economic and Management Research, The University of Oklahoma, 1974), pp. 36-40. 5. Cohen, Kalman J. and Eric Rhenman, “The Role of Management Games in Education and Research,” Management Science, Vol. VII, No. 2 (January 1961), pp. 131-161. 6. Dale, Alfred G. and C. R. Klasson, Business Gaming: A Survey of American Collegiate Schools of Business, (Austin: The Bureau of Business Research, The University of Texas, 1974) 7. Decoster, Don and George Prater, “An Experimental Study of the Use of a Business Game in Elementary Accounting,” Accounting Review, Vol. 48, No. 1 (January 1973), pp. 137- 142. 8. Dill, William R. and N. Doppelt, “The Acquisition of Experience in a Complex Management Game,” Management Science, Vol. 10, No. 1 (October 1963), pp. 30-46. 9. Dolbear, F. Trenery, Richard Attiyeh and William C. Brainard, “A Simulation Policy Game for Teaching Macroeconomics,” American Economic Review, Vol. LVIII, No. 2 (May 1968), pp. 458-468. 10. Fritzsche, David J., “The Lecture vs. the Game,” Proceedings of the 1974 Association for Business Simulation and Experiential Learning, (Norman, Oklahoma: The Center for Economic and Management Research, The University of Oklahoma, 1974), pp. 41-46. Simulation Games and Experiential Exercises in Action, Volume 2, 1975 98 11. Goosen, Kenneth R., “The Business Game: A New Approach to Managerial Accounting,” Proceedings of the 1974 Association for Business Simulation and Experiential Learning, (Norman, Oklahoma: The Center for Economic and Management Research, The University of Oklahoma, 1974), pp. 167-172. 12. Hartley, Dave and George Radley, “Games Managers Play,” Personnel Management, Vol. 6, No. 1 (January 1974), pp. 34-35. 13. Kibbee, Joel M., Clifford J. Craft and Burt Nanus, Management Games, (New York: Reinhold Publishing Corporation, 1961) 14. McFarlan, F. Warren, James L. McKenney and John A. Sieler, The Management Game: Simulated Decision Making, (New York: The Macmillan Co., 1970) 15. McKenney, James L. , “An Evaluation of a Business Game fn an MBA Curriculum,” Journal of Business, Vol. XXXV, No. 3 (July 1962), pp. 278-286. 16. Mood, A. M. and R. D. Specht, Gaming as a Technique of Analysis, (Santa Monica: The Rand Corporation, 1954) 17. Perritt, Roscoe, “Motivating Learning Through Simulation,” Data Processing Magazine, Vol. 13, No. 9 (Winter 1971), pp. 59-61. 18. Raia, Anthony P., “A Study of the Educational Value of Management Games,” Journal of Business, Vol. 39, No. 3 (July 1966), pp. 339-352. 19. Ricciardi, F. M. et al, Top Management Decision Simulation: The AMA Approach, (New York: American Management Association, 1957) 20. Roberts, Ralph M., “Experiences with the Harvard Management Game,” Proceedings of the 1974 Association for Business Simulation and Experiential Learning, (Norman, Oklahoma: The Center for Economic and Management Research, The University of Oklahoma, 1974), pp. 117-222. 21. Rowland, Kendrith M. and David M. Gardner, “The Uses of Business Gaming in Education and Laboratory Research,” Decision Sciences, Vol. 4, No. 2 (April 1973), pp. 268-278. 22. Schriesheim, Chester A., “Effectiveness of Business Games,” Reported in the Computing Newsletter, Vol. VII, No. 14 (December 1973), p. 7. Simulation Games and Experiential Exercises in Action, Volume 2, 1975 99 23. Shubik, Martin, “Gaming: Costs and Facilities,” Management Science, Vol. 14, No. 11 (July 1968), pp. 629-660. 24. Stanley, John D., “Management Games: Education or Entertainment?’ Personnel Journal, Vol. 4, No. 1 (January 1962), pp. 15-23. Table of Contents Volume 2, 1975 ABSEL Research - From Adolescence to Adulthood, Framing the Future of Business Simulation and Experiential Learning Inventory Simulation - A Time Sharing Television Output Simulation OMSIM: An Operations Management Game An Experiential-Cognitive Methodology in the First Course in Management: Some Preliminary Results Experiential Training Methodology, Traditional Training Methodology, and Perceived Opportunity to Satisfy Human Needs Operational Problems and Solutions of Business Gaming: A Primer One Experience with the V. K. Gadget Company - An Introduction to Managerial Accounting A conversational Marketing Mix Exercise The Use of Program BAYES in the Teaching of Sample Size Determination in Survey Research An Experiential Study of Performance in a Basic Management Course Using Student Opinions in Evaluation Results with a Business Game Some Impacts of Varying Amounts of Information on Frustration and Attitudes in a Finance Games Player Performance under Differing Player Configurations in the Investment Game: Some Preliminary Observations Integrating Across Functional Areas with a Computer-Assisted Case Using Computer Assisted Cases for Marketing Research Instruction ACQUIRES: ACcounting QUick Information REtrieval System A Simulation Approach to Data Processing Controls Motivating Simulation Game Performance and Satisfaction with Group Performance-Contingent Consequences The Educational Impact of Supplementary Personal Interaction in Computerized Business Games The Minnesota Manpower Management Simulation Games RAISE II, A Personal Simulation Management in a Test Tube: A Small Group Laboratory Simulation A Methodology for Measuring Decision Making in a Business Game A Case Study of a Capital Investment Simulation Combining Experiential and Clinical Methodologies in a Small Business Management Program Results of Using Gaming to Teach Ethics and Social Responsibility Business and Society: An In-Basket Simulation Crisis versus Non-Crisis Simulation Gaming Warm-up Company: A Business Simulation Business Simulations: Competition or Learning A Computerized Management Training System for Franchized Dealers Mode I Stores, Inc.: Computer Supported Cases on the Marketing Research and Problem Solving Process Experiential Learning in Statistics Through the Computer The COMPUSTAT Analysis System as an Instructional Resource Research on Experiential Learning: Enhancing the Process A Comparison of Lecture-Case Study and Lecture-Computer Simulation Teaching Methodologies in Teaching Minority Students Basic Marketing Student Evaluation of Reaction to a Marketing Simulation Game Under Varying Circumstances The Validity and Usefulness of Packaged Models in Game Play The Effectiveness of Experiential Methods in Training and Education: A Review Guidelines for the Future Development of Business Games The Future Potential of Structures Learning Exercises Turning Them on to Management by Turning Them Out to Managers with Video Tape Recorders