A RELATIVE EVALUATION OF EXPERIENTIAL AND SIMULATION LEARNING IN TERMS OF PERCEPTIONS OF EFFECTED CHANGES IN STUDENTS Developments in Business Simulation & Experiential Exercises, Volume 8, 1981 145 A RELATIVE EVALUATION OF EXPERIENTIAL AND SIMULATION LEARNING IN TERMS OF PERCEPTIONS OF EFFECTED CHANGES IN STUDENTS Stephen E. Blythe, University of North Dakota Jerry J. Gosenpud, University of Wisconsin-Whitewater ABSTRACT The study’s purpose was to assess five teaching techniques including experiential learning and the simulation in terms of student perceptions of ability, attitude, knowledge and career goal attainment. The data was collected via an 18 item questionnaire administered to 628 students enrolled in 24 sections of nine management courses. The results indicate that some teaching styles facilitate some attitude ability and understanding changes while other teaching styles facilitate others. INTRODUCTION A great deal of attention has been paid, in recent times, to the relative effectiveness of various forms of teaching. Of particular interest to ABSEL, are studies assessing the relative effectiveness of experiential and simulation approaches to teaching. Such studies have been undertaken, and certain conclusions can be drawn. Regarding experiential learning, these conclusions are as follows; (1) it does as well as the lecture in helping students attain cognitive mastery (2, 7, 10, 12 and 13), (2) it is superior to the lecture method in facilitating attitude change [of educators towards occupational education (15), of college students towards themselves (3 and 8), and of graduate students in social work towards social work (9)1 and (3) it is superior to the lecture approach in facilitating satisfaction and involvement with the learning experience (i.e., 10, 12 and 13). Regarding the simulation method, there have been more studies; therefore there are more data available. Unfortunately some of that data is contradictory, partially because simulations have been attempted by educators and studied by researchers in a wide variety of academic subjects including junior high mathematics, high school social studies, college information systems and graduate business policy. Nevertheless, tentative conclusions can be drawn. First, with respect to cognitive mastery, there is evidence that the simulation does as well as conventional methods (lecture /discussion, seminar and case methods) (5 and 14 for a review), there is some evidence suggesting it does better when the subject matter is quantitative (1 and 6), but there are those who say it does gorse (11 for a review). Second, the simulation is superior to other methods for facilitating satisfaction and involvement with the learning experience (1 and 14 for reviews; and 4 and 16).. The present study is similar to those reported above in that it assesses the relative effectiveness of various types of teaching, but It assesses effectiveness in different terms than those in other studies. Whereas the above efforts assessed teaching methods in terms of their facility for helping students gain cognitive mastery and greater involvement in the course, the present study assesses in terms of specific ability, attitude, knowledge and career goal attainment. In this study, students were asked questions regarding their perceptions of the degree to which (1) certain specific skills and abilities were attained during the course, (2) changes in specified attitudes occurred as a result of exposure to course materials, (3) specified types of knowledge or understandings were gained in the course and (4) career goals were made more accomplishable by exposure to the course. The purpose of the study was to assess each of five teaching methods as to its effectiveness in facilitating changes in the above- mentioned abilities, attitudes, understandings, and career impacts as students perceive them. The five teaching methods were (1) the lecture, (2) the case method, (3) the quantitative problem-solving method, (4) the experiential method and (5) the simulation. The specific skills abilities, attitudes, and understandings and career impacts are listed in Figure 1. METHOD Questionnaires were administered to 628 students enrolled in undergraduate courses offered by the Department of Management at the University of North Dakota. The items of the questionnaire are presented in Figure 1. The questionnaire is a modification of a part of the student evaluation questionnaire developed at MIT. It allows for investigation of several areas of change possibly effected by a business course. Other than requesting demographic and course information, it contains eighteen Likert-type items. FIGURE 1 ITEMS OF QUESTIONNAIRE A. Skill, Ability, Understanding and Attitude Items 1. Ability to analyze problems 2. Ability to apply techniques 3. Ability to think creatively 4. Ability to communicate ideas 5. Ability to identify problems 6. Ability to work with people 7. Attitudes toward people 8. Ability to make decisions 9. Knowledge of management techniques 10. Willingness to take risks 11. Understanding own abilities/limitations 12. Knowledge of business principles 13. Self confidence 14. Attitudes toward business B. Career and Future Course Impact Items 1. Pursue your career goals 2. Become more valuable in the labor market 3. Become a more effective manager 4. Increase your effectiveness in other business courses Of the 628 subjects, 418 were men and 210 were women. Developments in Business Simulation & Experiential Exercises, Volume 8, 1981 146 Two hundred and thirty were management majors, 348 were majoring in other business-related fields, and 50 were majoring in non-business areas. The average age was 22 years. Since many students were enrolled in more than one management course at the time the instrument was administered, some overlapping of subjects undoubtedly occurred. Courses were classified as primarily lecture vs. supplementary lecture, case method vs. non-case method, quantitative problem solving vs. non-quantitative problem solving, experiential to a great degree vs. experiential to a slight degree vs. non-experiential, and simulation vs. non- simulation by interviews with the course instructors. Each instructor was asked the extent to which he used each of the five methods in his class. These interviews revealed that the lecture was either the primary teaching technique or it supplemented other techniques. For example, in Principles of Management it was primary in that the instructor lectured at least ninety percent of the time, but in Organizational Behavior it was not primary. Cases and experiential exercises were the primary modes, and lectures were only used to summarize conclusions from the cases and exercises. With respect to cases, quantitative problems, and simulations; it was clear from the interviews that each instructor either used the method to great extent or nearly not at all. With respect to the experiential mode, there were three courses which used the method to a substantial degree and one which used it only slightly. Therefore the three category distinction for the experiential method and the two category distinction for the other methods. The questionnaire was distributed in twenty-four sections of nine courses given by the Management Department during the last week of the term. The nine were Principles of Management, Production, Personnel, Corporation Finance, Quantitative Techniques, Organizational Behavior, Organizational Theory, Small Business and Policy. Table 1 shows how each of the courses were categorized according to the instructor interviews. RESULTS The results are summarized in Table 2. The lecture, quantitative, case and simulation results are analyzed by t- test (comparing questionnaire results from (1) courses primarily lecture with those from courses supplementary lecture, (2) courses utilizing cases with those not using them, (3) courses using quantitative problems with those not, and (14) courses using simulations with those not). The experiential results are analyzed by analysis of variance across three degrees of experiential learning method utilization: to a great degree, to a slight degree and not at all. The results in Table 2 show that some teaching methods facilitate certain kinds of attitude, understanding and ability development, while other teaching methods facilitate other kinds of attitude, understanding and ability development. The lecture method was helpful in changing students attitudes towards people, it helped students acquire the knowledge of management techniques, and it enhanced their ability to work with people. The quantitative problem- solving method helped students apply techniques, and it helped them attain knowledge of business principles. The case method facilitated the development of four abilities -- to analyze problems, think creatively, communicate ideas and work with people. It also helped students change their attitudes towards people and to understand their own abilities and limitations. Simulations were helpful with the abilities to analyze problems and think creatively. This method also helped students to take risks and understand their own abilities and limitations. The experiential mode facilitated the abilities to communicate ideas and work with people. It helped students change their attitudes toward people and helped students change their levels of self confidence. With respect to the items suggesting longer run changes, lecture courses were perceived to be helpful for students in pursuing careers, becoming more effective managers and becoming more valuable in the labor market. Courses experiential to a slight degree were helpful for pursuing careers and in becoming more valuable in the labor market. Finally quantitative courses were perceived to help students increase their effectiveness in other courses. DISCUSSION The purpose of this study was to assess the relative effectiveness of five teaching styles in facilitating changes in certain abilities, attitudes understandings and career impacts as students perceived them. The fact that some methods facilitated some changes while other methods facilitated other changes is no surprise but valuable nonetheless. It is valuable in three ways. First, it confirms common sense notions that some methods are better than others for the pursuance of certain learning objectives. Second, it Developments in Business Simulation & Experiential Exercises, Volume 8, 1981 147 legitimizes instructors changing their teaching techniques as the learning objective changes. Finally, it legitimizes future attempts to discover the effects of different teaching styles. This study’s results provide evidence for something most educators have always assumed -- namely that students learn different things in different types of classes. The fact that we now have some evidence of this suggests that future research attempts to determine specifically the effects and consequences of various particular teaching methods. may very well be successful. Not only did the general result that some methods facilitated some changes while others facilitated other changes fit common sense notions, but most of the specific results fit these notions as well. For example, one would expect that case and experiential courses would enhance abilities to communicate ideas and work with people; one would also expect that lecture courses would enhance knowledge attainment and the results of this study confirmed these expectations. On the other hand, some results were contrary to those which might be expected from certain teaching methods. For example, one would expect case courses to help students take decisions and identify problems and the fact that the results indicate otherwise should cause some concern for those of us who use the case method. One of the concerns to those of us who teach through the experiential method should be the results that the experiential method did not facilitate risk taking and the understanding of ones abilities and limitations. One of the potential purposes for teaching experientially is to place students in situations where they can confront their own behavior and become more aware of the consequences of their actions. The fact that students in this sample did not perceive themselves as taking risks or understanding their abilities and limitations as a result of experiential courses suggests the possibility that these intended purposes for the experiential teaching method are not being accomplished. The result that the simulation did not facilitate the ability to make decisions should concern those of us who teach via the simulation method. Facilitating that decisive decision making ability is one of the intended purposes of that teaching method and the results suggest the possibility that this purpose is not being accomplished. It should be noted, however, that none of the teaching methods in this study were found to help students with the ability to make decisions. Results regarding the long term career impact indices were inconsistent with those regarding the more short Developments in Business Simulation & Experiential Exercises, Volume 8, 1981 148 term ability attitude and understanding indices. Case, simulation, experiential methods were seen as more valuable in developing many different varieties of abilities and understandings than the lecture, but the lecture was perceived to be more helpful than the experiential, case and simulation methods to students in their pursuits towards careers and becoming effective managers. In other words, cases, simulations and experiential methods were seen as more helpful than the lecture for the short term, but less helpful than the lecture for the long term. This is a curious result, and there can be many different explanations. One possible explanation is that students see some good value in experiential cases and simulations for the short term, but not for the long term. This suggests that we are neglecting important responsibilities to teach students how the skills which they acquire in our experiential case and simulation classes will actually help them with their careers and as managers. Another possible explanation for the fact that lecture courses were seen as more helpful in career pursuits than cases and simulations reveals a short coming of the study. The lecture courses in this study were introductory courses, while simulation and case courses were at the senior level , and some of the results of the study are likely to be caused by as much by the level of the students (introductory vs. advanced) as by the type of teaching methodology. The result suggesting that the lecture method is more helpful in career pursuits than simulations and cases was very likely affected by the level of student in his degree program. Introductory students taking management courses for the first time via the lecture will very likely perceive their courses as beneficial to their careers simply because the course exposes them to management material for the first time. By the time students have taken their later years? case and simulation courses, management material is old hat and they no longer perceive their courses as benefiting their career goals. The study has other shortcomings. The classification of courses used here is rather artificial. All classes have some lecture, many cases and simulations involve quantitative work. Thus, classifying an approach of one type or another when reality contains much overlap may be inappropriate. The sample of classes was small, twenty four sections. The instrument was a type of course evaluation, and as with any course evaluation, the results are probably influenced by the popularity of the instructor, and the sample size is not big enough to neutralize that influence. Thus this study is far from perfect. However, it provides evidence that different specific abilities, attitudes and understandings result with different approaches to teaching. This evidence helps to lay a foundation for further research efforts with purposes to determine the specific effects of various teaching methods. Discovering the effect of various teaching approaches is important to diagnose the relative strengths and weaknesses of different teaching methods. This study has helped in this discovery, and the fact that it is methodologically imperfect should not dampen further attempts to ascertain the specific outcomes of various modes of teaching. REFERENCES [1] Alien, Layman, Gloria Jackson, Jean Ross, and Stuart White; “What Counts is How the Game is Scored”, Simulation and Games, 1978 (Dec), 9(n), 371-392. [2] Basuray, Tom and Steven A. Scherling, ‘Cognitive Complexity Development in Lecture vs. Experiential Organizational Behavior Classes”, Journal of Experiential Learning and Simulation, 1918 (Fall) 1, (1). [3] Berenson, Bernard G., Robert R. Carkhuff and Pamela Myrus, ‘The Interpersonal Functioning and Training of College Students”, Journal of Counseling Psychology, 1966 (fl), 441-446. [4] Brenenstuhl, Daniel C. and Ralph F. Catalanello, “The Impact of Three Pedagogue Techniques of Learning”, JELS, 1979 (June), 1(3), 211-226. [5} Catalanello, Ralph F. and Daniel C. Brenenstuhl, ‘An Assessment of the Effect of Experiential Simulation and Discussion Section of an Introductory Management Course, 1977 ABSEL Proceedings, Wichita, KS, Carl C. Nielson (Ed). [6] Edwards, KJ, DL DeVries, and JP Snyder, “Games and Teams: A Winning Combination”, Simulation and Games, 1972, 3(3) 247-269. [7] Gilbaugh, John W .,A Viable Alternative to the Lecture Method”, Phi Delta Kappan, 1978 (May), 59 (4), 630-632. [8] Glenn, Kenneth H. , “The Changes in Perceptions of Self from Small Group Laboratory Experiences as Measured by Semantic Differential’, Dissertation Abstracts, 1911, Vol. 32A, Part 2, p. 6759. [9] Greenwald, Shayna R. and Margaret W. Linn, Graduate Students’ Semantic Judgments of Practice Methods Before and After Their First Academic Year in Social Work’, Psychological Reports, 1913, 32, 491-6. [10] Hoover, J. Duane and Carlton J. Whitehead, ‘An Experimental Evaluation of a Cognitive -Experiential Learning Methodology in the Basic Management Course, JELS, 1979 (March), 1(2), 119-126 [11] Jackson, M.W., ‘An Antipodean Evaluation of Simulation in Teaching”, Simulation and Games, 1919 (June), 10 (2), 99-139. [12] Kelly, Lane, “An Experiential Evaluation of a Didactic - Experiential Approach for Teaching Personnel Management’ TELS, 1919 (Jan), 1 (1), 39-44. [13] Osin, L., ‘The Survival Principle and The Comparative Evaluation of Instructional Systems”, Educational Technology, 1978 (Jan), 18 (1), 19-22. [14] Pierfy, David A ., "Comparative Simulation Game Research’, Simulation and Games, 1977, (June), 8 (21), 255-268. [151 Riggs, Donald D., “A Semantic Differential Study of Attitudes Towards Occupational Education After Exposure to Exemplary Concepts”, Unpublished Doctoral Dissertation, University of Kansas, 1971. [16] Wagner, H.A., “Simulation vs. Cases vs. Text; Analysis of Student Opinion”, JELS, 1919, (March), 1 (2), 113-118. Table of Contents Volume 8, 1981 The Promotion: Human Sexuality in Organization The Simulation of Chaos Leadership Development in a Simulated Urban/Suburban (U/S) Environment Tomed: A Computer Game Emphasizing Social Responsibility/or/why the Pop-Top Can? Integrated Brain Activity and the Manager's Job: Utilizing the Troika Model What does R2 Have to do with a Product Management Course? An Analysis of the Effects of Jungian Problem-Solving Style Dimensions on Marketing Decisions Bargaining Behavior in Personal Selling and Buying Exchanges Extending the Simulation Product Life Cycle A Generalized Algorithm for Designing and Developing Business Simulations Operationalizing a Test of a Model of the Use of Simulation Games and Experiential Exercises Pygmalion and Perception: An Experiential Exercise Behavioral Consequences of Reward Regarding Employee Absenteeism in an Industrial Setting: An Operant Conditioning Approach The Simlab Program: The Use of Experimental Simulation and Process Analysis for the Development of Management and Organizations Designs for Research on Simulation-Games, Cases, and Other Experiential Exercises The Role of Students in The Case Method Weaknesses in Research Design Critical Variables in Research on the Educational Value of Management Games Research Questions for Cases Research on the Learning Effectiveness of Business Simulation Games - A Review of the State of the Science The Effects of Valuation Techniques on Holding Cost During Inflationary Periods: A Simulation Exercise The Operations Simulation - A Study in Game Development Applying Guided Design to the Production/Operations Management Course: A Progress Report and Evaluation Terminal Data Entry and Retrieval Systems Simulations and Microprocessors Microcomputers and Related Technology for Simulation Gaming Microcomputers - A New Technology for Innovations in Business Simulations Microprocessor Controlled Interactive Video Simulation Business Game Design: From Theory to Practice The Success of a Computerized Simulation in Microeconomic Pedagogy The Test Preview Game: Applying the Game Show Format Providing a Real World View of the Personal Function: A Simulation Finding an Effective Means of Teaching Managerial Behavioral Skills: Two Different Experiential Teaching Methods Compared A Management Development Program Based on the Experiential Learning Model Participant Type Differences in Response to Experiential Methods: An Informal Look Preparing Student Groups to Participate in Experiential Group Projects: An Organizational Development Approach An Instrument for the Assessment of Learning Dimensions: A Progress Report of the Learning Dimension Scale (LDS) An Empirical Analysis Relating the Learning Style Inventory to Memory and Logical Ability Teaching Styles in Simulation Experiential Learning Versus Traditional Teaching Styles Student Perceptions of Effective Teaching Behaviors Problems in Evaluation of Experiential Learning in Management Education Students' Perceptions of Learning by Simulation A Relative Evaluation of Experiential and Simulation Learning in Terms of Perceptions of Effected Changes in Students Overview of Computer Based Business Games in Business Policy Classes Behavioral Decision Theory and Business Policy Giving Accounting Students Writing Experience as Job Preparation Getting to First Base with MBO: An exercise for Writing and Evaluating Objectives Dimensions of Conflict in Experiential Learning Consumer Alienation and Perceived Relevance of the Business Simulation Using the Self-Reference Criterion to Simulate Culture in Internationalized Business Course Experiential Learning in a Cross Cultural Setting- The Practice of Simulation Approach to Business Education in Japanese Universities Student' Perceptions of the Use of a Computerized Simulation in Teaching Management Information Systems Using the Case Study Approach to Develop a Microcomputer Bases, Fully Integrated, Data Base Driven, Management Information System The Case Study as a Tool for Organizational Change: Applying the Steel Ax to the Designers of Management Information Systems Toward a Theory of Teaching Business Policy A Case Study in the Use of Experiential Learning (A Management Game Simulation) to Enhance Student Understanding of Strategy Evaluation and Policy Formulation Suggestions for Integration of the Business Administration Core Publishing Opportunities and Requirements for Business Simulation and Experiential Learning Materials How do we Apply Experiential Learning Intercollegiate Case Competitions for M.B.A. Students: Initiation and Implementation Teaching Business Policy and Strategy Using the Incident Process The Learning Co-Op Approach to the Core Policy Course Meeting the Managerial Skill Shortage - Is Academia Up to It? An Empirical Analysis of Experiential Learning Reinforcement International Experiential Learning: Experience is the Best Teacher Encouraging Student Participation During International Academic Programs European Summer Study Program: Can you, Should you, What Does it Take? Travel Seminars in Europe: How can I Direct One? International Experiential Learning: Student Evaluation ABSEL: Empirical Findings on the State of the Association Sensitivity of Performance Scores in Business Simulations Improving the Learning of a Business Simulation Game by Increasing the Process Content Student Participation in Deciding Performance Criteria for Grading in an OB Course: An Exercise and a Case Study Markup for Profit: A Simulated Self-Administered Experience in Retail Pricing The Investment Decision Game: An Experiential Learning Approach to Stock Market Decisions Through Gaming CHIPS: A Marketing Channels management Game External Validation: An Experimental Approach to Determining the Worth of Simulation Games Teaching Performance Appraisal Skills: An Experiential Approach In Support of Experiential learning: Results of a Follow-Up Survey The Introductory Management Course: Taking Theory Application One Step Further Decision Efficiency and Effectiveness in a Business Simulation The Implications of Cognitive Processing Variables and the Complex Decision Simulating the Simulation for Enhanced Player Rationality Simulation/Experiential Learning Audit