AN ASSESSMENT OF THE EFFECT OF EXPERIENTIAL, SIMULATION AND DISCUSSION PEDAGOLOGIES USED IN LABORATORY SECTIONS OF AN INTRODUCTORY MANAGEMENT COURSE New Horizons in Simulation Games and Experiential Learning, Volume 4, 1977 51 AN ASSESSMENT OF THE EFFECT OF EXPERIENTIAL, SIMULATION AND DISCUSSION PEDAGOLOGIES USED IN LABORATORY SECTIONS OF AN INTRODUCTORY MANAGEMENT COURSE Ralph F. Catalanello, Northern Illinois University Daniel C. Brenenstuhl, Northern Illinois University ABSTRACT This paper examines the influence of three different teaching methodologies upon students’ cognitive learning, problem- solving skill development, perceived learning and satisfaction. Approximately 500 students in an introductory management course were randomly assigned to laboratory sections in which an experiential, simulation or discussion method of instruction was utilized. One-way analysis of variance identified significant differences between the students in the six discussion, the five simulation, and the five experiential sections. Perceived learning levels and satisfaction levels were found to differ significantly between teaching methodologies, while no statistically significant differences were found between the three experimental groups on cognitive learning or problem-solving skill development. INTRODUCTION Concern for the efficiency and effectiveness of various teaching methodologies has contributed to the development of an extensive research literature which unfortunately is filled with incongruencies. Although some professors will persist in believing that one method of instruction is superior to another, it would seem that attention may be more appropriately focused at building models of the learning-teaching processes in which pedagogy is only one input into the process. Research efforts seeking to identify the “best” way to teach students a specific subject matter are not new. [7] [2] Although traditional instructional methods of lecture and discussion remain the main stay of most college instruction, the use of simulations and more recently the development of experiential approaches to teaching have attracted increasing research interest. The impact of business games on learning has been extensively reviewed by Keys. [8] While reporting finds favorable and un favorable attitudes toward simulations, Keys concluded that in the teaching of business policy concepts the use of both cases and games appeared to be superior to either approach alone. The mixture of teaching methods seems to be supportive of what Byrne and Wolfe [3] suggest as a repetitive cycle c)f learning present New Horizons in Simulation Games and Experiential Learning, Volume 4, 1977 52 in the experiential process. The learner first engages in some concrete experience which leads to reflective observation from which the learner inductively derives abstract concepts and generalizations. Once formed these conceptualizations lead deductively to new hypotheses and new responses which will be tested by new concrete experiences that will initiate the cycle again. Explanations as to why simulations and experiential techniques have become accepted pedagogical techniques focus upon the greater degree to which a student is actively involved and participating in the learning process as well as the almost immediate feedback concerning the results of their action. [6] [4] Some researchers have, however, suggested that general principles may not receive sufficient attention and that the increased involvement and time demands may result in less effort being de voted to other learning activities. [11] Cherryholmes [51 concluded after a critique of a number of studies, that while games do motivate students, there is little evidence that they teach cognitive material or problem-solving skills, or that they in duce critical thinking any more effectively than other methods of learning. The purpose of this paper is to investigate the association between instructional method and students’ cognitive learning, problem-solving skill development, perceived learning and satisfaction. The broad question being addressed in this paper is whether the students in laboratory sections using either discus sion, experiential or simulation pedagologies exhibit significantly different levels of learning or satisfaction. METHOD Subjects The sample consisted of 503 college juniors and seniors en rolled in a basic Principles of Management course at Northern Illinois University. The students attended one of two lecture sections twice a week during the Spring Semester of 1976. Each student attended one of 16 available laboratory sections once a week. The focus of the study is on the laboratory sections which consisted of six sections using a discussion group instructional method, five sections using an experiential mode of instruction and five sections where a simulation was used for in struction. The subjects were randomly placed in one of the sixteen lab oratory sections. The experimental treatment assigned to each section was determined to minimize the possible effect of perfect meeting times. A review of the demographic characteristics of the students in each treatment group indicated that the groups New Horizons in Simulation Games and Experiential Learning, Volume 4, 1977 53 differed significantly in only three areas. Students in the discussion group tend to be a little younger and the experiential group a little older. Discussion group students expected a some what lower grade in the course while those in experiential expected a higher grade. More accounting majors are in the experiential sections and fewer are in the discussion sections. No significant differences were found between the treatment groups with respect to learning style preferences or personality characteristics. Measurements During the first week of class the students completed the various assessment inventories and the pre-test examination and case analysis. Two versions of a comprehensive objective question final exam were developed as the measure of cognitive learn ing. Students attending the 9:00 a.m. lecture period answered version A as a pre-test and took version B as their post- test at the conclusion of the course while those in the second lecture period at 11:00 a.m. took version B as their pre-test and version A as their post-test. The difference between a student’s pre test and post-test scores was used as the measure of their cognitive learning. The same procedure was used with a pre and post case analysis completed in class. Differences between scores provided the measure of problem-solving skill development achieved by the student. No additional case exercises were used in any of the laboratory sections or in the lecture. All cases were graded anonymously and grade consistency was verified by regrading a randomly selected sample. While responses to examination questions and written cases provided relatively objective measures of learning, the perceptions of the student as to what was learned is also an important aspect in the total learning process. If students experiencing certain instructional methods perceive greater opportunities to satisfy human needs than students using other instructional methods, the writings of Maslow [10], Argyris [1], and McGregor [9] seem to project a resulting relatively higher motivation level with accompanying increases in related student productivity, i.e., learning. At the conclusion of the course measures of perceived learning and satisfaction were obtained from student responses to a course evaluation form using a seven point Likert scale with low numbers being associated with positive responses and high numbers, negative responses. These forms were completed anonymously except for identification of the students’ laboratory section number. The following four questions were used as measures of perceived learning: New Horizons in Simulation Games and Experiential Learning, Volume 4, 1977 54 1. Did you learn more or less in this course than other courses? 2. The laboratory section helped to develop my managerial skills? 3. The laboratory section assisted in integrating the course material? 4. Did you learn more or less from your laboratory section than from similar experiences? The six questions used to assess the students’ satisfaction with the course and their laboratory section are the following: 1. Were you motivated to work more in this course than in more traditional courses? 2. Did you enjoy this course more than other courses? 3. Overall I am highly satisfied with my experience in this course this semester. 4. There was ample opportunity to get the feeling of worthwhile accomplishment in my laboratory section. 5. Were you more motivated to work in your laboratory section than in similar experiences? 6. Did you enjoy your laboratory section more than similar experiences? An additional measure of individual satisfaction or at least participation in the course was obtained from lecture and laboratory attendance records. Students were advised that attendance records were being maintained but no points would be subtracted or added for attendance. RESULTS Cognitive Learning and Problem-Solving Skill Development To determine whether the different instructional methods influenced cognitive learning, the difference between the pre and post test results of the students were analyzed. The results of analysis of variance (ANOVA) for cognitive learning (Table 1) and for problem-solving skill development (Table 2) show no significant differences among the groups. Examination of mean data (Table 3)1 indicates that the students in discussion group laboratory sections had somewhat higher levels of cognitive learning, while students in the simulation sections had higher scores on problem-solving skill development. Students in the experiential laboratory sections showed the least change in both cognitive 1 Mean Score Tables may be obtained from the authors upon request, i.e., Tables 3, 5, 7. New Horizons in Simulation Games and Experiential Learning, Volume 4, 1977 55 learning and problem-solving skill development. In terms of knowledge and skill the results indicate that the instructional method used in the laboratory section had no significant effect on levels of learning as measured by performance on an objective question examination and an in class written case. TABLE 1 ANOVA: Cognitive learning Source DF SS MS F ratio Between Groups 2 48.00 24.00 0.54 Within Groups 470 20734.75 44.11 Total 472 20782.75 TABLE 2 ANOVA: Problem-Solving Skill Development Source DF SS MS F ratio Between Groups 2 9.57 4.79 0.69 Within Groups 384 2649.87 6.90 Total 386 2659.44 Perceived Learning Analysis of the responses to the four questions reflecting perceived learning shown in Table 4 indicates that while no significant difference in perceived learning was found concerning the amount the student felt he learned in the course, significant differences between the groups concerning their laboratory learning were found. In Table 5,2 the mean scores and standard deviations associated with the four questions for the three different types of laboratory sections are presented. Students in the simulation sections perceived that their laboratory experiences helped to develop their managerial skills more (x=3.95) than did the students in the discussion section (x=4.50). The discussion type laboratory section assisted more in the integration of the course material (x=3.55), while the simulation sections aided integration the least (x=4.85). However, when asked if they learned more or less from their laboratory section, students in the simulation sections responded more positively (x= 3.72) than the discussion section students (x=4.44). The experi- 2 Mean Score Tables may be obtained from the authors upon request, i.e., Tables 3, 5, 7. New Horizons in Simulation Games and Experiential Learning, Volume 4, 1977 56 ential section students on all three of the questions associated with perceptions concerning their laboratory section tended to have average responses somewhere between the simulation and discussion groups. TABLE 4 ANOVA: Perceived Learning Question Source DF SS MS F ratio Learn more or less Between Groups 2 5.16 2.58 1.55 In this course Within Groups 318 528.67 1.66 Total 320 533.83 Lab section devel- Between Groups 2 9.57 4.79 0.69 oped managerial Within Groups 384 2649.87 6.90 skills Total 386 2659.44 Lab section assisted Between Groups 2 9.57 4.79 0.69 in integration of Within Groups 384 2649.87 6.90 material Total 386 2659.44 Learn more or less Between Groups 2 9.57 4.79 0.69 in lab section Within Groups 384 2649.87 6.90 Total 386 2659.44 Satisfaction The ANOVA results presented in Table 6 show that the groups did not differ significantly regarding their feelings of satisfaction concerning the overall course, but again significant differences were found in satisfaction levels associated with the laboratory experience. Reviewing the mean scores found in Table 7,3 it may be seen that the simulation group tended to believe that they had more of an opportunity to get the feeling of worth while accomplishment (x=3.18) than the discussion group (x=4.60). The simulation groups were more motivated to work in their laboratory sections (x=3.64) than either the experiential groups (x=3.74) or the discussion groups (x=4.61). Both the simulation group (x=3.41) and the experiential group (x=3.60) tended to en joy their laboratory sections more than the discussion groups (x=4.76). Although the groups did not differ significantly in their absences from lecture, students in the simulation section 3 Mean Score Tables may be obtained from the authors upon request, i.e., Tables 3, 5, 7. New Horizons in Simulation Games and Experiential Learning, Volume 4, 1977 57 groups had fewer lecture absences (x=3.76) than either the discussion groups (x=4.32) or the experiential group &=4.60). Absences from the laboratory sections were significantly different with the discussion group having the highest average absence level (x=1.68) and the simulation group the lowest (x=1.02), with the experiential groups’ absences falling in between these two groups (5r=l.36). TABLE 6 ANOVA: Satisfaction Question Source DF SS MS F ratio Motivated to work Between Groups 2 0.14 0.07 0.04 in this course Within Groups 318 507.24 1.59 Total 320 507.39 Did you enjoy this Between Groups 2 3.86 1.93 0.86 course Within Groups 318 711.63 2.23 Total 320 715.50 Overall highly Between Groups 2 11.71 5.85 2.49 satisfied with Within Groups 318 747.95 2.35 course Total 320 759.66 Feeling of accom- Between Groups 2 104.58 52.29 20.24** plishment in lab Within Groups 318 821.41 2.58 section Total 320 926.00 More motivated to Between Groups 2 63.40 31.70 15.50** work in lab sec- Within Groups 318 649.98 2.04 tion Total 320 713.39 Did you enjoy the Between Groups 2 118.92 59.46 21.15** lab section Within Groups 318 894.00 2.81 Total 320 1012.92 Absence from Between Groups 2 56.72 28.36 1.56 lecture Within Groups 479 8696.89 18.15 Total 481 8753.62 Absence from Between Groups 2 34.79 17.39 6.96** lab section Within Groups 478 1193.59 2.49 Total 480 1228.38 CONCLUSIONS This paper has been an attempt to appraise the effect of three instructional methods on the performance of students in a required introductory management course. On the basis of the New Horizons in Simulation Games and Experiential Learning, Volume 4, 1977 58 findings reported here, it may be concluded that the laboratory section method of instruction had no significant effect on the students’ performance in or satisfaction with the total course. The method of laboratory instruction, however, did result in significant difference in both the perceived learning and satisfaction levels of the students in the laboratory section. Al though one might hope that the positive laboratory section results associated with the less tradition al instructional methods of experiential exercises and business gaming might be strong enough to influence the total course results, this was not the case. On the other hand, it should be noted that these innovative techniques did not negatively effect the performance of the students in this course. REFERENCES 1. Argyris, C., Integrating the Individual and the Organization, (New York: Wiley, 1964). 2. Buskirk, R. H., Simulation Games and Experiential Learning in Action, (Austin, Texas: Bureau of Business Research, The University of Texas at Austin, 1967). 3. Byrne, E. T. and D. E. Wolfe, “The Design, Conduct and Evaluation of a Computerized Management Game as a Form of Experiential Learning,” in J. Kenderdine and B. Keys, Editors; Simulation, Games and Experiential Learning Techniques: On the Road to a New Frontier, (Norman, Oklahoma: The Center for Economics and Management Research, University of Oklahoma, April, 1974). 4. Carlson, J. G. H. and M. J. Misshauk, Introduction to Gaming: Management Decision Simulations, (New York: John Wiley and Sons, 1972). 5. Cherryholmes, C., “Some Current Research on Effectiveness of Educational Simulation: Implications for Alternative Strategies,” American Behavioral Scientist, Vol. 10, 1966, pp. 4-7. 6. Goetz, B. D. and W. G. Bennis, “What We Know About Learning and Training,” Guidelines for the Aspiring Professor, Edited by Otis Lipstream and James I. Doi (Cincinnati, Ohio: South Western Publishing Company, 1963), p. 80. 7. Husband, Richard W., “A Statistical Comparison of the Efficiency of Large Lecture Versus Small Recitation Sections Upon Achievement in General Psychology,” Journal of Psychology, XXXI, April 1951, pp. 297-300. 8. Keys, Bernard, “A Review of Learning Research in Business Gaming,” (Tennessee Technological University, Working Paper, 1976). 9. McGregor, D., The Human Side of Enterprise, (New York: McGraw-Hill, 1960). 10. Maslow, A.H., Motivation and Personality, (New York: Harper and Row, 1970). 11. Shuman, J.C. and J. A. Hornaday, “Experiential Learning in an Entrepreneurship Course,” Collegiate News and Views, Vol. 29, No. 1, Fall, 1975, pp. 5-9. Table of Contents Volume 4, 1977 Double Play for Gaming Effectiveness Adaptive Rule Changes in Computer Simulation Gaming Œ A Means of Pedagogical Reactive Interchange Monte Carlo Simulation in Personnel Management Training Teaching About the Implementation of Job Redesign Using Simulation and Group Discussions An Interactive Simulation of Private Sector Collective Bargaining Leadership Evaluation and training through Behavioral Simulations: Method, Results and Future An assessment of the Effect of Experiential, Simulation and Discussion Pedagologies Used in Laboratory Sections of an Introductory Management Course An Experiential Understanding of the Trust Dimension Using Consulting Cases to teach Business Policy An Experimental Testing of Teaching Methodologies in Marketing Interpersonal Skill Development: The Experiential Training Unit (ETU) and Transfer of Training An Analysis of the Relationship between Personality characteristics and Preferred Styles of Learning Analysis of Effective Communication Skill Development in Graduate Business and Engineering Experiential Education Changing Perceptions of Learning in a Simulated Environment Student Perceptions: Simulation and the Corporate Policy Course Degree of Uniformity in Achievement Motivation Levels of Team Member: Its Effect on Team Performance in a Simulation Game Channel Conflict, Cooperation and Control: an Experiential Learning Exercise Differences in Experiential and Non-Experiential Learners' Reactions to Conflict between Individual and Organizations Behavior The Evolution and Evaluation of a Required, Senior-Level Course in Experiential Business Applications Building Management Skills through Problem Solving A Live-Case Approach to the Business and Society Course Experiential Learning: Toward the Development of a Theoretical Base and the Identification of Variables and the Hypotheses to Guide Research The Role of the Administrator in Experiential Learning and Simulations Some Thoughts on a Theory of the Use of Games and Experiential Exercises Three Applications of the Management of Learning Grid An Analysis of ABSEL: Its Past Achievements and Future Prospects New Horizons in Simulation Research Prediction of Academic Achievement in a Simulation Mode via Personality Constructs Sex Differences in a Bargaining Simulation COM-GAME: A Commodity Trading Game for Use in an Introductory Business Statistics Course A Financial Institution Management Game with Direct Participant Interactions A Non-Computerized Marketing Planning and Strategy Game Delphi in the Classroom: A Demonstration of Forecasting Economic Activity The Potential of Programmable Calculators for Processing Small Business Simulations Can a Small Predominantly Clack University Incorporate the Computer Simulation Gaming Teaching Methodology into it Curriculum Measuring the Effect of an Experiential Exercise Experiential Learning - Analysis of a Partial Success Predicting Participants' Performance and Reactions in an Experiential Learning Setting: An Empirical Investigation A Simplified, Non-Computerized Marketing Channels Game Manufacturers and Retailers: A Negotiation Game for Beginning Management Students Petroleum Management Game A Securities Dealer Simulator SIM ECO SOC with Business Curriculum Modules: A Simulation for Business Ethics and Morals The Picnic: A Perceptual Errors Exercise Salt III; an Experiential Exercise to Highlight the Interpersonal Dynamics Involved in the Negotiation Process Experiential Exercise on Values, Attitudes and Conflict Resolution in Organizational Behavior Kick'N Go: A Product Management and Social Responsibility Dilemma The Use of Self-Assessment Work-shops in a school of Business Administration The Dilemma of Self-Perception