AN ANALYSIS OF THE IMPACT UPON THE EFFECTIVENESS OF TRADITIONAL INSTRUCTION, SIMULAITON GAMING AND EXPERIENTIAL LEARNING TEACHING METHODOLOGIES: AN EXPERIMENTAL DESIGN Computer Simulation and Learning Theory, Volume 3, 1976 463 An Analysis of the Impact upon the Learning Effectiveness Of Traditional Instruction, Simulation Gaming and Experiential Learning Teaching Methodologies: An Experimental Design BY: Daniel C. Brenenstuhl Ralph F. Catalanello Department of Management Northern Illinois University DeKalb, Illinois 60115 ABSTRACT The paper examines the current state of the development and operationality of innovative teaching methodologies. Support is given for utilizing an experimental design to assess the interaction of the variables upon the learning process. A proposed research design relevant to the development and integration of background, personality, and teaching methodologies is presented. Computer Simulation and Learning Theory, Volume 3, 1976 464 INTRODUCTION The quest for the “one best way” that dominated the thinking of the early founders of scientific management seems to be evident also in the thinking of many management educators. Research efforts seeking to identify the “best” way to teach students a specific subject matter are not new (Kanderdine and Keys, 1974). However, the current interest in contingency approaches, which suggest that the previously simplifying assumptions of the “one best way” may not be appropriate for explaining the processes and behavior present in modern organizations, may also provide new insight into the impact of various teaching methodologies on student learning. The predictable presence of a variety of learning styles among students suggests a need for equal variability in the learning process. This line of reasoning would suggest that given the sane relative past experiences and present conditions, different students are quite likely to react differently and even learn different things as a result of differential. levels of some moderating variable or variables. It may also be suggested, following contingency thinking, that where a particular student’s learning style is in contradiction with the instructor’s chosen learning process, education will be minimized if not rejected altogether. Bryne and Wolfe (1974) found many authors who have worked to understand some or all of the processes by which people learn cognitively. Most notable among these efforts are the works of Bruner (1966) and Piaget (1951), who have conceived of the learning process in terms of cognitive development. More recently Torbert (1973) and Kolb (1971) have developed. a sophisticated conceptualization for explaining how people learn from experience. Torbert and Kolb have woven the work of cognitive theorists and of proponents of the laboratory methods of learning into what may be called a whole-person approach to learning. At the base of the whole-person model is an underlining assumption that what a student learns is contingent upon the interaction of a number of variables. This model recognizes that there are different “best ways” of learning. Computer Simulation and Learning Theory, Volume 3, 1976 465 PURPOSE The purpose of this research project is threefold. First, the study will seek to determine if the traditional lecture method when combined with either a discussion, simulation, or experiential approach can produce significant difference in the students (a) cognitive learning, (b) problem-solving skills, (c) motivation, and (d) satisfaction. Second, it will attempt to identify whether students with certain background and personality characteristics could benefit more from one teaching methodology than another. Finally, the third purpose of the study is to provide base- line data for a longitudinal study that will seek to determine the possible influence that these approaches might have on the student’s performance in a senior level integrative capstone course that utilizes both cases and a simulation. The present study, while recognizing that the learning process may be influenced by a multitude of variables, represents a systematic, if not totally inclusive, attempt to provide knowledge concerning how sane of the variables influence and are influenced by different teaching methodologies. LITERATURE REVIEW The majority of educators continue to focus their attention on the cognitive aspects of learning, while utilizing a single traditional teaching methodology, such as the lecture format. A few academies have taken computer simulations and experiential approaches to learning as grand exercises in technology with little substantive research evidence regarding the educational worth of the techniques. The increased application of such innovative teaching techniques as the computer simulation has been well documented by survey (Graham and Gray, 1969, Gerstenfeld and Maynard, 1971). The comparative contributions to learning derived from various teaching methodologies, however, is not all document. Why the sudden upsurge in interest in computer simulations and experiential learning? What advantages do they afford the instructor and what benefits can the student derive from using such techniques? Questions like these must be answered in order to assure that our increasingly limited educational resources are experienced on teaching methodologies which will help our students as whole-persons to function in an intellectual, emotional and behavioral capacity in our increasingly complex business organizations. Computer Simulation and Learning Theory, Volume 3, 1976 466 Experiential learning has been defined by Hoover (1974) as an on going process. More specifically, experiential learning exists when a personally responsible participant (s) cognitively, affectively, and behaviorally processes knowledge, skills, and./or attitudes in a learning situation characterized by a high level of active involvement. Experiential learning may be viewed as a methodology of education whereby structure and individual or group experiences are contrived to develop learning and perceptual capacities, to develop and reinforce cognitions, to impact on emotions and attitudes, and importantly to function in developing capacities to behave consistently with the insights obtained from these processes and experiences. Experiential learning methods essentially attempt to combine the processes of learning with the content of learning. Byrne and Wolfe (1974) see the experiential process of learning as a repetitive cycle in which the learner first engages in some concrete experience. This experience leads to reflective observations on that experience from which the learner inductively derives abstract concepts and generalizations. Once formed, these conceptualizations lead deductively to new hypotheses and actions which will test their implications, and these new actions lead to new concrete experiences which initiate the cycle again. Several possible weaknesses to the experiential approach to teaching have been identified by Shuman and Hornaday (1975). First, the course might be too much of a “cookbook” or “How to” exercise with insufficient attention to general principles. Second, the students’ energy might be shifted too heavily to projects with a loss of effort elsewhere. Third, there may be difficulty in evaluating individual student effort. A fourth weakness is the potentially excessive tine requirements placed on the instructor. Even with the recognized weaknesses of experiential methods they have become accept~ pedagogical techniques in our universities and businesses. Barkin (1971) proposed as one reason for their adoption that they allow behavioral aspects to be studied as they occur in a dynamic system. Certainly the area of management must qualify as a dynamic system that needs more than a simple static pedagologic vehicle. This desire for a dynamic pedagologic methodology is clearly one reason for the increase in both simulation and experiential usage. Computer Simulation and Learning Theory, Volume 3, 1976 467 Goetz and Bennis (1963) have suggested another reason for the increasing usage of experiential techniques. They theorize that the greater the degree to which a student is actively involved and participating in the learning process, the more effective the learning. Carlson and Misshauk (1972) Suggest that the dynamic qualities of simulation lead to increased learning for the participant. They believe that the subjects in a simulation experience sharpen their decision making ability and their skills of analysis. Participants in simulations and experiential exercises also benefit from experiencing the immediate feedback from their decisions. This view is supported by the Skinnerism schools research on schedules of reinforcement. The sooner reinforcement is administered the more effective it is in shaping the behavior of t~ subject. Although, most computer simulations do not provide immediate feedback, they, like other experiential approaches when used as a long term teaching device, do provide one or two day turnaround. This is much more efficient from a reinforcement point of view than your typical midterm examination or term grade. Based on the above sample of the literature, it should appear that computer simulations arid experiential methods offer almost the ideal pedagogical tool. Cherryholmes (1966) after a thorough critique of a number of experiments involving simulation gaming concluded that while games do motivate, there is little substantive evidence that they teach cognitive material or problem solving skills, or that they induce critical thinking any more effectively than other methods of learning. Davis (1970) while citing the development of many technological advances in experiential approaches, laments that for all the innovations and gimmicks that have appeared on the scene, little solid research can be found to validate the claims made for computer simulations or experiential learning. SAMPLE AND SETTING The sample will consist of approximately 500 college juniors and seniors enrolled in a basic principles of management course at Northern Illinois University. The students will select one of the sections of Management 333 based upon availability of and self-preference for the 9:00 a .m. or the 11:00 a.m. lectures on Monday and Wednesday during the spring Semester of 1976. Each student is required to select from 16 available times on Thursday Computer Simulation and Learning Theory, Volume 3, 1976 468 or Friday a discussion section. This discussion section will be the central focus of the intended study. Each of four teaching assistants will be responsible for a total of four discussion sections. DESIGN (I) The experiment is conceptualized as a longitudinal study covering at least the semesters. The study will be broken into three phases-pre-experimental, experimental and post-experimental. The pre-experimental phase will deal primarily with coordinating the various details arid training the Teaching Assistants who will be responsible for the discussion sections. Also included in this pre-experimental phase will be a collection of demographic data and several pretests. The pretests will have t~ basic aims 1) to establish the level of cognitive content and the skill level of the students for a comparison of overall learning during the experimental phase; and 2) to provide checks on the random nature of t~ students distribution in the discussion sections. The demographic data will be set up for mark sensing and will be transferred to punched cards for analysis. Any discrepancies in the random assignment to discussion sections will be statistically removed by covariance analysis. Computer Simulation and Learning Theory, Volume 3, 1976 469 The pretests will consist of: 1) Two versions of the final exam will be given on the second day of class. Let us call these Version A and Version 13. Version A will be given to the 9:00 a.m. class and Version B will be given to the 11:00 am. class as pre-tests of cognitive content. These tests will be given at the end of the semester, however, at this time Version B will be given to the 9:00 am class and Version A will be given to the 11:00 a.m. class. The change in test score over the semester will be used as a direct measure of cognitive learning in the course. 2) Two cases will also be given early in the semester. Let us again can these cases Version A and Version B. Students in the 9:00 am. class will be assigned Version A to write out in class and the 11:00 a.m. class will be given Version B to write out in class. These sane cases will also be assigned late in the semester on a counter-balanced basis arid will be used as indicators of skill in problem solving acquired during the course. 3) Several personality tests will be given during the first discussion meeting as well as a learning profile. If necessary time will be made available in the second discussion meeting to finish the battery of instruments. The following instruments will be administered to the students before any experimental manipulation takes place. (1) Rotter’s Interpersonal Trust Scale (2) Rotter’s Internal-External Locus of Control (3) A social value questionnaire (4) A need achievement instrument (5) Machavellianism instrument (6) Social responsibility scale (7) Kolb, Rubin and McIntyre Learning Profile These personality scores will be analyzed for differential changes when the students enroll in the capstone courses in their senior year. They will also be examined as potential moderators of the effectiveness of any particular learning style and sate will serve as independent variables for use in the experimental hypotheses. The experimental phase of the study will begin during the second Week of the semester after the usual adjustments in a drop-add period have provided final class rosters for each of the discussion sections. The students Computer Simulation and Learning Theory, Volume 3, 1976 470 will be informed that the nature of this particular discussion section will be determined from the results of their learning profiles. In actuality, five sections will be assigned randomly to the experiential mode, five sections will be assigned randomly to the simulation game experience and six sections will be assigned randomly to the traditional lecture-discussion approach. Each teaching assistant will be assigned to one section in each of the three types of classes and will be assigned a fourth section from one of the three approaches. The assignments will be randomly made as far as possible-given typical academic constraints for each of the teaching assistants. The students will all hear the sane lectures during the two regular mass lectures each week and will all take the same examinations during the semester. However, each student will also attend one discussion section per week and will thus participate in only one of the three different approaches for the entire semester. The major independent variable will be the type of discussion section that the students are assigned. Secondary independent variables will be individual personality measures and learning profiles as well as data taken from the demographic material. The dependent variables will be: 1. Charge in cognitive content-pre-post test 2. Change in skill level-pre--post test 3 Mean differences over the regular test instruments 4. Absences in the mass lecture 5. Absences in the individual discussion sections 6. Individual satisfaction measures for: a) The course in general taken from student evaluations b) Their discussion section taken from student evaluations c) The professor presenting the mass lecture taken from student evaluations d) Their individual discussion section Teaching Assistant (taken from student evaluations) 7. Comments written on the student evaluations will be analyzed for content as a measure of satisfaction. Computer Simulation and Learning Theory, Volume 3, 1976 471 8. Effort will be measured from student statements as to the amount of tine spent in preparation for lectures and amount of time spent in preparation for discussion sections. Average amount of time spent in other three-major courses will also be obtained. Since the study involves a number of dimensions, specific hypotheses are not within the scope of this paper. However, several hypotheses will be presented to demonstrate the types of questions that the research will be seeking to answer. Hypothesis 1 - There is a significant difference between the cognitive learning achieved by students in discussion, simulation, or experiential groups. Hypothesis 2 - There is a significant difference between the problem solving skill achieved by students in discussion, simulation, or experiential groups. Hypothesis 3 - There is a significant difference between the satisfaction achieved by students in discussion, simulation or experiential groups. Hypothesis 4 - There is a significant difference between the motivation achieved by students in discussion, simulation, or experiential groups. Hypothesis 5 - There is a significant difference in the cognitive learning of students in discussion groups will have different majors. Hypothesis 6 - There is a significant difference in the problemsolving skill of students in simulation groups who have different learning profiles. Hypothesis 7 - There is a significant difference in the satisfaction of students in experiential groups who have different levels of N-Achievement. The above hypotheses are only suggestive of the range of questions that may be asked concerning the interaction of a range of relevant variables with the efficiency and effectiveness of several teaching methodologies. While study represents a systematic attempt to consider a number of variables, it is recognized that other potentially relevant variables must be excluded. The use of an experimental design in this research study provides a unique opportunity to secure information on relationships and particularly on the form of the relationships. The statistical analysis will be performed using Multiple Analysis of Variance (MONOVA) procedures as a primary tool for testing ideas and securing data upon which to base conclusions. Computer Simulation and Learning Theory, Volume 3, 1976 472 REFERENCES Barkin, S.R. “The Use of Management Games in Organizational and Behavioral Research,” Management Information Systems, Working Paper Series, MISRC-WP-71-03, (July, 1971), p. 2. Bruner, J. S. Toward a Theory of Instruction, (N~ York: W. W. Norton, 1966). Byrne, ET. and D.E. Wolfe. “The Design, Conduct and Evaluation of a Computerized Management Game as a Form of Experiential Learning,” The Proceedings of the 1st Annual National Conference on Business Gaining and Experiential Learning. Carlson, J.G.H. and M.J. Misshauk. Introduction to Gaming: Management Decision Simulations, (New York: John Wiley & Sons, 1972). Cherryholmes, C. “Some Current Research on Effectiveness of Educational Simulations: Implications for Alternative Strategies,” American Behavioral Scientist, Vol. 10, (1966) pp. 4-7. Davis, R.H. Review of “Simulation and Gaming in Education,” Simulation and Games, (December, 1970), pp. 437-440. Gerstenfeld, A. and P. Maynard. “A Study of Management Games as a Training Method,” Training and Development Journal, Vol. 25, (1971), pp. 12-15. Goetz, B.D. and W.G. Bennis. “What We Know About Learning and Training,” Guidelines for the Aspiring Professor, Edited by Otis Lipstream and James I. Doi (Cincinnati, Ohio: South-Western Publishing Company, 1963), p. 80. Graham, Robert G. and C.F. Gray. Business Games Handbook, American Management Association (1969). Hoover, J.D. “Experiential Learning in Conceptualization and Definition,” The Proceedings of the First Annual National Conference on Business Gaining and Experiential Learning. Kenderdine, J. and B. Keys, Editors. “Simulation, Games and Experiential Learning Techniques: On the Road to a New Frontier,” The Proceedings of a National Conference on Business Gaming and Experiential Learning, (April, 1974). Kolb, D.A. “Individual Learning Styles and the Learning Process,” Massachusetts Institute of Technology Sloan School Working Paper No. 535-71, (1971). Piaget, J. Play, Dreams and Limitations in Childhood, (New York: W.W. Norton, 1951). Computer Simulation and Learning Theory, Volume 3, 1976 473 Shuman, J.C. and J.A. Hornaday. “Experiential Learning in an Entrepreneurship Course,” Collegiate News and Views, Vol. 29, No. 1, (Fall, 1975) pp. 5-9. Torbert, W.R. Learning from Experience: Towards Consciousness, (New York: Columbia University Press, 1913).