EXPERIENTIAL STUDY OF PERFORMANCE IN A BASIC MANAGEMENT COURSE Simulation Games and Experiential Exercises in Action, Volume 2, 1975 83 EXPERIENTIAL STUDY OF PERFORMANCE IN A BASIC MANAGEMENT COURSE Daniel C. Brenenstuhl Indiana University The experiential mode of instruction is a relatively recent development in the academic community. On-the-job training has been used extensively in an industrial sector but for many reasons our academic sector resisted this action-oriented instruction. Many academics have taken the experiential mode of learning as an exercise in innovation with little substantive evidence regarding the educational worth of the various techniques. Some research has found positive evidence for one type of experiential learning, computer simulation. Boocock’s [2] studies, used computer simulation as a substitute for a more conventional classroom approach which typically involved reading material, listening to a lecture on the material and then discussing the material. It appears, as a result of the present study, that benefits may also be derived when computer simulations are used in conjunction with the conventional lecture-discussion method. Usage of experiential techniques has increased significantly among American schools of business in the past eight years. The results of a survey [7] conducted in 1966 showed a few computer simulations existed at that time. However, an updated survey [8] conducted in 1972 by the same researchers revealed widespread usage of computer simulations, including a number of specialized functional simulations. Graham and Gray 112) found that over 90% of the 90 business schools that responded to a survey they conducted reported usage of some type of computer simulation. Gerstenfeld and Maynard 110] found in a study of 20 universities that 75% indicated that they presently used management simulations. Perhaps more important to the practitioner is the fact that out of 169 industries sampled, 21% indicated that they used management simulations as training devices. It appears that the experiential mode is fast becoming an accepted pedagogical technique in our universities and industries. This paper primarily focuses upon computer simulations. However, computer simulation is but one of many different techniques that are commonly thought to make up the experiential mode. The findings of this paper may be loosely generalized to many of the other experiential techniques. Why the sudden upsurge in interest in the experiential mode? What benefits can the students derive from using such techniques? Barkin [1] proposed that to study behavioral aspects as they occur in a dynamic system, a dynamic system must be the research vehicle. Certainly the area of management must qualify as a dynamic system that needs more than a simple static research vehicle. This desire for a dynamic research methodology is one reason for Simulation Games and Experiential Exercises in Action, Volume 2, 1975 84 the increase in the usage of experiential techniques. Two researchers [11, 6) have suggested another reason for increasing usage of experiential methods. They theorize that the greater the degree to which a student is actively involved and participating in the learning process, the more effective the learning. Subjects are actively involved in making decisions which have real consequences in their immediate environment. Interest and participation can certainly be claimed by the supporters of computer simulation as attributes of that technique. Carison and Nisshauk [411 agreed that the dynamic qualities of computer simulation lead to increased learning for the participants. They feel that the subjects in a computer simulation experience sharpen their decision making ability and their skills of analysis. Participants also benefit from experiencing the immediate feedback from their decisions. This view is supported by the Skinnerian schoolsT research on schedules of reinforcement. The sooner reinforcement is administered, the more effective it is in shaping the behavior of the subject. Although most man- computer simulations do not provide immediate feedback, when used as a long term teaching device, they do usually provide one or two day turnaround. This is much more efficient from a reinforcement view than your typical midterm examination or yearly bonus. Based on the above sample of the literature, it would appear that computer simulation is almost the ideal pedagogical tool. Cherryholmes 15] was perhaps the first thorough critic to synthesize a number of experiments involving computer simulation usage. Cherryholmes concluded that simulations do motivate, but that there is not any substantive evidence that computer simulation teaches cognitive material or problem solving skills, or that they induce critical thinking any more effectively than other methods of learning. Davis [9] has cited the development of many technological advances in computer simulations. However, he laments, for all the innovations and gimmicks that have appeared on the scene, little solid research can be found to validate simulation usage. The equivocal nature of the literature prompted the author to conduct his own investigation into the relative benefits of computer simulations. The research design for the present study was in part suggested by Boocock and Schild 13, p. 20). After discussing the difficulties of designing research in the area of computer simulation, they go on to say: “Assignment by classes is usually the best the researcher can do, and he is lucky if he can match classes on some group measures (e.g., mean IQ or achievement scores), can have the matched classes taught by the same teacher, and can designate the classes as experimental or control by some random procedure.” In the current research, these conditions outlined by Boocock and Schilds were met. Two sections of a one semester management Simulation Games and Experiential Exercises in Action, Volume 2, 1975 85 course were taught by the author using a computer simulation as an adjunct aid in one section and not in the other. Students were not aware, in advance of section selection, of this difference in the two sections. Hypotheses The author developed several hypotheses based on the relevant literature and on intuitions from previous usage of computer simulations. Hypothesis 1. There will be a significant difference between the performance of the control group and the experimental group on the first examination. The control group is hypothesized to perform at a higher level. This hypothesis was developed as a result of expected cognitive overloading of the experimental group during the first few weeks of the course. Hypothesis 1 was developed from the author’s personal experiences with computer simulations. Hypothesis 2. There will be a significant difference between the performance of the control group and the experimental group on the second examination. The experimental group is hypothesized to perform at a higher level. The superiority of the experimental section will be a result of usage of the computer simulation technique. Hypothesis 2 was suggested by the research conducted to this point and from the author’s personal experience. Hypothesis 3. There will be a significant difference between the dissatisfaction displayed by the control group and the experimental group, as measured by the modified Needs- Satisfaction Questionnaire. The experimental group will display less dissatisfaction than the control group. Hypothesis 3 was suggested by current literature on participation and satisfaction [14, 15). Hypothesis 4. There will be a significant difference between the performance of the control group and the experimental group on the post-test. The experimental group will do significantly better on the post-test. Hypothesis 4 is related to Hypothesis 3 and based on the same arguments. Methodology Two sections of a sophomore level management course (Management 211) were used to carry out the experiment. The classes were scheduled at 9:00 A.M. and 10:00 A.M. in the same room in the same building on the campus of a small Eastern university. By using a random selection technique the 9:00 A.M. class was chosen as the control group and the 10:00 A.M. class was the experimental group who used the computer simulation. The experimental group was responsible for the computer simulation experience in addition to the material for which the control group was responsible. The experimental group may have been operating near the level of cognitive overload for the first few weeks as the students spent con- Simulation Games and Experiential Exercises in Action, Volume 2, 1975 86 siderable effort learning the computer simulation procedures and environment. This condition should have dissipated by the time of the first examination, roughly 5 weeks into the semester, as the computer simulation was a front-loaded simulation requiring larger amounts of time to initially learn the rules and significantly less time thereafter. This projected cognitive overload was fundamental in development of Hypothesis 1 (significantly different performance between sections on the first examination). Two pre-tests, two evaluative instruments, a post-test and a measure of dissatisfaction were administered during the experiment. Written student evaluations of the course were also compared and contrasted for each section. The second pre-test mea- sure was used again as the post-test instrument 2 weeks after the experimental treatment was terminated. Both groups were evaluated using the same instruments. The results were analyzed by conducting tests of difference between arithmetic means. The computer simulation covered the first 12 weeks of the 16 week semester. The simulation used was The Executive Game [13]. It is a computer simulation experience in a managerial context. The simulation was chosen as the vehicle of research primarily because of the excellent Interface with the university’s central computer. It is possible that with another simulation, the results of this experiment might have been different. The simulation was used as an adjunct aid in the experimental section. No attempt was made to teach the equivalent computer simulation material to the control group. A small amount of class time was devoted in the experimental section to cover simulation-related administration details. During the equivalent time period each week, the control section met as usual. An attempt was made to be sure that both sections received equivalent coverage of the traditional subject matter as taught in previous sections of the same course by the author. Examinations were exclusively over the traditional material presented in both sections. It is possible that the usage of the computer simulation contributed to substantial improvement in the skill areas of the participants in the course due to an increase in the scope of coverage. No attempt was made to measure these possible increases in the skill areas. The Sample Section time (9:00 or 10:00) was chosen by the students according to which section time best fit the individual student’s schedule and/or preferences. No attempt was made to evaluate the difference in performance between business majors and nonbusiness majors because of the relatively small samples involved in the experiment. The control section had 27 business majors and 6 non- majors. The experimental section had 26 business majors and 7 nonmajors. Simulation Games and Experiential Exercises in Action, Volume 2, 1975 87 To be sure of matched sections the groups were evaluated by two separate pre-testing devices. The first pre-test was a comparison of the students’ overall grade point average for both groups (see Table 1). The grades were based on a 4.00 system with A’s weighted at 4.00 and F’s weighted at 0.00. There was no significant difference in the overall grade point averages for both sections. TABLE 1 Comparisons of Means and Standard Deviations for the Experimental and Control Groups Over the Evaluation Instruments Pre-Test 1 Pre-Test 2 Post-Test Grade Standardized Standardized Point Final Exam Final Exam Averages Exam I Exam II Control µ = 2.62 µ = 48.47 µ = 74.94 µ = 77.69 µ = 6487 Group σ = 0.52 σ = 9.18 σ = 9.95 σ = 8.79 σ = 8.12 Experi- µ = 2.78 µ = 49.68 µ = 74.45 µ = 80.38 µ = 66.46 mental σ =0.58 σ = 7.52 σ = 9.35 σ = 7.31 σ = 8.43 Group P < .25 p < .56 p < .83 p < .09 p < .22 Two Tailed Two Tailed One Tailed One Tailed One Tailed The second pre-test was a standardized final examination in managerial concepts [18]. This pre-test was administered unannounced on the first day of class. In an attempt to eliminate potential experimenter bias, the grading and score tabulation was performed by a research assistant. The instrument was selected as a general test of the field of management and was not meant to cover only the material to be presented during the course under examination. Again, the test for a difference between the two means yielded no significant difference between the two groups (see Table 1). The Cognitive Tests and Results The examination instruments were prepared by the principle investigator relying heavily on material covered on past examinations to minimize experimenter bias. The examinations were a mixture of objective questions and essays. They were scheduled during the evening so that both sections could take the examination together. A large room was employed and the subjects were widely separated to avoid distractions. The first examination was given 5 weeks into the semester and showed no significant difference in performance between the control section and the Simulation Games and Experiential Exercises in Action, Volume 2, 1975 88 experimental section (see Table 1). This disconfirms the author’s first hypothesis. However, the result favors the experiential technique. The second examination was given 11 weeks into the semester and had the predicted significant difference in performance in favor of the experimental group (see Table 1). Thus, the second hypothesis received marginal support (p < .09). Another examination was given roughly 4 weeks after the experiment terminated. Since this examination covered additional material presented after the simulation ended, it was not included in the present analysis. Both groups were roughly equivalent on this extra examination. The comprehensive examination pre-test given on the first day of class was administered again 2 weeks after the computer simulation ended as an experimental post-treatment measure. Using the test for differences between two arithmetic means, the two sections showed no significant differences. Thus, Hypothesis 4 was not supported. This result is discussed under the summary and conclusion. The Individual Satisfaction Instrument and Results Individual satisfaction was measured by an adapted form of the Needs-Satisfaction Questionnaire [16]. In this study the phrase ‘~. . in Management 211” was substituted for the original Porter phrase “. . in my management position,” where appropriate. Each item had three sections to the answer. For Example: 1. The feeling of a job well done in Management 211: a. How much is there now? (mm.) 1 2 3 4 5 6 7 (max.) b. How much should there be? (mm.) 1 2 3 4 5 6 7 (max.) c. How important is this to me? (mm.) 1 2 3 4 5 6 7 (max.) The instrument was made up of 20 questions. The score for each question was calculated by taking the absolute value of the difference between the value for answer (a) and the value for answer (b). Answer (c), a measure of need importance, was included for possible future research and was not used in this study. The value (b-a) is a measure of need deficiency or dissatisfaction. Under this system, the higher the test score, the greater the level of dissatisfaction. The scores represent simulation over the 20 questions. The control group had a mean value of 28.79 with a range from 7 to 57. The experimental group has an average value of 24.48 with a range from 4 to 48. Because the scale was felt to be ordinal in nature, a two-tailed Mann Whitney “U” test [17] was utilized Simulation Games and Experiential Exercises in Action, Volume 2, 1975 89 to test for difference between the control group and the experimental group on the dissatisfaction scale. This difference is significant at the .01 level. This finding supported Hypothesis 3. It can also be concluded in three separate ways that the experiential technique did contribute a valid affective educational benefit to its participants. The first element of evidence was the decrease in the dissatisfaction of the experimental group as shown by the results of the Needs-Satisfaction Questionnaire. The second element of evidence supporting affective benefits derived from the experiential technique was obtained in an evaluation questionnaire completed by the students in both sections. The results of the evaluation indicated an increase in satisfaction in the section using the experiential technique. Many students in the experiential section mentioned the fact that they felt the class to be “less boring” compared to their other classes and “really interesting.” The third piece of evidence relating to the experiential section’s affective state was in a general attitude domain. The classes appeared to progress more smoothly and attendance was significantly better for the experiential section on days that decisions were to be rendered (p < .05). Several groups volunteered to pick up the computer printouts as early as possible so that they could see their results. The experimenter noticed that on days when the results were due to arrive, the students in the experiential section were constantly dropping over to the experimenter’s office to see if the results were ready. The experimenter made an effort to avoid differential attention to students in the simulation section when they visited his office. Students in the control section were encouraged to drop by the experimenter’s office to discuss the course at their convenience. The control section displayed no such increase in general affective state and, if anything, they became less energetic as the weather turned sunny late in the semester. Summary and Conclusion From the results of this experiment it can be concluded that the experiential technique displayed some cognitive benefit to pupils when used as an adjunct to a basic management course. Although the significance of the difference on the second examination was not quite up to standard levels, it was interesting in the context of this experiment. The diversity of background and academic majors of the participants appeared to dampen the results. A replication of the study using more advanced students or full-time managers with a specialized computer simulation is suggested. Based on the strength of this experiment, it appears worthwhile to increase the size of a replication sample. By using more advanced students or managers one would eliminate the relatively inefficient nonbusiness majors and would thus create a more uniformly skilled group of subjects. Simulation Games and Experiential Exercises in Action, Volume 2, 1975 90 The fact that the post-test did not reveal any significant differences between the sections created some dissonance in the author until he reviewed the content of the pre/post-test instrument. In an attempt to remain unbiased, the author had not reviewed the pre/post-test instrument but had used it on the basis of an esteemed colleague’s recommendation. Much of the material in the pre/post-test instrument was not covered by the course under investigation. The pre/post-test was heavily biased toward the classical approach to management taken by its designer, George Terry. The author taught a course with a stronger emphasis on organizational behavior. It was concluded in three separate ways, from the results of the test instruments, that the experiential experience did contribute a valid affective educational benefit to its participants. This positive feedback displayed on the test instrument and in the comments from the simulation participants was convincing to the author and appeared significant in terms of the educational benefits that this group of students derived from the experiential experience. An atmosphere of excitement prevailed on days when the results were returned to the experimental section. The students seemed vitally interested in their performance with regard to the rest of the teams in their industry. The enthusiastic attitude and increased interest were justification enough for the extra effort required of both the experimental subjects and the author as a result of utilizing the experiential format. The experiential experience in this study did not conclusively prove that it was cognitively superior to the traditional lecture technique. However, considering the increased work load that the simulation section carried, it was important to see that their average score was significantly better than the control group on the second examination (as suggested by Hypothesis 2) and was equivalent on the other instruments. The true value of simulation may well lie in the affective domain. Keeping cognitive performances up and increasing the level of satisfaction seems to be worth the effort. The interest that the students in the experiential section displayed was most gratifying. Many seemed to realize for the first time in their educational careers that their books and their teachers actually had some relevance to the world outside the university. This may be the most significant benefit of experiential learning when compared to the traditional lecture format. REFERENCES 1. Barkin, Stephen R., “The Use of Management Games in Organizational and Behavioral Research,” Management Information Systems, Working Paper Series, MISRC-WP-71-03, (July 1971), p.2. 2. Boocock, S. S., “The Life Career Game,” Personnel and Guidance Journal, (December 1967), pp. 328-334. Simulation Games and Experiential Exercises in Action, Volume 2, 1975 91 3. Boocock, S. S., and E. O. Schild, Simulation Games in Learning, (Beverly Hills, Ca.: Sage Publications, Inc., 1968). 4. Carison, J. G., and M. J. Misshauk, Introduction to Gaming: Management Decision Simulations, (New York: Wiley, 1972). 5. Cherryholmes, Cleo, “Some Current Research On Effectiveness of Educational Simulations: Implications for Alternative Strategies,” American Behavioral Scientist, Vol. 10, (1966), pp. 4-7. 6. Coghill, Mary Ann, “Games and Simulations in Industrial and Labor Relations Training,” Key Issues Series No. 7. (Ithaca, N.Y.: NY. State School of Industrial & Labor Relations, 1971). 7. Computing Newsletter for Schools of Business, (Colorado Springs: School of Bus. Administration, University of Colorado, 1966). 8. Computing Newsletter for Schools of Business, (Colorado Springs: School of Bus. Administration, University of Colorado, 1972). 9. Davis, Robert H., “Review of Simulation and Gaming in Education,” Simulation and Games, (December 1970), pp. 437-440. 10. Gerstenfeld, A., and P. Maynard., “A Study of Management Games as a Training Method,” Training & Development Journal, Vol. 25, (1971), pp. 12-15. 11. Goetz, B. E., and Warren G. Bennis, “What We Know About Learning and Training,” in O. Lipstream and J. Doi (Eds.), Guidelines for the Aspiring Professor, (Cincinnati: South- Western Publishing Co., 1963), pp. 80. 12. Graham, Robert G., and C. F. Gray, Business Games Handbook, (American Management Association, 1969). 13. Henshaw, Richard, and James Jackson, The Executive Game, (Homewood, I.: R. Irwin, 1972). 14. Herzberg, F., Work and the Nature of Man, (N.Y.: World Publishing, 1966). 15. Herzberg, F., B. Mausner, and B. Snyderman., The Motivation to Work, (New York: Wiley & Sons, 1959). 16. Porter, Lyman W., and Edward E. Lawler, Managerial Attitudes and Performance, (Homewood, Ii.: R. Irwin, 1968). 17. Siegel, Sydney, Nonparametric Statistics for the Behavioral Sciences, (New York: McGraw-Hill Book Co., 1956). 18. Terry, George, Self-Review in the Principles of Management, (Homewood, Ii.: R. Irwin, 1972). Table of Contents Volume 2, 1975 ABSEL Research - From Adolescence to Adulthood, Framing the Future of Business Simulation and Experiential Learning Inventory Simulation - A Time Sharing Television Output Simulation OMSIM: An Operations Management Game An Experiential-Cognitive Methodology in the First Course in Management: Some Preliminary Results Experiential Training Methodology, Traditional Training Methodology, and Perceived Opportunity to Satisfy Human Needs Operational Problems and Solutions of Business Gaming: A Primer One Experience with the V. K. Gadget Company - An Introduction to Managerial Accounting A conversational Marketing Mix Exercise The Use of Program BAYES in the Teaching of Sample Size Determination in Survey Research An Experiential Study of Performance in a Basic Management Course Using Student Opinions in Evaluation Results with a Business Game Some Impacts of Varying Amounts of Information on Frustration and Attitudes in a Finance Games Player Performance under Differing Player Configurations in the Investment Game: Some Preliminary Observations Integrating Across Functional Areas with a Computer-Assisted Case Using Computer Assisted Cases for Marketing Research Instruction ACQUIRES: ACcounting QUick Information REtrieval System A Simulation Approach to Data Processing Controls Motivating Simulation Game Performance and Satisfaction with Group Performance-Contingent Consequences The Educational Impact of Supplementary Personal Interaction in Computerized Business Games The Minnesota Manpower Management Simulation Games RAISE II, A Personal Simulation Management in a Test Tube: A Small Group Laboratory Simulation A Methodology for Measuring Decision Making in a Business Game A Case Study of a Capital Investment Simulation Combining Experiential and Clinical Methodologies in a Small Business Management Program Results of Using Gaming to Teach Ethics and Social Responsibility Business and Society: An In-Basket Simulation Crisis versus Non-Crisis Simulation Gaming Warm-up Company: A Business Simulation Business Simulations: Competition or Learning A Computerized Management Training System for Franchized Dealers Mode I Stores, Inc.: Computer Supported Cases on the Marketing Research and Problem Solving Process Experiential Learning in Statistics Through the Computer The COMPUSTAT Analysis System as an Instructional Resource Research on Experiential Learning: Enhancing the Process A Comparison of Lecture-Case Study and Lecture-Computer Simulation Teaching Methodologies in Teaching Minority Students Basic Marketing Student Evaluation of Reaction to a Marketing Simulation Game Under Varying Circumstances The Validity and Usefulness of Packaged Models in Game Play The Effectiveness of Experiential Methods in Training and Education: A Review Guidelines for the Future Development of Business Games The Future Potential of Structures Learning Exercises Turning Them on to Management by Turning Them Out to Managers with Video Tape Recorders