A PATH ANALYTIC STUDY OF THE EFFECTS OF ALTERNATIVE PEDAGOGIES Developments in Business Simulation & Experiential Learning, Volume 11, 1984 115 A PATH ANALYTIC STUDY OF THE EFFECTS OF ALTERNATIVE PEDAGOGIES Alvin C. Burns, Louisiana State University Dan L. Sherrell, Louisiana State University ABSTRACT An experiment comparing the use of: written answers to discussion questions; group case analysis, experiential exercise; and microcomputer simulation is conducted. As hypothesized, no meaningful increases in learning occur for the more participative methods over the one-way approach. Significant gains in attitudes are found, however. The application of path analysis reveals causal linkages among attitude components for the various teaching methods. Also, the suggestion of negative moderator effects played by attitude on learning is revealed. The authors discuss two possible explanations for this finding. INTRODUCTION The business educator has several alternative methods from which to choose in the teaching of a topic, and there are innumerable variations of any one method. For the purposes of convenience, however, one can distinguish four main pedagogies. These are: the one-way delivery; the team case study; the experiential exercise; and the computer simulation. Each method has unique aspects, proponents, and special requirements. Investigations as to the relative effectiveness of these, and other, methods as learning facilitators range across a broad spectrum. At one end are primarily anecdotal descriptions of their use; while at the other end are found rigorous experimental designs. At the same time, many different kinds of subject matter have been investigated in addition to those taught in business courses. Kulik, Kulik, and Cohen [9] have performed a meta-analysis of research concluding that the most common approach has been to compare some pedagogy with the “conventional” method of one-way delivery with review questions. They note that the vast majority of this research has not embodied good experimental design. A similar observation has been reached by Dekkers and Donatti [6]. Nonetheless, it is possible to summarize the general pattern of findings unearthed by previous comparative studies. With respect to computer simulation, the meta-analysis concluded that the most pronounced advantage of computer-based instruction involved the time savings for the instructor, while more favorable student attitudes toward the pedagogy and subject matter emerged. Only slight positive achievement differences have been associated with computer assisted instruction. Other studies have pointed out that simulation has positive impact on satisfaction and involvement with the learning experience [10]. This conclusion is substantiated by [6] based on their review of the research. However, Bredemier and Greenblat [31 have claimed that greater long-term retention is an outcome. The literature sometimes confuses case studies with experiential exercises, and differentiation between the two is difficult at the generalization level. Nonetheless, research findings for either one are generally consistent with those for computer assistance: attitudes, satisfaction with the experience, and involvement are heightened while cognitive gains are not significantly different from one-way delivery [2;4;8]. Positioned against this backgroup, the present study endeavors to compare the affective and cognitive effects of alternative pedagogies in the teaching of a particular business topic. The study adheres to experimental design procedures to derive results which are unconfounded. At the same time, the study makes use of path analysis which is a technique of revealing causal linkages among endogenous factors identified in the attitude-formulation and/or cognitive learning process(es). RESEARCH HYPOTHESES The hypotheses generated for this study were derived from two sources. The first was prior research, and generalizations have been noted above. The second source was predicated on the basic notion of path analysis in which a system of relationships is tested empirically and then examined for deeper understanding of their various causes and effects. Thus, the hypotheses: Hypothesis 1: No differences in cognitive learning will be found between the four methods tested. Hypothesis 2: Less positive attitudes will result with the Discussion Questions method than with alternative methods. Hypothesis 3: Attitude consequences and cognitive learning will exhibit structure similar across the three “participative” methods and dissimilar from the Discussion Question method. METHOD Sample and Procedure The experiment utilized a sample of 85 undergraduate business majors enrolled in 3 sections of the principles of marketing course. Among the factors considered in the choice of these students was the fact that they were relatively unfamiliar with the topic taught in the experiment. Also, sections which met at similar time periods were used to obviate differences between daytime and nighttime students. Similarly, all three sections were taught by the same instructor, thus minimizing any such differences, even though the instructor was not involved in the experiment. Developments in Business Simulation & Experiential Learning, Volume 11, 1984 116 The study was conducted in two sessions five days apart. In the first one, students were provided with a fictitious story that the authors were developing material for an instructor manual associated with a textbook they were writing. The students were given a text chapter on store site selection analysis drawn from Reidenbaugh [11]. The students were required to read the material in the classroom while keeping in mind that they would be asked to apply the material later on. Subjects were next randomly assigned to one of the four treatment conditions. Three of the treatment groups were led to separate rooms, and each was administered its treatment (see below for specifics). An attitude scale consisting of 43 items was administered at the end of the treatment session, and a 25 item true-false test was administered during the subsequent class meeting. Students were not informed that the tests would be administered. Treatments Because of the differences in treatments, it was decided to have some judgmental analysis of the comparability of topic coverage. Thus, 17 graduate students enrolled in a marketing class were provided with each treatment and asked to judge the degree of information contained on: (1) geographical layout; (2) population centers and growth; (3) demographic profiles; (4) competitor locations; (5) competitor attributes; (6) economic conditions; (7) site availability and characteristics; and (8) site location and marketing strategy interactions. From these evaluations, changes were made to equalize content coverage as far as possible. One-way Delivery. This treatment has been largely described. Subjects in this group were given four discussion questions and required to write an essay answer to each. In general, the questions required the students to recall chapter information and to relate it in their own words. Case Study. Subjects were given a four-page report describing a small businessman interested in starting a copy shop in one of two towns, each located next to different universities. Information was provided on trade areas, possible sites, competitor locations and their marketing strategies, population characteristics, and available sites. Subjects were broken into small groups and instructed to formulate a group analysis and recommendation. Experiential Exercise. Here, students were instructed to imagine themselves as the operator of a successful stereo store who was interested in locating a second store in a nearby city. The profile of the present customer bases was described, and census data on both households and retail trade for each city was provided. Two alternative sites were displayed on a map, and information on competitors’ characteristics plus possible marketing strategy choices were detailed. Small groups were instructed to make reports on site selection and marketing strategy appropriate for the stereo store operator. Microcomputer Simulation. The simulation was designed and run using an ATARI 800 microcomputer and TV monitor. It has been described in detail in a previous paper [5]. Using the keyboard, student groups were able to: (1) activate the programmed instruction for the rules of the game; (2) enter the required decision input; and (3) receive the simulation results without the need for instructor intervention. The simulation made ample use of graphics, sound, and took into consideration the sensitivity of demand to promotion, price and store location. Operational of Dependent Measures Two separate dependent measurements were administered to subjects. The first was a 43—item attitude scale, while the second was a measure of cognitive learning consisting of 25 true-false questions. The former items were measured on a 5-point Likert-like scale ranging from “Strongly Disagree” to “Strongly Agree.” Many of the items were derived from Greenblat’s [7] list of propositions on the pedagogical effects of simulations. About a dozen items were formulated specifically for this experiment and designed to measure dimensions of interest, involvement, and perceptions of learning. The objective test was generated from several test banks found in various instructor manuals. They covered the range of topics contained in the assigned chapter reading and various treatments. FINDINGS Factor Analysis The attitude scale was factor analyzed via principal components to determine the various dimensions tapped by the instrument. Varimax rotation was applied to aid in interpretation, and only loadings of (absolute) .50 or greater were considered. Five factors accounting for approximately 63 percent of the total variation in the data were found. (The table has been deleted to conserve space.) These dimensions generally follow Greenblat’s [7] original speculations. The first factor has been labeled a Perceived Knowledge dimension as its 9 items all concern perceptions of the information or understanding gained from the pedagogical experience. Factor 2 pertains to interest and involvement and is labeled the Enjoyment dimension. The third factor concerns Perceived Benefits of participation, while the fourth factor alludes to Student-Teacher Relations. The final factor is somewhat akin to Decision Skills. In sum, the five factors represent fairly diverse dimensions of student attitudes toward different teaching approaches. The internal stability of the factors was addressed through inspection of Cronbach alpha reliability coefficients. Findings for each factor were as follows: Perceived Knowledge, .92; Enjoyment, .84; Perceived Benefits, .87; Student-Teacher Relations, .81; and Decision Skills, (-).73. Thus, the five subscales displayed acceptable levels of reliability. Differences Tests Given the experimental design and the application of factor analysis, it was appropriate to apply analysis of variance to determine the effects of the treatments on the dependent measures. Statistically significant differences beyond the .05 level were determined in five of the six cases, and the other was found at the .06 level. Consequently, post hoc analyses of each set of means were performed with Duncan’s multiple range test. Tables are deleted to conserve space but Figure 1 displays the means for ease of comparison. Developments in Business Simulation & Experiential Learning, Volume 11, 1984 117 FIGURE 1 GRAPHICAL PRESENTATION OF TREATMENT MEANS With regard to the first hypothesis, the findings for the percent correct in the true-false test revealed no significant differences between Discussion Questions, the Experiential Exercise, and the Microcomputer Simulation. The Case Study group mean was not significantly different from the Microcomputer Simulation group mean; although, it was for the other two methods. The hypothesis, therefore, is largely supported with this exception. In all five cases, the attitude subscale mean for the Discussion Questions group was lower than the means for the other groups, and in two of them (Perceived Benefits and Student-Teacher Relations), it was significantly different from all other group means. For Perceived Knowledge, Enjoyment, and Decision Skills, the Discussion Questions group mean was not different from either the Case Study or the Experiential Exercise group mean. However, in all instances, the Microcomputer Simulation group attitude subscale means were different from the Discussion Questions counterparts. Thus, the second hypothesis was strongly supported in two out. of five of the attitude measures but only received qualified support for the other three. Path Analysis The identification of attitude subscales and findings of statistically significant differences among them as well as results for the cognitive learning measure is typically the stopping point for most empirical studies in this area. However, it is possible to apply the technique of path analysis for specifying and testing causal linkages between the various variables. Inasmuch as path analysis is not generally encountered in ABSEL publications, this section of the paper will present a short exposition and then proceed with its application. A handy reference on causal modeling is Asher’s [1] monograph. Consequently, most of the following comments are attributable to his work, but in no way do they completely describe the many nuances of path analysis. Asher points out that this method is essentially a heuristic device which forces the investigator to conceptualize variables as a system of causal linkages, or network, of paths between them. This network is cast as a series of simultaneous linear equations which each path represented by a beta coefficient. Regression procedures are applied to the data set to determine the significance and/or magnitude of the beta weights. A primary value of path analysis, apart from its mandate that the investigator formulate the causal model a priori, is its ability to determine both direct and indirect, or moderated, effects of variables in the system. Another important advantage is its ability to use survey data, rather than data generated by experiment, to reveal the causal paths. The heuristic nature of path analysis comes about from the refinements of the causal model based on the empirical findings. With these cursory observations on path analysis, the authors will proceed to its application. The initial phase requires a conceptual alignment of the variables into a causal system. Figure 2 presents the a priori formulation based on logical analysis. Here, the teaching method is cast as the single exogenous variable, external to the system of endogenous attitude/perceptual constructs, and learning is depicted as the final outcome. The attitude variables were case as intermediate or intervening factors mitigating the learning process. This formulation specifies that the teaching method will have three separate direct effects on: Decision flaking Skills, Perceived Knowledge, and Enjoyment. Decision Skills and Perceived Knowledge, in turn, impact the Perceived Benefits construct while Enjoyment is hypothesized to be a primary determinant of Student-Teacher Relations. This factor and Perceived Benefits were assumed to be direct determinants of Learning. FIGURE 2 THE A PRIORI CAUSAL MODEL The data set presented a complication in that it embodied experimental groups for the Method variable. Consequently, it was necessary to use dummy coding for this variable (use of the Method; 0=use of any other Method) and to perform four separate path analyses. Subsequently, ordinary least squares regression procedures were applied as the system was assumed to be nonrecursive. All specified paths were 72 71 70 69 68 67 5 4.8 4.6 4.4 4.2 4.0 3.8 3.6 3.4 3.2 3.0 2.8 2.6 2.4 2.2 2.0 Case Study Microcomputer Simulation Experiential Exercise Discussion Questions True-False Test Results Perceived Knowledge Decision Skills Enjoyment Perceived Benefits Student-Teacher Relations Enjoyment Student-Teacher Relations Method Perceived Knowledge Decision Skills Perceived Benefits Learning Developments in Business Simulation & Experiential Learning, Volume 11, 1984 118 tested to be nonzero as were paths of all antecedent variables tested to be zero. The results are reported in the forms of trimmed models in Figures 3, 4, 5, and 6. In these, the coefficients displayed are only those standardized betas of magnitude (absolute) .10 or greater. This cutoff value is consistent with Asher [1]. FIGURE 3 TRIMMED PATH ANALYSIS RESULTS FOR CASE STUDY FIGURE 4 PATH ANALYSIS RESULTS FOR MICROCOMPUTER SIMULATION FIGURE 5 PATH ANALYSIS RESULTS FOR EXPERIENTIAL EXERCISE FIGURE 6 PATH ANALYSIS RESULTS FOR DISCUSSION QUESTIONS The results reveal a rich set of causal paths within the systems identified. Unfortunately, the small sample size and exploratory nature of this study disallow more than general observations. In particular, it is apparent that the teaching method has a number of direct effects on attitudinal variables. The magnitudes and signs of the path coefficients reveal differential effects as well. For instance, the Microcomputer Simulation appears to effect direct, positive, and sizeable influences on Decision Skills, Perceived Knowledge, Enjoyment and Student-Teacher Relations. The Discussion Questions Method, on the other hand, generates negative influences on these same variables. The other two methods demonstrate a mixture of positive, negative, and even nonconsequential effects on the attitudinal constructs. A second general observation pertains to the substantial internal structure of the attitudinal variables. Path coefficients are all positive and most are robust; furthermore, the trimmed models are clearly suggestive of more complex relationships within the attitudinal constructs than was suspected in the original formulation. These findings suggest a refinement, probably in the form of recursive relationships, for future research. Finally, there is the persistent indication of negative or non-consequential effects of attitude components on learning. In fact, the only positive effect determined for learning flows from the use of the Experiential Exercise. In sum, the findings suggest that the third hypothesis is correct in its claim that the Discussion Questions method differs from the others, but it is altogether too simplistic in its failure to separate methods from each other. DISCUSSION This study has corroborated and extended findings of studies of the differential effects of alternatives to one-way teaching. Corroboration pertains to the positive attitudinal consequences of the use of more involving pedagogies attended by no real net gains in learning. The extention pertains to the specification of a series of attitudinal subcomponents which affect each other in a systematic pattern. Finally, there is the suggestion that methods which generate positive attitudes may do so at the expense of learning. Enjoyment Student-Teacher Relations Method Perceived Knowledge Decision Skills Perceived Benefits Learning -.16 -.10 .26c .17 .11 -.18 .25 .52a -.20 -.25b -.18 Enjoyment Student-Teacher Relations Method Perceived Knowledge Decision Skills Perceived Benefits Learning -.16 .26c .64a .24c .40a -.12 .28 .32c -.22 .13 .15 .25c ap < .01 bp < .05 cp < .10 Enjoyment Student-Teacher Relations Method Perceived Knowledge Decision Skills Perceived Benefits Learning .17 .11 .24c .17 .26c .24c .52a .22 -.21 -.18 Enjoyment Student-Teacher Relations Method Perceived Knowledge Decision Skills Perceived Benefits Learning ap < .01 bp < .05 cp < .10 -.27b -.29b -.39a -.27a .42a .11 .24c .23c -.17 -.29b .15 -.21 Developments in Business Simulation & Experiential Learning, Volume 11, 1984 119 Precisely why this suggestion has eventuated is a subject for conjecture. At least two competing explanations can be identified. The first might be termed an “interference hypothesis.” This hypothesis holds that teaching methods which require an investment of student involvement and generate positive attitudes toward the experience, course, instructor, and subject matter may actually inhibit learning of basic material. Given the context of the experiment, one would be forced to conclude that the participative method tends to attenuate affective outcomes which somehow interfere with retention of material read immediately prior to the experience. However, a second explanation could be couched as a “testing hypothesis.” In this explanation, the true nature of effects of participative teaching methods transcends the simple true-value test administered to measure learning. Here, one would argue that the consequent student beliefs about enjoyment with the experience, knowledge gained, decision skills acquired, better student-teaching relations, and overall benefits will ultimately affect behavior. Thus, the real measure of learning would be a multifaceted model of behavioral and cognitive acquisition consequences including class participation, group-task interaction, daily preparedness, ability to generalize to new situations, decision making capacities, or even recommendations to friends. At this point in time, the authors (and ABSEL members as a whole, we suspect) tend to endorse the latter explanation. REFERENCES [1] Asher, Hebert B., Causal Modeling, 3, Series: Quantitative Applications in Social Sciences, Sage Publications, Beverly Hills, 1976. [2] Basuray, Tom and Steven A. Scherling, “Cognitive Complexity in Lecture versus Experiential Organizational Behavior Classes,” Journal of Experiential Learning and Simulation, Vol. 1, Fall, 1978. [3] Bredemier, Mary E. and Cathy Stein Greenblat, “The Educational Effectiveness of Simulation Games: A Synthesis of Findings,” Simulation and Games, Vol. 12, No. 3, Sept., 1981, pp. 307-333. [4] Brenenstuhl, Daniel C. and Ralph F. Catalanello, “The Impact of Three Pedagogue Techniques on Learning,” Journal of Experiential Learning and Simulation, Vol. 3, No. 1, June, 1979, pp. 221-226. [5] Burns, Alvin C. and Dan Sherrell, “A microcomputer Simulation for Teaching Retail Location Strategy,’ Journal of Experiential Learning and Simulation, Vol. 3, Nos. 3/4; Dec., 1981, pp. 239-246. [6] Dekkers, John and Steven Donatti, “Integration of Research Studies on the Use of Simulation as an Instructional Strategy,” Journal of Education Research, Vol. 74, No. 6, July-August, 1981, pp. 424-427. [7] Greenblat, Cathy Stein, “Teaching with Simulation Games,” Teaching Sociology, Vol. 1, No. 1, October, 1973, pp. 62-83. [8] Hoover, Duane J. and Carlton J. Whitehead, “An Experimental Evaluation of a Cognitive-Experiential Learning Methodology in the Basic Management Course,” Journal of Experiential Learning and Simulation, Vol. 2, No. 1, January, 1979, pp. 119-126. [9] Kulik, James A., Chenlin C. Kulik, and Peter Cohen, “Effectiveness of Computer-Based College Teaching: A Meta-Analysis of Findings,” Review of Educational Research, Vol. 50, No. 4, Winter, 1980, pp. 525-544. [10] Pierfy, David A., “Comparative Simulation Game Research,” Simulation and Games, Vol. 8, No. 2, May- June, 1977, pp. 255-268. [11] Reidenbaugh, Larry D., Retailing Management: A Planning Approach, McGraw-Hill, New York, 1976, pp. 146-167. Table of Contents Volume 11, 1984 Simulation Gaming as a Means of Researching Substantive Issues: Another Look A Further Test of the Group Formation and its Impacts in a Simulated Business Environment Impact of Economic Patterns on Student Performance in Computer Business Simulation Games Majority Fallacy Game with Independent Student Simulation and a Case Introducing the Marketing Channel Laboratory A Comparative Evaluation of a Marketing Game A Study of Comparative Effectiveness of Problem-Solving Technologies The Impact of Hierarchical and Egalitarian Organization Structure on Group Decision Making and Attitudes Risk-Free Decision Making The EX-STRA Export Strategy Game Computer Education for Management Students Developing a Computer Game/Job Simulation to Teach Functional Literacy Skills Experiencing Socialization First Hand: An Experiential Exercise in Organizational Socialization Networking Distributive Versus Integrative Approaches to Negotiation: Experiential learning Through a Negotiation Simulation Managerial Education and the Real World: Foudations for Designing Educational Tools Diagnosing Group Climate to Improve Supervisory Effectiveness Student background as a Factor in Simulation Outcomes: The Collective bargaining Example The Use of Pre-Plays in Management Education Experiencing the Process Debrief: A Workshop ABSEL Megatrend Roots MEGATRENDS for Business Simulation and Experiential Learning The Effects and Consequences of the Megatrends on Simulation Gaming: One View Opportunities for the Future: ABSEL's Role Experiential Learning-Based Discussion vs. Lecture Based Discussion: A Comparative Analysis in a Classroom Setting An Evaluation of the Minitab Package in Teaching Business Statistics Concepts A Path Analytic Study of the Effects of Alternative Pedagogies Developing and Using Weighted Application Blanks: An Experiential Exercise Building Airplanes Individual vs. Group Grade: An Exercise in Decision making A Marketing Plan Exercise: Development of Interteam Cooperation Using a Coordinated Experiential Approach Using Student Experience as the Basis for a Consumer Behavior Learning Exercise Student Evaluations of Instructors: What do Students Believe? A Description of the SOFTCAT Computer Assisted Teaching System Comparisons of Practitioners' and Professors' Perceptions of Business Policy Content and Learning Methods The Perceived Relationship Between Pedagogies and Attaining Course Objectives in the Business Policy Course The Use of Simulation in the Teaching of Business Policy A Research Study on Strategic Decisions in a Business Simulation Strategic Management Decision Making Researched Via Simulation Gaming Using Simulation to Investigate Factors in Competitive Bidding Combining Experiential Learning and management Assistance A Model for Teaching Management Skills Putting Experience Back into Experiential Learning: A Demonstration The Teaching and Behavioral Measurement of Managerial/Organizational Competencies: Developing Experiential Exercises and Simulations A Simulation Game Model for Conglomerates QCLAB - A Microcomputer Laboratory in Quality Control CTSS: A Commodity Trading Simulation System Problem Solving: An Exercise on Learning, Coaching, and Operant Conditioning A Demonstration of the Effects of Feedback as a Category of Reinforcement The Assessment of Feedback and Disclosure in Interpersonal Relations: An Experiential Exercise A Study to Determine Whether the Teaching of Basic Grammar Skills in Business Communication Classes Improves Students' Business Letter Writing Corporate Maladies Through the Eyes of the Memo Writer: A Seldom Used Experiential Tool Executive Bailout at Shake & Spear, Inc. The H.E./L&P Merger Intercultural Nonverbal Communications: An Experiential Exercise The Evolving Business Policies Course - Is Management Gaming the Logical Pedagogy? The Use of Decision Simulations in Management Training Programs: Current Perspectives Humanizing the Business of Medicine: The Use of Simulated Patients to Train medical Students Systematic Integration of Simulation Methods in a Graduate Management Curriculum Modeling Non-Price Factors in the Demand Functions of Computerized Business Using Spacial Relationships to Estimate Demand in Business Simulations Two Algorithms For Redistribution Of Stockouts In Computerized Business Simulations Leadership And Strategic Behavior A Comparison Of Two Business Strategy Simulations For Microcomputers Incorporating Decision Support Systems Into Management Simulation Games: A Model And Methodology Using Micro-Computers To Support The Analysis Of Complex Cases: It's As Easy As 1-2-3 Strategic Formulation Consistent With Pims: A Micro-Computer Application