DOES STUDENT PREPARATION MATTER IN A SIMULATION? A COMPARISON OF PEDAGOGICAL STYLES Developments in Business Simulation and Experiential Learning, Volume 29, 2002 DOES STUDENT PREPARATION MATTER IN A SIMULATION? A COMPARISON OF PEDAGOGICAL STYLES Snow, Sheldon Graziadio School of Business and Management Pepperdine University ssnow@pepperdine.edu Gehlen, Frieda Graziadio School of Business and Management Pepperdine University fgehlen@pepperdine.edu Green, Jack Graziadio School of Business and Management Pepperdine University jgreen@pepperdine.edu PEDAGOGY AND EXPERIENTIAL LEARNING ABSTRACT Even though the simulation is an experiential form of learning, and presumably valued for that, the method by which it is introduced is often a traditional classroom discussion. A comparison of that approach with one that integrates the simulation into the course and includes practice sessions shows that the latter approach increased student confidence in their ability to do well on the simulation and the amount of effort they expected to put into it. Higher levels of confidence were, in turn, associated with the degree of importance they placed on succeeding in the simulation and the level of effort. Greater effort and confidence were associated with the perception of being effective. Their perception of their effectiveness, along with a sense of team cohesion, were significant in predicting whether they perceived the simulation to be a valuable learning experience. The fact that simulations are widely used does not imply that they are used uniformly, e.g., the manner in which they are introduced, the percentage of class time and effort devoted to them, the impact on the students' grades, or the degree to which the simulation is integrated with the rest of the class material. In fact, barring evidence to the contrary, the opposite is assumed to be true. Learning theories suggest that these differences will affect how and what students learn from the experience. "Behavior, design, and atmosphere variables, both alone and in combination, probably produce unique and substantial effects on student learning in simulation experiences" (Gosen and Washbush, 1999, p. 302). The work of David Kolb (1984) on experiential learning has been very influential in simulation research. Herz and Merz (1998) used Kolb's model to compare student learning between groups that participated in a simulation experience with those in a traditional seminar and found that "the simulation/game seminar outperforms a conventional seminar with respect to all aspects of the learning cycle" (p. 248). However, they note that "game complexity and prior knowledge strongly influence the learning process of participants" (p. 249). Business simulations are a commonplace part of current business school education. Faria's (1998) extensive survey of both academia and business found that more than a quarter of all collegiate business teachers use a simulation in any given term, and more than 60% of large businesses use them in their training programs. He also reports on a survey of AACSB-member business school deans which suggests that close to 98% of all AACSB member schools use at least one simulation somewhere in their programs. Also referencing Kolb, Burns and Gentry (1998) focus on motivation to learn. When talking about preparation for a simulation, they state that a certain degree of tension or gap between what is known and what can be learned is useful for motivation, but that too large a gap may be discouraging. 208 mailto:Sheldon.Snow@pepperdine.edu mailto:Frieda.Gehlen@pepperdine.edu mailto:Jack.Green@pepperdine.edu Developments in Business Simulation and Experiential Learning, Volume 29, 2002 That is, students with a high internal drive to master material may not be discouraged by a lack of information about the process, but students whose primary motivation is external -- e.g., receiving a good grade or completing a degree -- may not try to succeed if there is a perceived "information overload created by the need to understand the structure of the exercise on top of the need to understand the underlying theory" (p. 147). They further note that "the familiarization stage can be extremely frustrating and can result in high levels of anxiety" (p. 146), and therefore instructors need to understand "the knowledge level that students bring to the scene . . . (and) pay close attention to the introduction of the experiential exercises so that the student does not become discouraged at the onset" (p. 147). Wenzler and Chartier (1999) agree, arguing that "Confidence in one's ability to be successful is a prerequisite for having motivation and commitment to act" (p. 381). Similarly, Warr, Allan and Birdi (1999) studied the importance of trainees confidence in their ability to learn material and found that “post-training learning attainment was also predicted by learning confidence.” A separate line of research that may also impact how students perform in simulations has to do with teamwork and cohesion. The literature on the relationship between cohesion and effectiveness is extensive. (See Gully, Devine and Whitney, 1995; Mullen and Copper, 1994; and Mudrack, 1989 as examples of literature surveys and meta- analyses.) This paper will not review that literature except to note there is general agreement that cohesion is a multi- dimensional concept and that at least two different dimensions relate to effectiveness: interpersonal attraction and task cohesion (Zaccaro and McCoy 1988; Zaccaro and Lowe, 1988). There is, however, some disagreement as to whether high interpersonal attraction increases or decreases task orientation and effectiveness. Included in the questions to be explored here are potential relationships between these two types of cohesion and other attitudes about the simulation experience. Teams are an integral part of most business simulations; certainly they are of the one examined here. Again, while there is an extensive literature on teams, including the use of teams in learning situations (Kolb, J, 1999; Alie, Beam and Carey, 1998; Roebuck, 1998; Methany and Methany, 1997; Hendrix, 1996), our primary concern here is whether teams which are more cohesive will try harder, be more confident of success, or otherwise be more effective. STATEMENT OF THE PROBLEM A natural experiment of two different ways to introduce a business simulation to a class was used to explore the themes outlined above. The focal course is the capstone strategy class of an MBA program for working adults which includes a required weekend business simulation. Two professors who regularly teach the course have developed different ways of approaching the simulation requirement. One treats the weekend simulation experience as essentially a stand-alone project or enhancement to the course curriculum. It is weighted fairly lightly in terms of the final grade. The second professor makes the simulation an integral part of the course, and accordingly gives it a heavier grade weighting. This paper investigates whether or not these two pedagogical approaches create different attitudes about the simulation among the students about how much effort they put into the experience and whether they view it as providing a real educational benefit. Specifically, we investigate whether there are differences in the degree of importance students place on doing well on the game, the amount of effort they are willing to exert, the degree of confidence they have that they understand the game itself, the degree of cohesion among team members, and the degree to which they found it a useful learning experience. For comparison, these attitudes were measured at the beginning of the simulation weekend and then again a week later, with the latter measurement including questions on how much effort they actually put into the experience, how effective they thought they were in achieving their goals, and how useful they found the simulation to be as a learning experience and as a means of integrating their business education. THE SIMULATION EXPERIENCE Eight classes were studied over two terms, four taught by each of the two professors. The focus of all simulations was a weekend experience that began on a Friday evening, reconvened on Saturday morning, and ended mid-to-late afternoon on Saturday. Teams consisted of three to five students, with most having four members. One class had only three teams, but the others each had four or five teams. Each team was assigned its own room for the weekend while the professors had a separate central-administration room. Decisions were input into computers located in each room and transferred to the central-administration room through a LAN (local area network). The simulation used was The Business Policy Game (4th Edition) by Cotter and Fritzsche (1995). Various additional experiences were provided by professors who played other roles throughout the weekend. These included Insurance Agent, Political Official, Banker, and Union Representative. Interventions by these professors included such things as natural disasters that shut down some operations for a quarter or two, labor disputes, the opportunity to bid for exclusive use of a new technology, political requests for bribes, insurance problems, and the unexpected "hospitalization" of some team members. Some interventions involved adjusting the response of the main computer program to create a modified response to their quarterly inputs. For example, a team that refused to negotiate seriously with labor might find that they had increased material costs due to poor quality, or, a wild-cat strike might shut down a plant or two, halting all production for that quarter. The purpose of the interventions was to 209 Developments in Business Simulation and Experiential Learning, Volume 29, 2002 provide qualitative challenges along with the more structured computerized interactions. While the game scenarios and the interventions therefore varied somewhat from one weekend to another, the faculty involved tried to be certain that all simulations included the same types of problems. It was determined that these differences in scenarios did not significantly or systematically affect the variables under consideration here. An independent observer who was present at all sessions for both professors concurred. PEDAGOGICAL STYLES Style A: Professor A treats the simulation basically as a stand-alone experience that occurs on the one weekend. Students work on other team assignments early in the term. Formal preparation for the simulation does not begin until four or five weeks prior to the simulation weekend. At that time the simulation manual is reviewed in class and general requirements for the simulation are explained. Students develop spreadsheet models for production, purchasing, marketing, finances, etc., and discuss the mechanics of entering decisions and the variables included in the game. These activities take part of the class period each of the remaining weeks prior to the simulation weekend. This particular simulation begins with eight quarters of data that are common to all teams. These are made available to the students ahead of time, and each team prepares a strategic plan that sets forth the team's mission, objectives and strategies for the game and plans its first quarter of entries. Teams present their strategic plans to the professor when they arrive for the weekend. When students arrive on Friday evening they receive instructions regarding the use of the computer system for inputting their decisions and retrieving reports and enter their decisions for the ninth quarter. Professor A reminds the students that the more they put into preparation for the simulation, the more they are likely to gain from it. However,the entire simulation project is weighted at only 10% of the final grade. He basically leaves it up to them to decide how much time and effort to put into preparation on their own outside of the classroom. Given his objectives for the course, he chooses to assign other non-simulation projects that involve significant amounts of out-of-class preparation. Professor A also uses the same scenario of the simulation for each section. Style B: Professor B integrates the simulation into the course work throughout the term. After reviewing the manual with the class early in the term, he sets aside one- half a class period, or approximately two hours, for teams to go to the computer lab and begin making trial decisions for one scenario of the game. They download their decisions onto diskettes and turn them in to the professor who enters them into his computer, which has the simulation installed on it. The results are returned to them on their diskettes, and they generate reports. This process is repeated for two or three decisions. The professor reviews the overall results of decisions with the class as a group, using them as a means of illustrating key strategy concepts as well as illustrating how certain strategies tend to affect outcomes in the simulation game. At the conclusion of these trial decisions, this professor distributes the initial eight quarters of financial and operating statements for a different scenario of the game, and the actual simulation is begun. Not only is the actual simulation a different scenario from the trial period, Professor B chooses to use a different scenario of the game for each section of the class. The goal is to be sure that students who go through the weekend first cannot inadvertently, or otherwise, “tip off” later participants as to how the simulation responds to certain inputs. Teams have two weeks to bring their first set of decisions to class on a diskette. The results are entered and, at the end of the class period, students receive their diskettes back with the results from that quarter. This is repeated each week for three more weeks, with the final decision entered two weeks prior to the simulation weekend. While Professor B answers questions as they arise during this period, there is little formal instruction about the simulation after the one initial class. A strategic plan is required from each team based on the situation of that team at the end fourth decision. . (This gives each team a total of 12 quarters of information.) When teams arrive for the weekend simulation experience, their four quarters of decisions already have been transferred to the LAN, and they begin the weekend making decisions for the thirteenth quarter. Minimum time is required to explain the mechanics of the simulation since they have already been doing it. Professor B weights the various elements of the simulation more heavily in terms of the grade than does Professor A. Altogether, simulation-related activities account for 25% of the grade in his class. MAJOR SIMILARITIES AND DIFFERENCES Both professors allow students to self-select their teams. Classes in this MBA program typically have fewer than 25 members, so many students know each other well and have worked together on projects in the past. These students are adults with full-time jobs -- and often with families -- who frequently live many miles from the school site and from each other. It is important to them to be able to select their team members, at least in part because of logistical and time constraints. Approximately 60% the respondents in this study had worked with one or more of their team members on at least one previous project. Both professors spend about the same amount of in- class time on preparation for the simulation although they use that time quite differently. Since Professor B requires decisions on a regular basis over several weeks of the term, students must spend a significant amount of time outside of class learning about the simulation and how it reacts to their input as well as refining their strategy. 210 Developments in Business Simulation and Experiential Learning, Volume 29, 2002 The basic difference is that the students in Professor A's courses have not had any "hands-on" experience with the simulation prior to arriving for the weekend and they all begin in the ninth quarter with identical historical information. Professor B's students, on the other hand, begin the weekend in the thirteenth quarter with the experience of having made decisions, both during the practice session and for four quarters of the game. From our theoretical perspective, the fact that Professor B's students begin the intense weekend experience having already become familiar with how the simulation operates and the kind of results they are likely to get from various types of decisions is of particular interest. Additionally the two professors differ in the weight of the simulation on the final grade. Professor A weights it at 10% of the grade while Professor B weights it at 25%. (In both cases this includes the writing of the strategic plan and the analysis after the weekend of how well the team did relative to its own team goal and how well it followed its strategy.) It is impossible in this type of “natural experiment” to separate out the effects of the manner in which the simulation is introduced from the effect of the weight of the grade. In part, they are a package reflecting the relative importance that each of the professors places on the simulation as a learning experience. This difference in attitude undoubtedly is conveyed to the students as well. We assume it has some impact on the attitudes they hold as well. While we focus in this paper on the experiential introduction as the key independent variable, we are aware that difference in grading policies and the professor’s enthusiasm may have some independent effect that we cannot separate out. METHODOLOGY The data reported here were collected as part of a larger study designed to use the simulation experience as a laboratory to investigate the relationship of teamwork, team cohesion, and effectiveness. This study deals only with those data that are relevant to the focal research question. Students completed a questionnaire at the beginning of the simulation weekend that included a six-item inter- personal attraction cohesion scale, questions about the importance they personally placed on succeeding in the simulations, the amount of effort they were willing to make in order to do so; and how confident they were that they understood how to do the simulation. They were also asked how important they believed the simulation was to their team members and how much effort they thought the team would make. Finally, they were asked about team-building exercises and the relative percent of their team preparation time that was spent on task activities versus socializing. Students completed a comparable questionnaire at the debriefing session a week later that included the same cohesion scale and asked about the amount of actual effort they and their teams made. Based on conversations with the students at the end of the first term, it was decided to add two questions to the post-simulation survey the second term: How useful was the simulation in helping you integrate your business education? And, how much did you learn from the simulation overall (about business, about teamwork, etc.)? Research on group cohesion and its impact on group performance dates back to the work of Festinger (1950). Despite that lengthy history and numerous studies, operationalizing the concept of group cohesion has proved very difficult. Rather than attempt to create yet another scale, we chose to use the one reported in Zaccaro and Lowe (1986). The interpersonal attraction scale consists of six items which are measured using an 11-point Likert scale, with "1" being the most positive response and "11" the most negative. Respondents were asked to rate their teams on the following characteristics: warm, dependable, pleasant, courteous, friendly and likeable. The scores on the six items were totaled and divided by six to create a single cohesion score. Because the positive end of the scale is anchored at "1" and the negative end by "11," the lower the numerical value of the score, the greater the degree of expressed cohesion. The task-oriented questions about importance and effort and the questions about value and usefulness were measured on similar 11-point Likert scales. Individuals were given anonymity in order to encourage honesty, but they did identify their teams. There were 153 students in the eight sections combined, 57 in Professor A's classes and 96 in Professor B's. For the two questions that were included only in the second term, the N's were 33 in Professor A's classes and 46 in Professor B's. Missing data were treated using pairwise deletion. Responses were analyzed using SPSS. PRESENTATION OF THE DATA To facilitate data presentation, shortened variable names are used. The names and the corresponding definition of the variables are given in Exhibit 1. 211 Developments in Business Simulation and Experiential Learning, Volume 29, 2002 Exhibit 1: Variable Names Variable Name Variable Description Cohesion 1 Cohesion Scale Score from Pre-Simulation Administration Cohesion 2 Cohesion Scale Score from Post-Simulation Administration Personal Importance Importance to the individual of succeeding on the simulation. Team Importance Perceived importance to the team of succeeding. Personal Effort Effort the individual is willing to make to succeed. Team Effort Perceived effort the team is willing to make to succeed. Confidence Confidence level about understanding how to do the simulation. Actual Effort Perceived effort actually made by the team. Effective Perceived team effectiveness in reaching its goals. Number Number of teammates with whom the respondent had previously worked. Pedagogy Style of introducing the simulation to the class. Task Percentage of team planning time that was task related rather than social. Build Whether the team engaged in deliberate team-building exercises. Useful Perceived usefulness of the simulation in integration of business education Learn Perceived value of simulation as an overall learning experience. Our major hypothesis was that the difference in pedagogical styles would impact attitudes toward the simulation experience, although we did not hypothesize a direction. The descriptive statistics for the variables, broken down by pedagogical style, are presented in Table 1. (The lower the mean or mode, the more positive the response.) 212 Developments in Business Simulation and Experiential Learning, Volume 29, 2002 Table 1: Descriptive Statistics and Test for Equality of Means by Pedagogical Style Variable Simulation as Stand-Alone Professor A1 Simulation Integrated Professor B2 t-test of means Mean Mode Std.D ev Mean Mode Std.D ev Mean Differ . t df Sig Cohesion 1 2.66 1 1.63 2.33 1 1.20 0.33 1.32 92.57 .190 Cohesion 2 2.37 1 1.51 2.38 1 1.61 -0.01 -.03 119.4 .98 Personal Importance 3.32 1 2.63 2.43 1 1.88 0.89 2.24 90.95 .028 Team Importance 3.47 2 2.34 2.47 1 2.09 1.00 2.72 148 .007 Personal Effort 2.70 1 1.89 1.85 1 1.32 0.85 2.99 89.55 .004 Team Effort 3.11 2 1.83 1.98 1 1.14 1.13 4.16 81.26 .000 Confidence 4.40 3 2.53 2.81 2 1.66 1.60 4.24 85.99 .000 Actual Effort 2.80 2 1.89 1.86 1 1.14 0.94 3.35 78.05 .001 Effective 3.64 3 2.13 2.68 1 2.03 0.95 2.70 77 .024 Useful 3.82 3 2.59 2.35 1 2.62 1.47 2.48 69.51 .016 Learn 3.48 2 2.31 2.28 1 2.28 1.20 2.30 68.56 .025 1N=57 for all but last two variables, where it was 33. 2N=94 for all but last two variables where it was 46. First, no significant differences in measures of team cohesion were detected between the two sets of students, and the cohesion scores are quite positive. The two pedagogical styles appear not to have made a significant difference on cohesion. This is not surprising since both approaches make extensive use of teams and allow students to self-select teams. Next, all students appear generally positive about the simulation. Each of the mean scores, with one exception, was less than a “4” on an 11-point scale with “1” being the most positive response. The one exception was the Confidence variable among Professor A's students. That mean score was 4.40. There was a pattern among the variables other than the cohesion variables, however. The mean scores were more positive on each of these variables for the classes where the simulation was integrated into the course compared to the stand-alone approach. The difference in means was less than two scale points in all cases, but it was statistically significant using a t-test. The modal response in the classes where the simulation was integrated was “1” on all variables except the question about confidence, where it was a "2." In the classes with the stand-alone approach, the modes of the variables ranged from “1” to “3”. Individual respondents within both sets of students tended to rate themselves more positively on the importance and effort scales than they rated their teams, but the differences were larger among students in stand-alone classes. The results in Table 1 suggest that the different pedagogical approaches did result in somewhat different attitudes among students about the importance of doing well on the simulation, the effort they were willing to make, their level of confidence in their understanding of the game, the usefulness of the simulation for integrating one's business education, and overall degree to which the simulation contributed to learning something valuable. Having the introduction to the simulation integrated into the rest of the course was associated with more positive responses on all of these measures. We next began to look for relationships among variables. An initial exploration of the variables indicated some multicollinearity. To limit the possible effects of collinear relationships, the decision was made to use stepwise multiple regression. This procedure enters the independent variables into the regression equation in the order in which they add to the explanation of the variance, thus minimizing the likelihood of a variable being selected as significant if most of its effect has already been explained by another variable to which it is related. A check of the Variance Inflation Factor (VIF) statistics for each regression equation indicated that there was no significant multicollinearity that remained when using stepwise regression. 213 Developments in Business Simulation and Experiential Learning, Volume 29, 2002 Table 2 shows the regression statistics and models for the variables measured prior to the simulation experience. The dependent variables in which we were interested were Personal Importance, Team Importance, Personal Effort, Team Effort, and Confidence. As general independent variables, we selected Pedagogy, Cohesion 1, Task, and Number. Confidence was also included as an independent variable in equations other than the one where it was the dependent variable. Personal Importance was entered as an independent variable in the equation for Personal Effort, and Team Importance was included in the analysis of Team Effort. To analyze Confidence as a dependent variable, the independent variables used were Pedagogy, Cohesion 1, Number, and Task. Table 2: Pre-Simulation Dependent Variables -- Stepwise Regression Models Dependent Variable Model R R2 F DF Sig. Level Personal Importance 1. Cohesion 1 .394 .155 26.238 1, 143 .000 2. Cohesion 1, Confidence1 .515 .265 21.220 1, 142 .000 Team Importance 1. Cohesion 1 .330 .109 17.485 1, 143 .000 2. Cohesion 1, Confidence .411 .169 14.395 2, 142 .000 3. Cohesion 1, Confidence, Number2 .443 .196 11.490 3, 141 .000 Personal Effort 1. Personal Importance .697 .486 135.227 1, 143 .000 2. Personal Importance, Cohesion 1 .717 .514 75.020 2, 142 .000 3. Personal Importance, Cohesion 1, Confidence1 .730 .532 53.481 3, 141 .000 Team Effort 1. Team Importance .624 .390 91.243 1, 143 .000 2. Team Importance, Cohesion 1 .762 .581 98.288 2, 142 .000 3. Team Importance, Cohesion 1, Pedagogy .790 .624 70.344 3, 141 .000 4. Team Importance, Cohesion 1, Pedagogy, Confidence3 .799 .638 54.018 4, 140 .000 Confidence 1. Pedagogy .358 .128 21.054 1, 143 .000 2. Pedagogy, Number4 .400 .160 13.508 2, 142 .000 1 Excluded variables in the model: Pedagogy, Task, Number; 2 Excluded variables in the model: Pedagogy, Task; 3 Excluded variables in the model: Task, Number; 4 Excluded variables in the model: Task, Cohesion 1 There is an obvious pattern in the results reported above. Cohesion 1 and Confidence are included in the models for each of the Importance and Effort variables, with Cohesion 1 being entered in the equation before Confidence in each case. The relationships are all statistically significant at p>.001. 214 Developments in Business Simulation and Experiential Learning, Volume 29, 2002 Looking specifically at pedagogy, which was our key variable of interest, there were only two models in which pedagogy was a statistically significant variable in the equation: team effort and confidence. In the case of confidence, pedagogy was the first variable to enter the equation with an R2 of .128. It was the third variable in the model for Team Effort. The degree to which it was important to the students individually and to their teams to do well on the simulation was the key variable in the models for the Expected Effort variables. The results of the analysis of questions asked following the simulation experience are given in Table 3. Potential independent variables included were Cohesion 1, Confidence, and Pedagogy. In addition, Team Importance and Team (expected) Effort were included in the analysis of Actual Effort. Actual Effort, in turn, was included in the analysis of Effective. Table 3: Post-simulation Dependent Variables -- Stepwise Regression Models Dependent Variable Model R R2 F DF Sig. Level Actual Effort 1. Team Effort .494 .244 45.433 1, 141 .000 2. Team Effort, Confidence1 .528 .279 27.043 2, 140 .000 Effective 1. Actual Effort .538 .289 57.04 1, 141 .000 2. Actual Effort, Team Effort .558 .311 31.57 2, 140 .000 3. Actual Effort, Team Effort, Confidence .590 .348 24.696 3, 139 .000 4. Actual Effort, Team Effort, Confidence, Team Importance2 .613 .376 20.760 4, 128 .000 Useful 1. Effective3 .588 .346 38.654 1, 73 .000 Learn 1. Effective3 .513 .263 26.107 1, 73 .000 1 Excluded variables: Cohesion 1, Confidence, Pedagogy, Team Importance 2 Excluded variables: Cohesion 1, Pedagogy 3 Excluded variables: Cohesion 1, Confidence, Pedagogy, Actual Effort, Team Importance It is immediately apparent that the variables that explained most of the variance in the pre-simulation attitude analyses are not primary in the post-simulation evaluations, although Confidence does appear in two of the equations. Instead, there seems to be a progression of attitudinal effects. Among the variables that were available, the best at explaining the variance in Actual Effort was the amount of effort they had predicted their team would make prior to the simulation weekend (Team Effort) and their confidence level. Actual Effort, in turn, was the first variable to appear in the Effective model, suggesting that the more effort they made, the more effective they believed they had been. Effective was, in turn, the only variable that was statistically significant in explaining the variance in the analyses of how useful or valuable the simulation had been as a learning experience. DISCUSSION Although this study involves a non-residential graduate program for working adults, the student grapevine works very well, and war stories about the required simulation experience are passed from class to class. Informally, many students admitted to some level of apprehension about the experience and how well their teams might do in it. Despite that, our data show that students in both sets of classes were generally positive about the simulation. Almost all of them believed it was important to succeed and expected to make a good deal of effort in succeed in the simulation. This provides support for the position that experiential learning has an intrinsic interest for students, including working adults, and engages them in an active and positive way. The primary focus of the paper, however, had to do with the question of whether the manner in which the required simulation was introduced to the students would have an impact on their attitudes toward the simulation experience and its value to them as a learning experience. Two pedagogical styles were compared. In one, the simulation was treated largely as a stand-alone assignment with some class discussion ahead of time about material in 215 Developments in Business Simulation and Experiential Learning, Volume 29, 2002 the manual and some modeling of variables. In the second, the introduction to the simulation was more nearly integrated into the coursework. It included trial runs and the opportunity to make the first few decisions of the actual simulation in a less-pressured situation. Our results show there was a consistent difference in attitudes that was related to pedagogical style, as evidenced by the test of means. The exception to this pattern was interpersonal cohesion variables where there was no significant difference. On all of the other variables, students experiencing the integrated pedagogical style were more positive about the simulation and about its usefulness as a learning experience. Yet it should also be noted that while the differences in means were very consistent and statistically significant, they were not very large. All were less than two points on an 11-point scale. Using Burns and Gentry's (1998) approach, it appears that students who had the simulation introduction integrated into their class had a smaller gap to overcome in terms of the basics of the game than those in the more traditional approach. They were also beginning to have some idea about their competitors' business strategies. For them, the unknowns involved the non-computer-based role-playing of the professors and the much-shorter decision periods. Students with the more limited introduction to the simulation had to engage in a steeper learning curve with regard to the game itself at the same time they were trying to figure out how the other teams were attempting to position themselves. It is not surprising that they were less confident. To further examine factors that might affect student attitudes toward the simulation experience and to test pedagogical style against them, we did a series of stepwise multiple regression analyses looking at the attitudinal variables of importance, effort, confidence, perceived effectiveness, and usefulness and value of the simulation as dependent variables. Pedagogical style appeared as a significant explanatory variable in the equations for only two dependent variables: confidence and the expected team effort. The variance in the level of confidence was best explained by the pedagogical style, and then by the number of teammates with whom the respondent had previously worked. The amount of effort they expected their teams to make was the one other model in which pedagogical style entered the equation. It entered after the variables of how important it was to their team to succeed and the internal team cohesion variable. Therefore, although we know from the t-tests that there were significant differences between students who experienced the two different pedagogical styles, it would appear that any effects from pedagogical style may be indirect rather than global. The literature suggests that confidence is important because it affects how willing students are to engage in the learning experience and therefore to benefit from it. Our results support this. Confidence was a significant variable in explaining the variance in all four of the other pre- simulation attitudinal variables. It also was significant in the analysis of the actual effort put forth in the simulation and the degree to which the students believed they were effective in playing the simulation game. Cohesion was central to the explanation of the pre- simulation responses about the importance of succeeding for both the individual and the team and the effort the respondents expected to make, entering into the regression equation on either the first or second step for all four of these variables. However, it did not appear as directly significant in explaining the variance in confidence or in any of the post-simulation variables: actual effort made, perceived effectiveness, usefulness for integrating business education, and overall value as a learning experience. The relatively small amount of variation in cohesion scores may have limited the significance of findings in this area as well. Although the form of analysis used here does not really allow us to trace effects through several variables to an ultimate dependent variable, there are some interesting possible paths suggested by the data. These data suggest a possible indirect progression of the impact of the pedagogical style on student responses to the simulation and their perception of its educational value for them. Pedagogy entered the equation as the first variable in the Confidence model and as the third variable in the Team Effort variable. Confidence also explained some of the variance on Team Effort. Team Effort was, in turn, the first variable entering the equation on Actual Effort, and Actual Effort was the first predictor variable for how Effective the respondents felt their team had been. Effectiveness, in turn, was the only variable we had that was significant in explaining how useful they believed the simulation to have been as a means of integrating their business education and as a learning vehicle overall. CONCLUSIONS As simulations become more widely used in educational situations, it is important to consider whether they are serving the educational function that is claimed for them. If students are focused on the mechanics of the game or on dysfunctional team dynamics, they will have more difficulty focusing on the underlying purpose of the simulation and what it is designed to teach them about a substantive subject. Therefore, when teachers consider using simulations, they might also consider providing opportunities for the students to practice and become familiar with the mechanics of the game before the formal simulation begins. This would appear to be especially important for building confidence. It would appear that a higher level of confidence affects the level of involvement in the game, which is related to how valuable a learning experience it is perceived to be. It must be acknowledged that this type of introduction can be time consuming, and as such must be weighed against other uses of the time, given the educational objectives of the faculty member for that course. Taken 216 Developments in Business Simulation and Experiential Learning, Volume 29, 2002 together, however, these data suggest that introducing a simulation in an experiential fashion may be a way to increase the educational value of this type of activity. While we suggest a possible set of links or relationships that create this effect, further research is needed to test these propositions individually. Another area for further research would involve segregating out the effect of differential grade weighting and professor enthusiasm or bias toward simulations from the manner in which the simulation is introduced. REFERENCES Alie, R., Beam, H., and Carey, T. (1998) The use of teams in an undergraduate management program. Journal of Management Education, 22, 707-720. Burns, A. C., & Gentry, J. W. (1998) Motivating students to engage in experiential learning: A tension-to-learn theory. Simulation and Gaming, 29, 133 - 151. Cotter, R.V. & Fritzsche, D. J. (1995) The Business Policy Game: An International Simulation, 4th Edition Englewood Cliffs, N.J.: Prentice Hall. Faria, A.J. (1998) Business simulation games: Current usage levels -- and update. Simulation and Gaming, 29, 295- 308. Festinger, L., (1950) Informal social communication. Psychological Review, 57, 271-282. Gosen, J. and Washbush, J. (1999) As teachers and researchers, where do we go from here? Simulation and Gaming, 30, 292 - 303. Gully, S.M., Devine, D. J., & Whitney, D. J. (1995) A meta- analysis of cohesion and performance: Effects of level of analysis and task interdependence. Small Group Research, 26, 497 - 520. Herz, B. & Merz, W. (1998) Experiential learning and the effectiveness of economic simulation games. Simulation & Gaming, 29, 238-250. Kolb, D. (1984) Experiential learning: Experience as the source of learning and development. Englewood Cliffs, NJ: Prentice-Hall Kolb, J. (1999) A project in small-group decision making. Journal of Management Education 23, 71- 80. Metheny, D. & Metheny, W. (1997) Enriching technical courses with learning teams. College Teaching, 45, 32- 36. Mudrack, P.E. (1989) Group cohesiveness and productivity: A closer look. Human Relations, 43, 771 - 785. Mullen, B. and Copper, C. (1994) The relation between group cohesiveness and performance: An integration Psychological Bulletin, 115, 210 - 227. Roebuck, D (1998) Using team learning in business and organizational communication classes. Business Communication Quarterly, 61 (3). 35 - 50. Warr, P, Allan, C., and Birdi, K. (1999). Predicting three levels of training outcomes. Journal of Occupational and Organizational Psychology, 72, 351-375. Wenzler, I. And Chartier, D. (1999) Why do we bother with games and simulations: An organizational learning perspective. Simulation & Gaming, 30, 375 - 384. Zaccaro, S. J., and McCoy, M. C. (1988) The effects of task and interpersonal cohesiveness on performance of a disjunctive group task. Journal of Applied social Psychology, 18, 837-851. Zaccaro, S.J., and Lowe, C. A., (1988) Cohesiveness and performance on an additive task: Evidence for multidimensionality. The Journal of Social Psychology, 128, 547-558. Sheldon C. Snow is Associate Professor of Organization and Management at Pepperdine University’s Graziadio School of Business and Management. He has had prior experience in finance and banking and for the past 32 years has been on the faculty of the Graziadio School. He has had experience in consulting with both profit and non-profit organizations. (310) 325-2786, email ssnow@pepperdine.edu Frieda Gehlen is a Research Associate at Pepperdine University's Graziadio School of Business and Management. She earned a doctorate in political sociology from Michigan State University and an MBA from the Anderson School at UCLA. Prior to coming to the Graziadio School she served on the faculties of Purdue University and the University of New Mexico, as well as working for several years on the Government Relations staff of the Los Angeles Area Chamber of Commerce. (310) 568-5537; email fgehlen@pepperdine.edu Jack C. Green is Assistant Professor of Strategy at Pepperdine University’s Graziadio School of Business and Management. After 28 years of experience at Weyerhaeuser Company and Pacific Enterprises, he received his Ph.D. from Claremont Graduate University in Executive Management. He was a member of the Los Angeles County Quality and Productivity Commission for ten years and was its chairman for five years. Also, he served on the Los Angeles Countywide Criminal Justice Coordination Committee for two years. He is a consultant to nonprofit organizations and for-profit businesses. (626) 355-5553; email: jgreen@pepperdine.edu 217 mailto:ssnow@pepperdine.edu mailto:fgehlen@pepperdine.edu mailto:jgreen@pepperdine.edu Table of Contents Volume 29, 2001 Threshold Marketer A Family Of Marketing Simulations: Basic Marketer And Advanced Marketer Team Mode And Solo Mode Globalization As An Extended Experiential Exercise The Benefits And Planning Considerations Of Short Term Study Abroad Programs How 2 Setup Your Office Computer To Run Linux For Teaching E-Commerce Without Messing Everything Else Up Incorporating Cosmopolitan-Related Focus-Group Research Into Global Advertising Simulations Demonstration Of Advanced Features In Computer-Assisted Gaming Of International Business A Comparison Of Discrimination-Based Versus Conventional Simulation Game Scoring A Universal Mathematical Law Criterion For Algorithmic Validity The Impact Of Public Policy On Innovation: A Simulation Project For Research And Teaching Participant Identification Of Competitors In A Marketing Simulation Competition Simulation Research In The Hospitality Industry "Computer Simulation, Games And Roleplay: Drawing Lines Of Demarcation" Simulation Distribution Alternatives: Author/User Considerations Managing The Curiosity Gap Does Matter: What Do We Need To Do About It? Use Of External Interventions In A Computer Based Simulation Putting Service Learning Into Orbit It's A Wonderful Life: Simulating The Golden Years Adventures In Creating An Outdoor Leadership Challenge Course For An Emba Program Vbotz: A Pedagogical Cross-Disciplinary, Multi-Academic Level Manufacturing Corporate Simulation Use Of Computer Modeling In Management Accounting Is Simulation Performance Related To Application? An Exploratory Study Learning Cooperatively May Not Be Learning Collaborately! Perception Is Reality: Sharing Frames International Management Virtual Teamwork: A Simulation Financial Plan For Your Life And Career Goals Using Project-Based Experiential Learning Groups In The Principles Of Marketing Course Futures Course: Learning How To Anticipate The Future Of Business Interactive Online Strategic Market Planning With The Web-Based Boston Consulting Group (BCG) Matrix Graphics Package Strategy Learning In A Total Enterprise Simulation Investigation Of The Impact Of Decision Parameters For A Dutch Auction Simulation For Ipo Issues Integrating In-Class Learning With Out-Of-Classroom Experiences Through A Managerial Competency Development Framework War And Peace: Managing Students Learning Experience In A Competitive Simulation Game Virtually Experiential Classrooms Exercise: Conducting Role Plays Using Student Generated Cases Procedural Justice And Acceptance In Group Decision Making The E-Commerce Game: A Strategic Business Board Game Does Student Preparation Matter In A Simulation? A Comparison Of Pedagogical Styles The Game Of Business - A Weekend MBA Course Volume-Dependent Money Exchange Model For Gaming Simulations Implementing Service Learning For Accountants: The Not For Profit Project To Teach Vikings To Behave Among Mandarins: Lessons From Teaching With A Simulation Model Of Applied Business Ethics In International Management Maze Bright Teachers In The Classroom The Validity Investigation Of A Test Assessing Total Enterprise Simulation Learning What Makes Strategy Possible: An Illustration Using Paper And Scissors Learning Micro-OB Skills While Making Top Management Decisions In A Multinational Industrial Firm The Absel Research Heritage And The Bkl: Leveraging Their Value For Future Research New Product Development (Npd) Simulations: Some Challenging Questions And Tough Modeling Issues The Biofeedback Stress Test The Power Circle Exercise Total Enterprise Simulation Learning Compared To Traditional Learning In The Business Policy Course A Business Game Distance Education Application: Learning Outcomes And Experiences Is the Tobin's Q a good Indicator of a Company's Performance? A Critical Examination of the "Experiential" Premise Underlying BUsines Simulation Usage