Issues in Simulation Implementation: Lessons from a Freshman Seminar 42 Developments in Business Simulations and Experiential Learning, Volume 37, 2010 ISSUES IN SIMULATION IMPLEMENTATION: LESSONS FROM A FRESHMAN SEMINAR Granger Macy Ithaca College gmacy@ithaca.edu Aimee Dars Ellis Ithaca College aellis@ithaca.edu Don Lifton Ithaca College lifton@ithaca.edu ABSTRACT Business simulations are a widely used educational strategy. Although simulations are generally considered an effective tool in creating innovative and successful learning environments, not everyone is convinced. There are many risks in undertaking simulations which can impact on their success. There are risks for students and faculty alike. It is important to understand these hazards in order to effectively implement simulations in a classroom setting. This article reviews important implementation concerns, presents qualitative data from a new simulation implementation and develops important implications and considerations for effectively implementing simulations in the business classroom INTRODUCTION Business simulations are a widely used educational strategy. Simulations are thought to have many benefits. Ben-Zvi and Carton (2008) note that traditional college education is often poor at helping students integrate knowledge needed by modern business but suggest that business games can help to meet this need. Hemmasi and Graf (1991) surveyed students after graduation and found that they felt the simulations were more useful than current students did. Students after graduation felt that simulations helped with skills like problem solving and planning. This suggests that simulations do indeed provide an integration that is not accomplished by content oriented teaching methods. Herz and Merz (1998) also found that "the simulation/game seminar outperforms a conventional seminar with respect to all aspects of the (Kolb) learning cycle" (p.248). Although simulations are generally considered an effective tool in creating innovative and successful learning environments, others are not convinced. For example, Klabbers (1994), and Anderson and Lawton (1997)) have found little evidence to suggest that simulations improve learning. McLaughlin and Bryant (1987) also suggest that simulations have disadvantages, and conclude that simulations lack validity. There are also risks in undertaking simulations. Peach and Hornyak (2003) comment that it takes several semesters of experience to feel comfortable with using and explaining the simulation, so there is a tendency to continue with a familiar simulation than search for something better. This comment hints at the underlying perils facing faculty members when undertaking a simulation. Thus, while risk is a key aspect that drives student learning in simulation, it is also ever present for the faculty member as well. Such faculty concerns are rarely addressed in the literature. Especially when first starting a simulation, an instructor faces several hazards. The most basic and uncontrollable vulnerability is software stability. But others loom including the faculty member’s uncertainty about their own knowledge in understanding the simulation and the uncertainty in knowing how best to support students’ learning as they progress through the exercise. In short, simulations are not always successful. Simulations are a risky endeavor for students and faculty alike. When looked at in the light of day, truly succeeding in the use of a simulation may be quite the challenge. Indeed, simulations may be quite difficult to implement successfully. There are many issues that can impact on the successful use of simulations. In this article we will first review some of the key considerations from the literature that may impact on the success of a simulation exercise including student motivation, and implementation factors specific to 43 Developments in Business Simulations and Experiential Learning, Volume 37, 2010 simulations and problem based learning. We then examine a difficult implementation of a simulation in a first-year business course in the spirit of learning issues related to student acceptance and learning, and the faculty concerns inherent in the process. And finally we will attempt to understand the implications of these finding to improve the success of simulation based learning. STUDENT MOTIVATIONAL ISSUES IN SIMULATIONS An important factor in the success of simulation learning activities may be the students’ motivation to learn (Burns and Gentry, 1998). In essence, a simulation creates a gap between what a student knows and what a student needs to learn in order to succeed. This gap creates a tension that can be motivating. However, there is also a risk. As expectancy theory has shown if there is too much of a gap and the goal feels unattainable then the motivation to succeed can disappear (Pinder, 1998). Gentry, Dickinson, Burns, McGinnis and Park (2006) offered many insights on this topic which will be considered here. Gentry et al (2006) identified two underlying motivational goals that can impact on student performance; learning goals and performance goals. Performance goals might be considered as grade oriented behavior. On the other hand, learning goals are self–referential; i.e., more personal in nature driven by the students desire to improve. Students motivated by learning goals are interested in task mastery and in increasing their competence. (Dweck 1990). However, students motivated by performance goals are more concerned with being viewed as competent (Diener and Dweck 1978, 1980; Gentry et al., 2006). In other words, performance motivated students are grade seekers and not knowledge seekers. Students motivated by learning goals may be expected to learn more from simulations and other problem based experiences and feel more positively about the experience. Such students will tend to work to understand the simulation and be willing to figure out the problems that they encounter. However, students with a performance expectation may not try to succeed if uncertainty is encountered and will blame the simulation for not having the clarity they require to assure that their performance goals can be met. Relatedly, Warr, Allan and Birdi (1999) found that trainees who are confident in their ability to learn material are more likely to learn more in a training. Performance-oriented students may not exert effort if they receive negative feedback; i.e., poor results from the simulation. Such students may attribute this to the lack of their own ability and see the simulation as a threat to their self–esteem. In this case they may become defensive and blame the simulation or the instructor rather than themselves. However, learning oriented students approach the difficulty by trying to learn to improve since their actions are less influenced by their performance in the previous actions. They would be less disheartened or threatened by a lack of immediate success in the simulation. Students with a learning-orientation will instead track their performance and strive for improvement (Harackiewicz and Elliot 1993). Difficulties become challenges to master through effort. IMPLEMENTATION ISSUES IN SIMULATIONS Differences in student motivational goals suggest that students learn differently in a simulation experience. Gosen and Washbush, (1999) also suggest that student behavior and simulation implementation have significant impacts on student learning. Herz and Merz (1998) note that "game complexity and prior knowledge strongly influence the learning process of participants" (p. 249). Burns and Gentry (1998) in discussing learning expectations state that students can feel frustrated and experience high levels of anxiety (p. 146). This last comment focuses on the importance of uncertainty and in terms of expectancy theory about how students perceive their skills and knowledge. Students must feel that the goal is reachable or else they will lose motivation and not fully attempt to succeed and learn. Setting manageable expectations and controlling student anxieties may be key factors in successfully implementing a simulation. For most instructors, it takes several semesters of use to become comfortable at fielding questions from students (Peach and Hornyak, 2003). For such reasons, it is helpful to have a clear sense of what one wishes students to learn from the simulation so that manageable learning expectations can be established. Snow, Gehlen and Green (2002) have suggested that factors such as the how the simulation is introduced, how much class time is devoted to the simulation and how it is integrated into the class, the impact on students’ grades and can also affect the success of the implementation. Anderson and Lawton (2005) have examined the literature on problem based learning (PBL) in order to understand how simulations might be more effective. They noted that Duch, et al., (2001) assert that PBL problems should engage student interest and require them to develop and implement principal concepts of the course to solve the problem. Lohman (2002) also contends that it should be an ill-structured problem so that the nature of the problem is unclear and the information needed to solve the problem should be incomplete with multiple ways to solve the problem and no single right answer. In their review of the literature, Anderson and Lawton (2005) found that simulations can meet these requirements. Anderson and Lawton (2006) found in their study that performance in a simulation was related to student perceptions of their knowledge of the discipline but not to a favorable attitude about the simulation. Too much gap between the knowledge of the student and the knowledge required for success simulation can lead to significant dissatisfaction with the activity. All of this reinforces the centrality of risk to the process of any simulation activity. 44 Developments in Business Simulations and Experiential Learning, Volume 37, 2010 Anderson and Lawton (1997) suggest that the activities selected by the game administrator will influence the learning. In essence, implementation is a matter of risk management. Just how much stretch, how much uncertainty is effective and acceptable is difficult to estimate as it will depend on student motivational goals as well as on other demands placed on them in perhaps other concurrent course work or from a student’s extramural activities. Teach and Schwartz (2004) note that there are three measures of learning that instructors should consider when using a simulation;: what the student learns relative to professor expectations; what students learns relative to their expectations; and what the students actually learns compared to what the instructor measures. Effective Team Performance is also an important issue. Since most simulations in business programs are conducted in teams, it is reasonable to assume the team effectiveness could have a significant impact on the success of a simulation exercise. Effective teamwork may help moderate the uncertainty and performance expectations of the students as they undertake the simulation. Gully, Devine and Whitney (1995) in their metaanalyis on the relationship between team cohesion and performance noted the importance of this factor. Many factors such as interpersonal attraction and task cohesion (Zaccaro and McCoy 1988) are important. In essence, a cooperative learning environment should be established within the team to smooth the implementation of a simulation. THE STUDY This study examines a First Year course designed to provide an introduction to business and into the business degree that employed a simulation at a Northeastern United States comprehensive college. The business program is AACSB accredited and housed in a new building that was just opened the same year as that of the course offering. This study examines six sections of the course taught by two faculty members who are also authors of this article. The two teaching faculty will be called Professor A and Professor B in this study. The classes were conducted similarly but not in lockstep. Each faculty member made different choices in how to grade the simulation and in how to introduce teamwork into the simulation. The simulation is not referenced by name, but instead as “The Simulation”. “The Simulation” was a fairly simple but integrated simulation built around a small business activity. It incorporated financial, marketing, pricing, and operational decisions. Professor A was a new faculty member that year, and had just completed the PhD prior to arriving on campus. Grades were not assigned for performance in the simulation. Instead, grades were assigned based on a team reflection paper which was worth 15% of the overall grade. Professor A’s syllabus stated that the “simulation is an opportunity for you to work with a team to make real-world business decisions. …(the) grade will be determined by successful completion (i.e., submitting four decisions at predetermined times) as well as a peer evaluation. Each team will be responsible for creating a team identity, including a name, slogan, and logo”. Students were also asked to reflect on their team process in developing their identity. They were given class time to work on their identity. Professor B is highly experienced, having taught the course for numerous years in a single large class format. Professor B assigned grades competitively for the simulation. The simulation provided 10% of the course grade. Grades assigned were A+, A, A-, B+, B, B- or C based on the order of team performance. Professor B devoted two class sessions to teamwork. In the first class, students were introduced to Glenn Parker’s views on the different components of successful team work (Parker, 2003) Team members were asked to self-declare their individual strengths to determine if their team covered all of Parker’s components. Finally, near the end of the simulation rounds students were taught to apply the concepts of JoHari's window to their team members and themselves. Professor B also taught an ‘“at risk”’ section.. Students in this section had lower SAT scores than students in other sections of the course so that they could be given more support during their first semester at the college. As part of this study, each instructor wrote on reflection on their learning as a result of using the simulation, and shared syllabi, student evaluative comments about the course, and student reflections about the simulation experience. The lead author was not involved in the instruction of the course but did assist in the design of the simulation exercise and developed the initial analysis of the data. The study data will be presented in three sections. The first is student evaluative comments about the courses that are collected as part of the standard course evaluation practices of the school. The second is the student reflections that were required by Professor A. As mentioned previously, Professor A assigned no grade to performance in the simulation and instead graded the simulations based on a reflective paper turned in by each team in the simulation. The third and final data source consists of course reflections submitted by both instructors to the lead author. The lead author conducted the initial analysis of these reflections before submitting them for review to the instructors. STANDARD COURSE EVALUATIONS The student course evaluations were content analyzed with the results assembled into a table for analysis. It should be noted that Section 5 was the at risk class section mentioned previously. The number of comments about the challenging nature of the simulation would seem to be due to the students background rather than the course itself. It was clear that Professor B drew the most comments from students about needing more understanding. This may have been due to Professor B’s approach to teaching the 45 Developments in Business Simulations and Experiential Learning, Volume 37, 2010 course, or to the fact that the performance rank was a grading element in the course. Some of these comments follow: • An improvement would be either to leave out “The Simulation” project or to at least if the professor went over during each round to explain what to do, b/c it is very challenging • (Need) more time spent on understanding “The Simulation” (area to improve) • Could do a better job explaining process of our group project • I would also add more background to some of the elements of “The Simulation” (which was part of a comment to expand other projects as well) • Discuss “The Simulation” more in class. I feel many people dismiss it. • I don’t believe that “The Simulation” should be a part of the final grade seeing that we don’t go through many important aspects of it in class. After speaking to many students, they agree its more of a guessing game. • “The Simulation” should be taken away. Most of the time we are just making blind decisions and hoping for the best. It is a waste of time. STUDENT REFLECTIVE PAPERS As noted previously, Professor A assigned a reflective paper as part of the simulation exercise. In analyzing the content of student responses reflecting on course content relating to the learning process created by the simulation and its implementation the comments could be separated into four different areas: knowledge, grading, decision process, and teamwork. It should be remembered that Professor A did not assign grades based on performance in the simulation. KNOWLEDGE: Even though Professor A drew far fewer comments than Professor B about preparation for the simulation, this issue drew the heaviest comments in the reflective paper as well. • We do believe that a few things should have been covered in class that could have helped us succeed even more. Some of the terms that the simulation used were not always discussed in class or completely understood when taught, such as, net income, income statements, or what an industry benchmark report is. We think that if we spent some more class time going over what was expected of us for the next simulation, we could have done a lot better • Before diving into the “The Simulation”, I wished that I had known a lot more information, particularly pertaining to the various financial terms. I was unaware of what SCU meant during the first round decision, as well as what the market shareholder values meant. Because I did not understand these terms, I was unable to properly calculate how to make decisions. It would have been very beneficial if each term was defined before the simulation began, allowing the students to grasp a complete concept of the simulation. • Before the first simulation it would have been nice to have the program, its purpose, and procedures explained more clearly to us. We would have preferred a clear visual example in class of the expectations and an example of the assignments we were to be graded on each term. • Going into this simulation was difficult because we did not know what to expect. Would have been nice to know what all of the values meant in the beginning and how the data entered would affect everything. It was difficult because we had not covered a lot of information in class before starting the simulation. TABLE 1 CONTENT ANALYSIS OF STUDENT EVALUATIONS Professor A Professor B Section 1 Section 2 Section 3 Section 5 Section 6 Section 7 Class size 13 26 25 26 26 25 Drop Simulation 5 4 6 2 3 5 Other areas a problem rather than simulation 5 19 15 6 10 10 Areas commented on in simulation: Challenging 0 0 1 6 0 0 Needed more understanding/ explanation 0 0 1 5 4 7 Teamwork 1 (strength) 1 (weakness) 2 (strength) 1 (strength) 46 Developments in Business Simulations and Experiential Learning, Volume 37, 2010 • Before the simulation I wish I had known more in depth about what numbers meant. When I first launched “The Simulation” I did not know what the meaning of increasing one thing and the result of it was. Percents, shares, and bonds were even more complicated; it would have been good if we had some material on that. • I wish we would have had more background information on “The Simulation” before we started the simulation. I think going over it in class would have benefited our team more. • (by round 2) We did understand the terms a bit more and adjusted our input numbers slightly. What we learned in class was beneficial because it introduced us to the terms we encountered. Still, the simulation helped us understand the effects of the terms. WHAT TO DO: The lack of knowledge mentioned in the above comments then led to student uncertainty and frustration in attempting to make decisions • Coming into the first round of the simulation, all of the team members were clueless. We struggled to make efficient decisions, not actually understanding the point of the simulation. We attempted to decode the instructions of the project, but were not very productive. Before we completed our team decision, we attempted to test our ideas on one of the team member’s individual “The Simulation”. • As we began making decisions, we became more and more confused; all we were doing was putting in numbers without understanding the business aspect. We even tried to practice the program on our own time, but it wasn’t helping us out. Although every round was challenging for us, • Before starting the simulation, it would’ve been nice to have some kind of an introduction to the program by someone who better understood it. Throughout the simulation, we all were confused about what exactly to do even though we read the help guides. Submitting decisions was often a blind attempt. • Before we began the simulations, we wish we would have known to use the program and its purpose. It would also have been useful if we knew what had the greatest impacts on sales such as the significance of promotion and pricing. As a group, we believe that the business scenarios were too advanced for a basic business course. • After we read all the help screens, we had to come to some conclusions because otherwise we were going to sit there all night. After completing our first decisions, we exited the program having absolutely no idea on how the results were going to turn out. SUGGESTIONS FOR IMPROVEMENT: Students also offered several ideas on how instruction could be enhanced • The “The Simulation” would have been better if we had done an in-class practice day. During this practice day it would have been helpful if we all had our laptops and worked as a class. If we performed a practice round know what everything meant it would have been beneficial. It would have been helpful to know what each of the results were and how we could improve them versus just looking at results online. • Before we started the simulation, it would have been helpful: • Talking about the program in class because we had no idea how to make the decisions, • Seeing example decisions so we had a better idea of how to make the decisions, • Receiving more data to help us make the decisions, • Doing a practice decision in class with instruction about what to look for and what to consider when making the decisions. • If we could go back and do the simulation over, we would: • Read the reports more carefully to get a better idea on how to make the decisions • Not wait until the last minute to make the decisions, because it did not give us enough time to get the information we needed to make the correct decisions. • Practice with the program before the first round. • Before we started this simulation it would have been easier if our team played around with the program properly. Seeing where we needed to input our decisions, rather than franticly clicking on all the tabs, would have been easier on our parts. Another bit of information that would have been useful is to know the key terms that were used in this simulation • Before beginning “The Simulation”, I think the most helpful thing would have been to do a round together in class with a professor or someone very familiar with the program to facilitate and explain its logistics and let us know what and where we have to look in order to base our decisions effectively • I would liked to have more instruction on how to us the simulation better before we started - Maybe require that each person had to make at least one decision on their account in order to involve everyone • We believe there was sufficient information and material given before the simulation and no more was needed. If we were to do this experiment over 47 Developments in Business Simulations and Experiential Learning, Volume 37, 2010 we would gone over the information given more thoroughly. It would of made the decision making process easier more effective. We think that having this simulation graded would have provided the incentive and motivation to read the given information more thoroughly resulting in a more significant learning experience. • If we were to redo “The Simulation”, we would have changed the timeline of the decisions. Had the decisions been correlated with what was happening in the course, the simulation would have been more effective. By either condensing the simulation to making decisions only while we were discussing it in class, or making class relate to the decisions, “The Simulation” could have proved to be a very valuable tool. Several comments hint at shortcuts taken by the students in not availing themselves of the learning opportunities that were offered to them for practice. This suggests that student lack of responsibility may have contributed to their frustrations with poor understanding of the simulation. BACKGROUND: The numerous comments about challenge in Professor B’s at risk class suggested that inadequate student background can affect the usage of the simulation. Here we see the nature of such concerns in the reflections for Professor A’s classes. • In terms of setbacks affecting our team’s performance, we thought that one of the major problems was not having enough business background (especially on the first two decisions) to come to an educated conclusion on what is best for the company based on the statistics and probabilities given to us. Although we didn’t have any technology issues using the simulation program, we felt that the program wasn’t the best at explaining what made the profits or shareholder values go down after a certain decision. • For those students who did not take economics yet, it was unclear how to make smart decisions. For example, when there is a market shortage, then for the next decision it would be smart to lower prices. Students who weren’t already taking an economics class had no way of knowing what to do. It would have been a lot better if we were allowed to do a simulation as a class or with our group without it counting for a few times. That would have been a good practice. • We think that it is fair to say that we learned nothing from the simulation regarding business decisions. The level of complexity of the overall program, as well as each specific decision definitely hindered our ability to learn. We felt that the directions were uninformative, and when you combine that with lack of prior business knowledge, as well as inexperience, the situation is bound to present problems. COURSE GRADING: The comments contained many examples of both performance and learning motivations in the students. The following are some example of performance motivation found in the class. • If the grade was based on performance, we would have put more effort into the simulation. We had no real reason to care about how our company did, and because of this, we only put enough effort in the project to get the decisions done. If we had needed to perform well we would have read the reports more in depth to gain a better understanding of how to make the decisions and we would have learned more from the simulation. • If our grade were based on the simulation then we think everyone would have been more interested and engaged in the simulation Knowing that the simulation was not going to be graded on our performance, we believe that team members did not care so much for the outcome. If our grade was based on performance, team members would have been more interested and engaged in the simulation. Personally, we believe that each team member would have made wiser decisions and researched the outcomes of dividing funds. Also, if our grade was based on performance, there would have been a competitive edge which would have made students feel engaged and ready to combat their fellow classmates! • Having the grade be based on performance might have made us more interested and engaged in the simulation. We probably would have tried to make more knowledgeable decisions based on more factors than just previous rounds. Having the grade based on performance would not have been as fair because we did not know much going into the simulation. Also, if we had worked more as a group because we were more engaged, the person who knew the most about business would have done the entire project. There were many other comments that also showed that students felt they would have worked harder if grades depended on performance. Learning motivation examples were less common. Here is an example • However, we felt much better because we knew that we weren’t going to be graded on our performance and that rather the point of this whole simulation was for us to try to learn as much as possible and try to correct our mistakes from round to round. And the following reflections help suggest how students see the effect of grading on the simulation. 48 Developments in Business Simulations and Experiential Learning, Volume 37, 2010 • If our grades were determined by our performance on “The Simulation” I’m nearly 100% sure that our attentiveness to our decisions and our thought processes would have been increased. This is due to the fact that “The Simulation” would become more of a competition than a simple project as whichever group had higher stockholder value would get a better grade, and when it comes down to it competition usually draws out a desire to perform better. I know that if my grade counted on this one project most of my attention would have been drawn to it and not the subject matter of the course however • Honestly, if we were being graded by letter as a group for the simulation, I would have prepared myself more and been less dependent on other group members to know certain information. • I think that having my grade based on performance would have made me a little more engaged in the simulation and would have in turn caused me to pay more mind to the market research and forecasts. It probably would have given the group more incentive to be present at every meeting we had to make decisions in each round, and we would have taken it much more seriously. We wouldn’t have taken as many guesses in order to get it done quickly • Being graded based on performance would have greatly changed the situation, I believe that everyone in the class would have been much more involved and serious about making sure that their company did well • Having our grade based upon performance would have made us more interested and engaged in the simulation. This is because having the threat of getting a poor grade would cause us to work more diligently and research the topic more. • Grading students on performance would have definitely encouraged students to engage in the simulation more than this year where most of the time one person was doing the work. And others had different thoughts about the effects of grading. • I think that if a grade were waiting for us at the end of the simulation, we still would’ve treated it the same, but perhaps with a little more engagement. In the case of each decision, we didn’t know exactly what the consequences of our choices were in entering the numbers, thus causing us to lose interest. I don’t think a grade should be given for this simulation without more information prior to beginning it. • Having our grade based on simulation would have made us a lot more anxious and perhaps a little more motivated, but I do not think it would have changed much. I still feel that we, and most likely the other groups as well, tried our hardest and wanted to finish on top of the market whether there was a reward or not. DECISION MAKING PROCESS: It is also clear that, despite the problems, many thought they developed a better sense of decision making in business with comments that showed an understanding of issues around risk and integration. • I learned that when making decisions there is always a risk involved. With that being said, you have to be willing to take a chance, and if you fail then there’s always a way to fix it and to improve your numbers. Also, since the market is always changing, what is the right decision during one period may not necessarily be the same in following periods. You always have to be able to recover and readjust from your mistakes. • The main reason the simulation was so successful was due to the fact that this experience was something completely foreign to all of us. None of us had ever done something like this so the decisions being made were learning experiences for all of us. We learned something new each round and after it was all said and done learned that you must consider every decision that is previously made to make a new decision, and consider the results the decisions have on the future of the company. • Finally we learned consistency is essential in a well rounded, well run company because decisions made cannot be rash and must be well thought through. • Overall, we thought “The Simulation” was a good experience in learning about the many aspects of running a business and the decisions they have to make. If we had to do this simulation again we would have gathered more information on each decision. We also could have analyzed the results better to make future decisions • If I had to do the simulation over, I would try to pay more attention to detail and try to better learn what the effects were of each decision • “The Simulation” taught us that there are many aspects of business that we must consider when trying to make decisions regarding the company. There are many small details and they all matter. You must be knowledgeable in a variety of subjects because each decision you make is based on different numbers and different factors. Every part of the business must work with each other because each department within the company is connected and the decisions of one sector depend on the decisions of another sector. 49 Developments in Business Simulations and Experiential Learning, Volume 37, 2010 • However, “The Simulation” showed that there are so many factors in business decisions and you as the owner has to decide without really knowing what will happen. It’s a tough job and that’s why the people that are good at it are the ones that lead the business world. • . The things we learned about business decisions were: o Even the smallest decisions have a big effect on the entire business. Every decision we made had an effect on the financial health of our company. o Looking at past performance to make the decisions is very helpful. That is how we made most of our decisions, and our group did the best out of all the groups in the class. o Making a superior product, even if it is more expensive, can be an effective plan. That was our strategy, and it proved to be effective. TEAM PROCESSES: Team processes showed examples of both the problems and benefits of teamwork that would be expected in any class commenting on both the frustrations and values of teamwork. • We gained experience through working in a group. Making business decisions as a group can be very beneficial but at the same time it can be extremely frustrating and difficult. One benefit is that every group member brought his or her own knowledge to the group. With this combined knowledge, we were able to make the best decisions to our ability. We weren’t one sided on any decision because of the varying views of everyone in our group. • Similarly with our business decisions, as a group, we all learned that communication is the key to success. It is hard for one to truly understand that concept until they are put into a situation such as this. If we were to do it all over again we would have tried to meet at a more regular time and communicate better. We tended to rush meetings in at the last minute, rather than having a specific time to meet. • During our group meetings, each team member often had a different opinion when it came time to make a decision. Therefore, each team member was required to explain their reasoning behind their decision. As a team, we learned that listening to others and respecting their ideas was important. Every team member had the right to express their individual opinions without being criticized. Despite the differences in opinions, the team eventually came to a decision which was agreed upon by all team members. Respect was important in the group decisions! Some of the comments clearly showed that the work of the simulation allowed students to learn what we might call the many heads advantage. • We realized that if it was just one person doing this assignment, they would be completely lost, as opposed to 5 heads coming together and then approaching a great choice….We learned that group work should not just be thrown at one person. We all contributed to the decisions that we made • Others people’s ideas are sometimes better then my own and it is good to listen and learn from them. • What we did learn about group work was that although we may be presented with challenging tasks that we do not necessarily want to complete, we can succeed by working together and communicating on all levels. Others comments on some of the organizational issues inherent in team work. • I learned that it is a necessary to assign each member their assignments before starting the decision making process. By assigning each member with different tasks, we can save time as well as prevent any social loafing in the group. I learned that it is important to allow everyone to be heard in the group. Though there is only one person who enters the data, we all can contribute our different ideas on what needs to be done to have a successful result. • What we learned about group work is that no matter how cohesive the team may be outside the work environment when it is time to get down to business there is always a leader and there is always a group member who is the slacker. We as a team got a long very well. Some individuals stepped up and became leaders while others fell back into the shadows. There were many comments on group formation as well and showed some important aspects • It may have been more helpful if everything was talked about in class more and a small portion of class time was dedicated to “The Simulation” so groups could get together and ask questions if needed. If we had to do the simulation over, we would work more as a group. It was difficult to find time outside of class to get together as a whole group because of everyone’s different schedules. The scheduling conflicts could have been avoided if some class time was set-aside for groups to get together. • At the beginning of the semester, our group members were not so familiar with one another during the first “The Simulation” assignment let alone the first week of school. Therefore the lack 50 Developments in Business Simulations and Experiential Learning, Volume 37, 2010 of relationships we had with one another did not foster a very team oriented atmosphere. We believe that if more time would have been set aside during class to exchange contact info (i.e. phone numbers, email address) then there could have been a better understanding and comradely established earlier in the semester. Because we barley knew each other’s names or where we lived on campus, the only time to talk about the project was a few sentence exchanges after class. In that minimal amount of time, two of our group members decided to conglomerate on the project and get to work deciding a team logo, name and slogan. • We felt that there was no connection with the simulation. Although as a team we were able to develop a personal team identity, slogan, and icon, the simulation seemed to do everything for the team besides input values. The simulation always informed the team members when it came time to develop new products or contribute funds to publicity. Also, the simulation also gave the team members suggestions, allowing limited intuition and personal thought. Personally, we believed that the simulation limited the team members input. This made the simulation less interesting. DISCUSSION This discussion integrates the finding of the literature review with student reactions and faculty reflections. The complete faculty reflections can be found in Appendix A. FACULTY RISKS AND FACULTY OBJECTIVES: The notion of risk for faculty members is not often considered but upon review this must certainly be considered as an important issue. There are a number of factors which were found to impact on this and the notion of risk was certainly an issue in the reflections of both instructors. Professor A stated, “When I learned that a required simulation was part of the course, I felt some degree of trepidation. I’d never run a simulation before, and having to learn the game while prepping for the new course was nerve-wracking”. From another perspective Professor B noted, “Three years ago the Dean persuaded the faculty to call for an experiential learning component to be added to the course. I was hesitant, in part, because the course was already well-received by the students…. I ‘compromised’ and supported the computer simulation as an acceptable experiential learning alternative.” Not only was risk an issue but buy-in or commitment to the process is also seen here. Professor A made a similar comment, “I perceived “The Simulation” as a burden to be overcome rather than a useful learning experience for the students.” This is an important issue that warrants further research. Another critical issue from the literature review was the appropriateness of the simulation to the learning objectives of the course, and the appropriateness of the simulation to the desires and interests of the faculty. Professor A noted, “I ended the semester feeling very discouraged about ‘The Simulation’ and confident that it was an inappropriate simulation for the introductory level course…..the options for the various decision rounds had little connection to course topics (focusing more on financial issues than basic business decisions)... Because of the nature of the decisions, I felt I had to present topics in a counter-intuitive order to prepare the students.” Professor B observed, “As a social scientist, I was unfamiliar with the financial number-crunching emphasis of “ ‘The Simulation’ ” I did not wish to apply the time needed to be authoritative in the original semesters. Thus, my approach to the course simulation was, instead, to let time -- a series of semesters -- be my tutor. It proved a wrong idea.” These comments show the lack of fit of the simulation to the needs of the instructors and the course. Professor A goes on to note, “I attended ABSEL (the following semester) and realized that not only was ‘The Simulation’ a very poor match for the course, my presentation and/or integration of ‘The Simulation’ was completely off target as well. … had not formulated any learning objectives to communicate to the students.” STUDENT ANXIETY AND MOTIVATION TO LEARN: In introducing simulations instructors should be attentive to motivational issues. The issue of student anxiety was clearly found in the literature review and can have a significant impact on the student’s motivation to learn. Faculty should attempt to bring some clarity to the simulation for all students but especially for those students motivated by a performance expectation. Gentry et al. (2006) noted that research “suggests that the use of effortful and effective learning strategies is associated with goals that emphasize the importance of learning and mastery”. In the general class comments there were echoes of newness, uncertainty and also how one could learn from that. “I learned that when making decisions there is always a risk involved….. you have to be willing to take a chance….You always have to be able to recover and readjust from your mistakes.” Another reflection stated “…..this experience was something completely foreign to all of us. None of us had ever done something like this so the decisions being made were learning experiences for all of us. “ One of the key elements uncovered in the literature review was the importance of the types of student motivation to learn. This came across very clearly in both the student comments and the faculty reflections. This takes on several different aspects. At the most basic level, the instructor must consider how much competition with others in class will be a positive motivation. We run the risk though of heightening student anxiety to a place where it may become dysfunctional. This potential duality was 51 Developments in Business Simulations and Experiential Learning, Volume 37, 2010 certainly evident from the student responses when comparing across the classes of the two instructors. In our study Professor B’s class stressed the competitive aspect the most where 10% of the student’s grade was based on the competitive position of the team within the class. Professor B notes, “It may have informed student reaction. Teams that were doing well had excited, happy students. Others expressed frustration and disappointment. I also distributed a modified version of a “The Simulation” output at the end of each decision point to let each team see where they stood competitively.” In contrast Professor A reflected, “I decided I was uncomfortable with having the grade based on a competition, so I required a reflection paper from the teams. I also required a peer evaluation to minimize the effect of social loafing.” While students may have been less anxious about the simulation it also impacted on their motivation to become actively involved. Professor A continues, “While I had included practice for ‘The Simulation’ on the syllabus, the end-of-semester team reflection papers revealed that the students in fact did not practice and were unprepared when the decision rounds began. Since the student teams were not judged on their performance on the simulation, they gave it little effort, and they did not see the value in the results...even when we went over the financial performance indicators in class, they were too advanced for the students.” McLaughlin and Bryant (1987) found from student critiques that simulations offer benefits in feedback, and responsibility, integration of content and group decision- making. The learning needs to be ongoing throughout the simulation rather than a closure exercise. Professor A commented, “Furthermore, although I had the students do a final reflection paper, I realized that reflection papers were needed throughout the simulation so that the students could grapple with the meaning behind the results.” Overall, it seems highly important to help students reflect on their experiences in the simulation and to learn from their own reactions to the simulation. This requires additional research since it has great potential to improve student learning specific to the simulation as well as improve student life skills. The potential lessons from making students aware of their learning motivations are significant. They are too important to just be ignored in simulations and should become part of the learning inherent in a simulation – since the learning orientation will be important to future success in school and career. Research suggests that managers who learn from experience, and are open to other points of view and criticism can take a proactive stance toward problems and opportunities (Bigelow, 1998). TEAMS PROCESS, INTERPERSONAL ATTRACTION AND COHESION: A well developed student team committed to succeeding with the simulation can do much to support student learning from the simulation. The care and feeding of teams can be an important strategy in creating a successful simulation experience. Gentry et al. (2006) notes that one advantage of simulation is that students must live with their decisions throughout the game unlike other class activities or assignments. Poor decisions can leave teams in a very difficult situation. Poorly performing teams may need support to feel competent and not give up to easily on the learning that is needed to succeed in the simulation. With the right learning orientation this can be a positive learning experience but for those students with a performance motivation this situation can be debilitating. Unfortunately, student reflections often mentioned that student teams did not have enough chance to get to know each other and to become involved in discussions to help understand the simulation. Professor B noted, “It was enjoyable to see the delight among those students whose teams were doing well. The reactions of the other teams ranged from quietude to despondent chagrin bummed me. I tried each time to assure them that they still had chances to improve their position.” Professor B also comments that “with rare exception, I let the student teams "teach" themselves by doing.” This is one of the greatest potential benefits the coupling of a simulation with an effective team structure. We see clearly from the student comments and faculty reflections that it is important to develop an effective strategy. Some teams commented on how much they learned from the process while others seemed disconnected from their teams and struggled to work with the differing personalities in their team. Team based learning is the most common approach to implementing a simulation. This area deserves much greater attention and research in order to improve student learning and make simulation activities more effective. LESSONS LEARNED Overall, our faculty was quite despondent about the outcomes of the simulation experiment. Professor B noted that “negative student feedback prompted our colleagues on the School Retention Committee to call for a different simulation… Undoubtedly, my disengagement contributed to this outcome….But the whole drill was basically distasteful to me. I dreaded dealing with it but coped. The students were usually able to distinguish their feelings about the rest of the course and me from the “The Simulation” component.” In returning to a different simulation in the following year, Professor A found observed, “I've given the teams much more time in class, and it has been more structured…. I've also given them time to work on their decisions. ….they also had to come up with a team name and logo, as well as do a more detailed strategy paper. This year, the students must complete a quiz (provided by the simulation company) to demonstrate they’ve participated in practice rounds. Each team must complete a strategy paper prior to beginning the simulation, and there are additional reflection papers assigned throughout the term.” It is clear that many of these issues can help to increase motivation and knowledge. 52 Developments in Business Simulations and Experiential Learning, Volume 37, 2010 Ultimately they may also help control student anxiety as they proceed through the simulation. Professor B commented, “Since a simulation still is part of WoB, I'm gearing up a bit more this time out. I'm devoting, for example, a whole class to orienting them to the simulation before they submit their first of 5 decisions. My attitude is more upbeat about this simulation but I'm fragile.” The fit of the simulation to the course is critical and resulted in a changing the simulation as well as many of the implementation activities surrounding it. Professor A in discussing the revised efforts in the following year notes, “We have already spent one day in class going over the case on which the simulation is based and making sure everyone is properly registered.” In addition the new simulation better fits the areas of emphasis in the course. Professor A goes on to say, “I have reminded them several times that many of them stated they want to be entrepreneurs or executives, and that the simulation will help them get experience running their own company. I have retained the peer evaluation.” In addition, Professor A now incorporates some point values for relative team performance “to ignite their competitive spirit and give them incentive to perform well.” Overall, we believe that this study helps to identify just how complex and difficult the effective implementation of a simulation can be in a classroom. There are many risks, but also many benefits for the students as well. The difficulties of implementation deserve much more study to help assure that students gain the greatest possible learning from their participation in a simulation. The potential benefits of a simulation experience are not automatically obtained by students. It is critical that the implementation of a simulation be carefully thought out. Hopefully, this paper provides some fruitful directions for improving the successful implementation of simulation activities. BIBLIOGRAPHY Anderson, P. and Lawton, L.. (1997). demonstrating the learning effectiveness of simulations: where we are and where we need to go. Developments in Business Simulation and Experiential Learning, 24, 68-73. Anderson, P. H., Lawton, L. (2005). The effectiveness of a simulation exercise for integrating problem-based learning in management education. Developments in Business Simulation and Experiential Learning, 32,10- 18 Anderson, P. H., Lawton, L. (2006). The relationship between students’ success on a simulation exercise and their perception of its effectiveness as a pbl problem. Developments in Business Simulation and Experiential Learning, 33, 41-47 Ben-Zvi, T., Carton, T. C. (2008) Applying bloom’s revised taxonomy in business games. Developments in Business Simulation and Experiential Learning, 35, 265-272 Bigelow, J. (2004), Using problem-based learning to develop skills in solving unstructured problems. Journal of Management Education, October 2004, 28, pp. 591- 609. Burns, A. C., & Gentry, J. W. (1998) Motivating students to engage in experiential learning: A tension-to-learn theory. Simulation and Gaming, 29, 133 - 151. Diener, C. I. and Dweck, C.S. (1978), An analysis of learned helplessness: continuous changes in performance, strategy, and achievement cognitions following failure, Journal of Personality and Social Psychology, 36, 451-462. Diener, C. I. and Dweck, C.S. (1980), An analysis of learned helplessness: ii. the processing of success, Journal of Personality and Social Psychology, 39, 940- 952. Duch, B., Gron, S. and Allen, D., editors (2001) “The Power of Problem-Based Learning, A Practical ‘How To’ For Teaching Undergraduate Courses in Any Discipline”, Stylus Publishing, LLC, Dweck, C. S. (1990), Self-theories and goals: their role in motivation, personality, and development, in Perspectives on Motivation: Nebraska Symposium on Motivation, Richard Dienstbier (Ed.), Lincoln, NE: University of Nebraska Press, 38, 199-235. Gentry, J.W., Dickinson, J. R. Burns, A.C., McGinnis, L. and Park, J. (2006). The role of learning versus performance orientations when reacting to negative outcome. Simulation Games Developments in Business Simulation and Experiential Learning, 33, 79-84 Gosen, J. and Washbush, J. (1999) As teachers and researchers, where do we go from here? Simulation and Gaming, 30, 292 - 303. Gully, S.M., Devine, D. J., & Whitney, D. J. (1995) A meta- analysis of cohesion and performance: Effects of level of analysis and task interdependence. Small Group Research, 26, 497 - 520. Harackiewicz, J. M. and Elliot, A.J. (1993), Achievement goals and intrinsic motivation, Journal of Personality and Social Psychology, 65 (5), 904-915. Hemmasi, M. and Graf, L. (1991). Educational effectiveness of business simulation gaming: a comparative study of student and practitioner perspectives. Developments in Business Simulation and Experiential Learning, Vol. 18:53-56. Herz, B. & Merz, W. (1998) Experiential learning and the effectiveness of economic simulation games. Simulation & Gaming, 29, 238-250. Klabbers, J. H. G. (1994). The 25th Anniversary of ISAGA: The Orchestration of Organized Complexity. Simulation and Gaming, 25:502-513. Kolb, D. (1984) Experiential learning: Experience as the source of learning and development. Englewood Cliffs, NJ: Prentice-Hall Ledman, R. E. (2005). Student expectations of simulations. Developments in Business Simulations and Experiential Learning, 32, 183-185 53 Developments in Business Simulations and Experiential Learning, Volume 37, 2010 Lohman, M. C. (2002) Cultivating Problem-Solving Skills through Problem-Based Approaches to Professional Development. Human Resource Development Quarterly, 13, 3: 243-261. McLaughlin, F. and Bryant, G. (1987). A Comparison of Student Perceptions With Accepted Expectations for Business Simulations. Developments in Business Simulation and Experiential Learning, Vol. 14:135-140. Parker, G. M. (2003) Parker team player survey, Tuxedo Park, NY: Xicom, Incorporated. Peach, E. B. and Hornyak, M. (2003). What are simulations for?: learning objectives as a simulation selection device. Developments in Business Simulation and Experiential Learning, 30, 220-224 Pinder, C. B. (1998). Work motivation in organizational behavior, Upper Saddle River, NJ: Prentice-Hall Snow, S. ,Gehlen, F., and Green, J., (2002). Does student preparation matter in a simulation? A comparison of pedagogical styles. Developments in Business Simulation and Experiential Learning, 29, 208-217 Teach, R.D. and Schwartz, R. G (2004). Are business games really delivering what students are led to believe?? Developments in Business Simulation and Experiential Learning, 31, 264-272 Warr, P, Allan, C., and Birdi, K. (1999). Predicting three levels of training outcomes. Journal of Occupational and Organizational Psychology, 72, 351-375. Zaccaro, S. J., and McCoy, M. C. (1988) The effects of task and interpersonal cohesiveness on performance of a disjunctive group task. Journal of Applied social Psychology, 18, 837-851. APPENDIX A FACULTY REFLECTIONS This appendix presents the full text of the reflections prepared by the two faculty members. These reflections include their experiences in preparing for the simulation activity, observations about the conduct of the class, and finally their final assessment of the experience as well as plans for the future. PROFESSOR A’s REFLECTION: When I learned that a required simulation was part of the course, I felt some degree of trepidation. I’d never run a simulation before, and having to learn the game while prepping for the new course was nerve-wracking. It was also not the type of requirement I enjoyed when I was in school, and I was skeptical about the appropriateness of a simulation in a freshmen-level course. Conversations with the two faculty members who had taught World of Business the previous year (2007-2008) revealed that they had very different approaches to the simulation. I decided I was uncomfortable with having the grade based on a competition, so I required a reflection paper from the teams. I also required a peer evaluation to minimize the effect of social loafing. From faculty members as well as from students who had taken WOB in previous years, I heard that students hated the simulation. I perceived “The Simulation” as a burden to be overcome rather than a useful learning experience for the students. When I started exploring the simulation myself, my negative feelings were quite solidified, though I tried to hide them from the students. The interface was plain, the options for the various decision rounds had little connection to course topics (focusing more on financial issues than basic business decisions), and the program seemed outdated. Because of the nature of the decisions, I felt I had to present topics in a counter-intuitive order to prepare the students. While I had included “practice “The Simulation”” on the syllabus, the end-of-semester team reflection papers revealed that the students in fact did not practice and were unprepared when the decision rounds began. Since the student teams were not judged on their performance on the simulation, they gave it little effort, and they did not see the value in the results...even when we went over the financial performance indicators in class, they were too advanced for the students. I ended the semester feeling very discouraged about “The Simulation” and confident that it was an inappropriate simulation for the introductory level course. The simulation, while having excellent customer service, is more appropriate for an accounting course, and really needs to have a more exciting interface for students. In March, I attended ABSEL and realized that not only was “The Simulation” a very poor match for the course, my presentation and/or integration of “The Simulation” was completely off target as well. The simulation was included in World of Business due to a decree from administration, and I had not formulated any learning objectives to communicate to the students. Furthermore, although I had the students do a final reflection paper, I realized that reflection papers were needed throughout the simulation so that the students could grapple with the meaning behind the results. Most importantly, I felt inspired to find a simulation that was at a more appropriate level and more connected to the topics of the course. I had the teams complete the attached Team Identity Assignment. They were given time in class to work on that. I assigned it way too early in the semester, and the submissions were very superficial. (The following comments refer to changes made in the subsequent year based upon learning from the course.) This semester, I've given the teams much more time in class, and it has been more structured. They had to do one activity about group norms and one scavenger hunt activity. I've also given them time to work on their decisions. This year (the year following the class examined here), they also had to come up with a team name and logo, as well as do a more detailed strategy paper. This year, the students must complete a quiz (provided by the simulation company) to demonstrate they’ve participated in practice rounds. Each team must complete a strategy paper prior to beginning the 54 Developments in Business Simulations and Experiential Learning, Volume 37, 2010 simulation, and there are additional reflection papers assigned throughout the term. We have already spent one day in class going over the case on which the simulation is based and making sure everyone is properly registered. The current simulation has decisions in the areas of Management, Marketing, Operations, and Accounting, and provides a more user-friendly, fun, and intuitive interface. I have reminded them several times that many of them stated they want to be entrepreneurs or executives, and that the simulation will help them get experience running their own company. I have retained the peer evaluation. A portion, though not all, of their grade will be based on how the student teams perform relative to other groups in their class to ignite their competitive spirit and give them incentive to perform well. As we start the relevant subject matters, I plan to use the current simulation as an example in class. PROFESSOR B’s REFLECTION: I am seen, informally, over the past many years as the lead instructor in the course. A few other colleagues have also taught sections of it. Three years ago the Dean persuaded the faculty to call for an experiential learning component to be added to the course. I was hesitant, in part, because the course was already well-received by the students. My biggest objection was the Dean's preferred addition: assign each student to a community agency to offer public service. I spent a lot of political capital to successfully oppose the risk of putting 180 17 year olds in busy community settings. I had none left to oppose the general thrust of an experiential learning insert so I "compromised" and supported the computer simulation as an acceptable experiential learning alternative. “The Simulation” was selected. This research focuses on the second time it was included in our course. Each time, 5-member teams were assembled randomly in each section. They competed against each other through 5 decision points to achieve the best shareholder value. With rare exception, I let the student teams "teach" themselves by doing. On occasion, I distributed an orientation sheet (attached). I also distributed a modified version of a “The Simulation” output at the end of each decision point (example attached) to let each team see where they stood competitively. 10% of their grade was at stake.. It may have informed student reaction. Teams that were doing well had excited, happy students. Others expressed frustration and disappointment. As a social scientist, I was unfamiliar with the financial number-crunching emphasis of “The Simulation”. I did not wish to apply the time needed to be authoritative in the original semesters. Thus, my approach to the course simulation was, instead, to let time -- a series of semesters -- be my tutor. It proved a wrong idea. Negative student feedback prompted our colleagues on the School Retention Committee to call for a different simulation. We have switched this semester. Undoubtedly, my disengagement contributed to this outcome. Time and the excellent “The Simulation” staff nurturing support would have resolved this but, alas, time was not in the offering. I never got the hang of it (simulation) because I was content to let it come to me over many semesters thru "osmosis." Thus, I was often speaking to their support team to interpret the results I was distributing so I could offer "advice" to the teams on what to improve for their next decision. Fortunately, the inexperience of their first semester of college study left them unprepared to challenge my "expertise" with probing questions. It was enjoyable to see the delight among those students whose teams were doing well. The reactions of the other teams ranged from quietude to despondent chagrin bummed me. I tried each time to assure them that they still had chances to improve their position. Once the end was nearing and their "destiny" was assured, I then reminded them that I had discretion in the range of grade to assign them within each competitive final result slot (see the former attachment on the grading scheme). This helped them a bit. But the whole drill was basically distasteful to me. I dreaded dealing with it but coped. The students were usually able to distinguish their feelings about the rest of the course and me from the “The Simulation” component. Since a simulation still is part of WoB, I'm gearing up a bit more this time out. I'm devoting, for example, a whole class to orienting them to the simulation b4 they submit their first of 5 decisions. My attitude is more upbeat about this simulation but I'm fragile. Table of Contents Volume 37, 2010 A Simulation in Organizational Behavior: Development and Beta Test Simulating Networks: An Experiment Pick Your Group Size: A Better Procedure to Resolve the Free-Rider Problem in a Business Simulation The Impact Of Playing A Marketing Simulation Game On Perceived Decision Making Ability Among Introductory Marketing Students If the Games Work, Why Aren't More Faculty Willing to Play? Modeling Cascading Demand: Accounting for the Effects of Captive Consumer Relationships Issues In Simulation Implementation: Lessons From a Freshman Seminar ENDURANCE: A game for current economic times Simulating Processes: An Application In Supply Chain Management Another Look At The Use Of Forecasting Accuracy On The Assessment Of Management Performance In Business Simulation Games The Effect Of Advertising On Demand In Business Simulations The Relationship Between Learning And Performance In A Total Enterprise Simulation: Revisited And New Data. Is the Gold /Pray Simulation Demand Model Valid and is It Really Robust? Mission Possible - Using Simulation Games For Management Training In A Transition Economy Business Game Modeling For The Negotiation Between General Contractor And Subcontractors The Relationship Between Goal Orientation And Simulation Performance With Attitude Change And Perceived Learning: A Follow-Up Study The Invalidity Of "Natural Market Structure" Pims Validation Of Marketing Games Examining A Beta Test An Experiment On Group Decision-Making Using A Business Game: An International Comparison Of Mba Students In Japan; China; And Russia A Platform for Business Games - From Game Playing to Game Making: The Case of Yokohama National University Checking Financial Balance of Target Brand Portfolio with the Strategic Market Plan Cash Flow Package About Simulations and Bloom's Learning Taxonomy Managing Complexity: Applying the Conscious-Competence Model to Experiential Learning Using Accumulated Profits to Assess Performance in Simulations And Now Let's Really Innovate: The Notion of Perpetual Online Courses Distance Learning: A Game Application Web Based Total Enterprise Simulation Learning Business Administration Using Simulation Software A Review of Experiential Exercises in Absel's Published Proceedings Moving Outside the Classroom: An Experiential Exercise Involving Video Clips Peer To Peer: A New Tool for Student Peer Evaluation Continuing To Assess Project Management: What's Been Done? What's It Means? Looking At New Twists Strategies for Promoting Knowledge Formation and Dissemination in Business Simulation and Experiential Learning Introducing Experiential Exercises To Inexperienced Faculty Members Conveying Concepts Through Cartoons: Preliminary Results From Software Engineers In An Industrial Organization Theory Class Utilizing Institutional Problems as an Experiential Learning Opportunity for Students Considerations in the Design of Vicarious Learning Environments Enhancing Critical Thinking Through Service-Learning As A Consultative Process Encouraging "Established" Faculty to Adopt a "New" Games and Simulations Pedagogy IMA Candy Company: An Experiential Exercise in Managerial Accounting Mustard Seeds as Means for Creative Problem Solving, Ethical Decision Making, Stakeholder Alliance, & Leader Development through Experiential Learning in Management Education Interactive Classroom Technology: Adding a Benefit to the Traditional Classroom Excelling in the Capstone Simulation: An Application of Spreadsheet Technology as a Decision Support Tool GEO: An Entrepreneurship-Oriented Computer-Assisted International Strategy Simulation ERP Simulation Game: A Distribution Game to Teach the Value of Integrated Systems