Use of a Simulation in a Large Class Environment for a Marketing Principles Class: A Qualitative Analysis of Whether Learning Objectives were Met Page 293 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 ABSTRACT The purpose of this paper is to examine the use of a simula- tion in a large class environment for a Marketing Princi- ples class. Specifically the goal is to analyze if the simula- tion meets the learning objective of the course, based on student groups’ reflective papers. The results of this study indicate that the Simulation in this class supports the appli- cation level of thinking from Bloom’s Taxonomy and is adding to the learning outcomes and objectives for the class. INTRODUCTION This paper provides the narrative of a story common in ABSEL‟s history, that of a first-time simulation game user in a Principles of Marketing class. The “new” here is provided by the assessment of learning in the class, as a systematic analysis of the students‟ reflection papers pro- vided insight as to whether the game experience accom- plished its mission. In the Principles of Marketing course, we are continu- ally faced with the growing class size and diminishing out- side teaching assistance. Since the mid 2000s class sizes have grown from 45 students to 200 students with less and less teaching assistance being provided. The course objec- tives have evolved as the class size has continued to grow. Today, the objective for the class is to be able to apply the concepts and definitions of marketing. Under Bloom‟s taxonomy, the basic learning objective is application of the terms and concepts introduced in the course. In January 2009, we were faced with a situation where the marketing department was going to teaching 2 classes averaging 150 students each and one of the two research assistants was reassigned, approximately three weeks before class was to start. In that case, some imme- diate changes had to be made to the class and one of the changes implemented was the use of a simulation. At that time, there was no opportunity to test an intui- tive sense that the simulation would support a key learning objective of helping students learn to apply and use the terms and concepts in marketing and other situations. We decided to test the intuitive sense in a formal way by doing a research initiative focused on exploratory research into the learning outcomes for the marketing simulation. CLASS OVERVIEW The Principles of Marketing class is a required class for all business majors, as well as for advertising students, some hospitality students, professional golf students, and some construction management and engineering students. The class also has majors from music, agriculture, and sev- eral liberal arts fields. The class is an overview of the field of marketing with an expectation that the students will become familiar with the basic concepts and terms of marketing and be able to use the concepts of marketing in their chosen field. It has been taught in a large class format, with lecture and recita- tions in the fall and lecture only format in spring. Class size is approximately 200 students for day time classes and 100 students for night time classes that only meet once a week. The goals and objectives for this class have evolved over time as the class size has increased. The learning objectives today are for the students to get to application level of the concepts and terms in marketing. There is also have an unstated objective of the students understanding how they can use marketing in their fields of study and convincing some students that marketing is a good choice for a major. The Marketing department is moving in the direction of this class becoming a lecture only class without recita- tion sections. For purposes of the research, we focused on a large lecture section with 100 students that met once a week in the evening. The class had a typical cross section of majors. USE OF A SIMULATION IN A LARGE CLASS ENVIRONMENT FOR A MARKETING PRINCIPLES CLASS: A QUALITATIVE ANALYSIS OF WHETHER LEARNING OBJECTIVES WERE MET Rob Simon University of Nebraska-Lincoln Rsimon2@unl.edu James W. Gentry University of Nebraska-Lincoln Jgentry1@unl.edu mailto:Rsimon2@unl.edu mailto:Jgentry1@unl.edu Page 294 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 Assessment in the class was made up of four compo- nents: Grading Tests (100 points each) 400 points 63% Group Marketing Plan 100 points 16% Simulation 75 points 12% Chapter Quizzes 60 points 9% Total 635 points The tests are multiple choice with a strong emphasis on application oriented questions. The Group marketing plan is a project in which the group makes up a product or service and develops a plan to market the product or ser- vice. The learning goal for this project is to be able to ap- ply the concepts and tools introduced in the course to a plan for a new and different product or service. As in the case of the simulation game, this experience is designed to have the successful students demonstrate their ability to apply or demonstrate their understanding and use of the terms and concepts covered in the course. As noted earlier, the learn- ing goal is to get the students to the application level think- ing in Bloom‟s Taxonomy. The quizzes are open book online quizzes with an ob- jective to get the students to understand the material in the book before it is covered in class. This assessment would fall under the knowledge or remembering level of Bloom‟s Taxonomy or the revised taxonomy. BLOOM’S TAXONOMY Bloom‟s Taxonomy has been used as a learning frame- work in ABSEL for over 30 years. Our cursory search found that the Gentry, McCain, and Burns (1979) paper was the first to discuss the framework at ABSEL, and that more than 100 ABSEL papers since that time have refer- enced the taxonomy. The taxomy is a six-level hierarchy to classify thinking from the basic level of knowledge to the most abstract level of evaluation. The classifications build on each other, as you cannot have application thinking without knowledge and comprehension. For an instructor the key is to look at the level of thinking we want to achieve through an assignment (Forehand, 2005). Listed below is below is a brief review of the Bloom‟s Taxonomy, taken from Daniel Royse‟s book, Teaching Tips for College and University Instructors: A Practical Guide. Bloom‟s Cognitive Taxonomy (Arranged from con- crete to abstract and simple to complex)  Knowledge: (Remembering factual materials)  Students must remember, memorize, recog- nize and recall.  Comprehension: (Grasping the meaning of materi- als)  Students must interpret, describe and explain knowledge.  Application: (Problem Solving)  Students must apply facts, rules, and princi- ples to produce some results.  Analysis: (An understanding of the structure and components of knowledge)  Students must be able to break down knowl- edge and show relationships among the parts.  Synthesis: (Creating a unique, original product; combining ideas to form a new whole)  Students must bring together parts and com- ponents of knowledge to form a whole and build relationships for new situations.  Evaluation: (Making value decisions about issues and resolving controversies)  Students must make judgments about the value of materials for given purposes (Royse, 2001, pp. 42-43) OBJECTIVES OF THE STUDY The objective of this study is to understand what level of learning under Bloom‟s Taxonomy the simulation as- sessed and is it consistent with the learning objectives of the class. SIMULATION OVERVIEW The simulation used is Mars Marketing Management Simulation by Robert Cook, James Cook, and Kathryn Cook. This simulation is typically for marketing manage- ment courses, and is much more challenging than simula- tions for most introductory marketing courses. The basic overview of the simulation is that the students have to make decisions involving up to four video game products over several business periods. The groups compete with three other groups over 6 periods and the “winner” has the highest cumulative profit over the six periods. The decision matrix for the students includes product variables such as the number of products, the attributes of the products, the product dimensions or market segments targeted, the violence level of the product and the cost of the products. The decision matrix includes price variable such as wholesale and retail cost and gross margin. The matrix also includes promotion or Integrated Marketing variables such as expenditures for advertising, sales promo- tion and publicity. The matrix also includes distribution or place variables such as where to sell the products and sales force allocation. The decisions impact brand awareness, target marketing and perceived product positioning, which need to be factored into the decisions. Finally the matrix has a seasonality factor in it with each decision covering a quarter of the year with some quarter such as the 2nd and 4th quarter having higher sales than the other two quarters. Page 295 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 The decision matrix tries to duplicate the decisions marketing managers have to make in a real world environ- ment. The students have the opportunity to buy additional research reports to help them make decisions. The stu- dents are told that there are no right or wrong answers, and the results will be based on decisions made by their firms and their competitors. The students are dealing with a moving target and they have to make decisions every week with incomplete or partial data. I also provide the students with a competitive analysis worksheet (Appendix B: Simu- lation Competitive Analysis Worksheet) to help analyze what their competitors are doing. The simulation was a group project with each team having 5 students in it. The groups were self-selected. I then assigned the teams to a league of 4 teams which they competed against. The class had 5 leagues of 4 teams each and each team only competed against the teams in its league. The simulation was assessed based on cumulative profits. Each segment had 4 teams competing with the team with the highest cumulative profit receiving 60 points (100 %); the second highest team earned 57 points (95%); the third place team earned 53 points (89%) and the last place team earned 50 points (84%). I agreed with the game‟s authors that it did not make sense to grade the low- est team below a B, because some of the factors are beyond the team‟s control, just like in business and there is a steep learning curve to the simulation (Cook, Cook, & Cook, 2006). Further, the percentage of the class grade allocated to the simulation experience was relatively low compared to the levels used typically according to past discussions at ABSEL, but I was being cautious as this was my first ex- perience with a simulation game. Teams also had the op- tion of keeping decision logs for each set of decisions (Appendix B: Simulation Decision Logs). The logs were reviewed and points were awarded to teams that may have started off poorly but made good strategic decisions, but were never able to catch up to other teams. Teams usually got two additional points for well thought out decisions that were logged in. This practice will be evaluated more closely in future game plays. One possible issue for future research is that the grading on the simulation should put more emphasis on finishing first and the spread of the grades should be wider. Also, after the game play, I found out about former ABSEL president Dick Teach‟s strategy of having the last place teams chip in money for end-of-the -game liquid refreshments, and I see the opportunity to vary performance rewards (punishments) in order to inves- tigate their influence on learning. The teams earned up to 15 points based on a simula- tion reflection paper that was done after the simulation was finished. This paper was intended as a means of the team discussing what they took away from the simulation. It was a method to assess the learning that went on in the simulations and also an evaluation of the use of the simula- tion in the class. The standard simulation experience evaluation has been based on post-play surveys, and it is our premise that the systematic analysis of the reflection papers will provide greater insight into the nature of the learning taking place. METHODOLOGY As discussed above each team completed a simulation reflection paper. The instructions were as follows: The final assignment for the simulation is for the group to write a brief reflective paper on what you learned from the simulation and how it integrates with the material in the course. This paper is a group assessment of the con- cepts and skills you learned from the simulation. Start the reflective paper with the place your team came in on the simulation competition. Think about the effort your group made in the simula- tion, including understanding the instructions and docu- mentation. You should include what data and tools you used to make your decisions, what was the overall strategy for the simulation and what you would have changed after reflecting on that. Looking back what could your group have done to have improved its decision making for the simulation? What were the most difficult parts of the simulation for your group, and what were the easiest parts? What were the most interesting and most surprising results from the simulation? The most important part of your group reflection is what did you learn from the simulation? Did the simula- tion help you understand marketing and marketing strat- egy? Did the simulation add to your understanding of the material in the course? Did it integrate well with the con- cepts and material in the course? Be specific in what you learned and how it integrated into the course. This is a reflective or group assessment paper. I don’t expect or want outside research. I am interested in the group’s consensus on the learning and understanding that your group took away from the simulation. Be honest in your assessment. I want you to clearly explain what you as a group took away from the assessment. The reflections were turned into the teaching assistant. The students were told that the teaching assistant would grade them and the instructor would not see them until after final grades were posted. This procedure was followed for analyzing the papers. The teaching assistant graded them on the basis of whether the team addressed what they learned. He also graded them on effort, grammar, spelling, etc. The reflective papers were the data for an exploratory study of what the students learned from the simulation. The papers were initially organized by place. The first place teams‟ papers together were analyzed together, the second place teams‟ papers together, etc. The papers were coded, using a qualitative approach to coding and classify- ing the data into themes (Creswell, 2007). The analysis focused on frequency of general themes (i.e. giving more Page 296 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 weight to themes that were found in multiple papers). The themes were then analyzed as indications of learning and thinking that related to Bloom‟s Taxonomy (Forehand, 2005). The initial analysis included comparing the results from the first place teams‟ to the reflections from the teams in the other places on the competition. The initial assump- tion that team placement was important was rejected and all the themes were categorized all the themes together. No consideration was given for placement, other than for some learning outcomes from the last place teams that differed from the other teams. Several of the last place teams gave up fairly quickly and did not put the effort in that teams in the other three places did. Two out of the five last place teams attributed their results to chance or considered the simulation a guessing game. Further research to confirm this is needed, but for the purpose of this analysis, place of finish on the competition is not considered as a relevant factor in terms of what kind of learning and thinking out- comes the students took from the simulation. Looking at the themes, they were classified them into four categories, Marketing Strategy and Tactics, Business Strategy and Tactics, Operations and Learning Outcomes. A majority of themes fell under the category of mar- keting strategy and tactics. It indicates that the successful groups were using the concepts in the class to develop strategies and solutions for the simulation. The categories and themes are listed below. MARKETING STRATEGY AND TACTICS  Use research and analytics to make decisions. As one group noted “fly by instruments,” letting the data di- rect your decision-making. This includes purchasing and using all available research reports.  Use lifestyle segmentation and niche marketing strat- egy. The products must line up with the needs and wants of the target market. The successful groups real- ized it made sense to narrow the number of products offered.  Develop an integrated marketing communication plan with balanced efforts in advertising, publicity and sales promotion.  Balance the benefits of advertising, sales promotions, publicity and product development against the costs of these elements.  Understand the pricing issues involved, especially the relationship between wholesale and retail pricing.  Focus on data on consumer behavior and consumer perceptions.  Balance all elements of the marketing mix (Price, Pro- motion, Product and Place). Place or distribution plays an important role and you need to keep your re- tailers satisfied.  Be first to market. BUSINESS STRATEGY AND TACTICS  Focus on the Competition and be flexible.  Strategy is like a chess match. Opening moves are key to success. OPERATIONAL ISSUES  Develop group cohesiveness and focus on group dy- namics  Read, study and understand the manual for the simula- tion. To be successful need to continually reference the manual.  Buy all the research reports and understand what they tell you. LEARNING OUTCOMES  Reinforced and helped students retain concepts from the course. Correlated directly to class material such as segmentation, target marketing, marketing mix and the four P‟s of marketing (product, place, price and promotion). The simulation made the course material easier to understand and more entertaining.  Focus on how to implement the concepts from the course.  Illustrate the complexity of marketing decisions and the interplay of multiple variables when making deci- sions. This leads to the groups making decisions with incomplete information.  Realistically reflected how interdependent all decisions are that impact the marketing mix.  Shows how marketing strategy fits together.  Not beneficial, did not learn anything from the simula- tion (this was from a last place team).  Do just as well guessing, not a good use of time (these comments were from two last place teams) ASSESSMENT OF THE RESULTS Initially the interest in the study was in determining if the simulation was a worthwhile exercise and successfully encouraged students to reach the application level in Bloom‟s Taxonomy. This section will discuss what was learned from the analysis of the reflection papers and what the reflections suggest about the level of learning that goes with the simulation. To begin this discussion we will briefly discuss the simulation author‟s thoughts on the learning and level of thinking that the simulation generates for students. We will also review the elements of the Revised Bloom‟s Taxon- omy that are relevant to this analysis and what the analysis of the reflection papers indicates for the learning objec- tives. Robert Cook, the primary developer of the simulation, notes several primary learning benefits of the simulation Page 297 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 method in the Instructors Manual for the simulation. Many of these are directly related to the results from the student reflection papers. Cook‟s learning benefits are listed be- low:  Development of group management skills.  Exposure to simulated real business type environment.  Dynamic environment that changes with each decision by all the competing groups.  Interaction of multiple variables simultaneously.  Active learning of the concepts in the class.  Competitive environment that is typical of the business environment.  Fun and engaging learning experience.  Development of problem-solving, decision-making, and analytical skills. (Cook, Cook, & Cook, 2006) The important learning contents/processes noted in the students reflective statements are the group management skills; dynamic environment; interaction of multiple vari- ables; competitive environment; fun and engaging learning experience; and development of problem-solving, decision- making and analytical skills. The simulation met many of the learning benefits the author‟s proposed for the simula- tion. Most importantly for this analysis, it appears that the simulation facilitated the development of problem-solving, decision-making and analytical skills for students. Problem -solving relates with the application level of thinking in Bloom‟s Taxonomy. Does the data support the idea that the simulation indi- cates the learning objective of the Application level of thinking from Bloom‟s Taxonomy for the Introduction to Marketing Class? Themes from the reflective statements will be analyzed to understand if the simulation experience leads to an application level of thought for the marketing concepts. Point #1 is using research and analytics to make deci- sions. This includes purchasing and using all available research reports. The first theme is a tool to help with problem solving and decision-making. As one group noted, “we take a very scientific approach to „flying the company by instruments‟ with the marketing report being our instruments to guide our decisions… this was very helpful in tracking trends.” The students are definitely ap- plying facts and data to problem solving, which indicates application thinking. Point #2 is using lifestyle segmentation and niche mar- keting strategy. The products must line up with the needs and wants of the target market. The successful groups real- ized it made sense to narrow the number of products of- fered, which is what in marketing is called a niche market- ing strategy. This is a concept covered over and over again in the lecture and many of the groups used this concept in their strategy and noted it in the reflection statement. The groups applied concepts developed in the class to the tool- box they used in making decisions in the simulation. In this case again an application level of thinking is evident. Point #3 is developing an integrated marketing com- munication plan with balanced efforts in advertising, pub- licity and sales promotion. These were concepts stressed in the classroom lectures, but are more subtle to understand than other concepts in the class. The successful groups picked up the importance of this element. Most students think advertising is the key to communication, but to be successful in the simulation the groups needed to focus on all the elements of the communication mix. Most of the groups came to realize they needed more than advertising to be competitive. Here the students used principles that came from the lectures, indicating an application level of thinking. Point #4 relates to the balance between the benefits of advertising, sales promotions, publicity and product devel- opment and the cost of these elements. The better groups realized they needed to do a cost-benefit analysis prior to each decision period. As one group stated, “a large part of our success came from our two conflicting strategies. As a strong push for increasing advertising , increasing sales and modifying the products came from one side, while a con- stant pressure to cut costs came from the other side. To- gether a balance was found, resulting in high profits.” This is not something that came directly from class, but is more an understanding of what is happening in a competitive marketplace. This theme indicates there is higher level thinking taking place at the analysis level of the taxonomy. Point #5 is understanding the pricing issues involved, especially the relationship between wholesale and retail pricing. This is something that is taught in the class, but can be difficult for students to understand. It is really about understanding how a business makes money. Many of the groups came to the realization that this is a key factor in the simulation. This also indicates application thinking. This ties very closely with point #6 below about understanding how all elements of the marketing mix work together. One of the most successful groups commented on how they used pricing as a competitive element in the simula- tion., “By applying a little game theory and trying to an- ticipate what our completion was going to do, we moved ahead of them. The main application for guessing what our competition was doing came in the form of prices. Our wholesale price changed every decision period. The goal was to keep the retailers away from our competitors‟ prod- ucts, and give them a higher incentive to push our products. This lowered our sales dollars, but consequently increased our sales. Retailers seem to be more receptive to lower prices. The reports showed this through a one in the re- tailer satisfaction survey. We moved the prices in abnor- mal numbers using a price of 13, rather than a 10, 15, or 20. In anticipation of competitors copying out prices, we would lower them in advance.”. Point #6 is the need to balance all elements of the mar- keting mix (Price, Promotion, Product and Place). This is something that is covered continuously in the class both in Page 298 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 lectures and the texts, but it is a difficult concept for many of the students to understand. It is especially difficult for the students to understand place or distribution as an impor- tant part of the marketing mix. One group said, “Our focus up until the last period was on advertising to final consum- ers…, but we should have taken into account the impor- tance of getting our product into stores. Without the ability to place our products in more retailers, we fell behind in profit, even though we felt the ratings and advertising of our video games were at good levels... We have come to realize that building relationships with other businesses is just as important as creating positive relationships with customers.” The simulation helped the students realize the importance of a marketing mix that works together. This is another example of applying the principles in real world type situation, which also indicates an application level of thinking. Point #7 is focusing on data on consumer behavior and consumer perceptions. This is an ability to understand and use perceptual maps, based on how the product is perceived by the target segment (Cook et al., 2006 p. 15). It also in- cludes being able to understand the results of market re- search reports and using them in decision-making. While this is discussed in class, the students have to learn on their own what the maps and research reports are indicating. In this case the students need take principles discussed in class and the book and apply them in a particular situation. Again this indicated application level of thinking. Point #8 being the first to market is a principle that has become a statement of faith in business today. It is empha- sized in the popular and business press and is somewhat over used. There is a case for it to be important in some instances, but is not always a good strategy. We do not focus on it in class, so I am assuming the students are learn- ing this from the popular press and other classes. Many of the groups mentioned it in their reflection papers, so it is no doubt a concept from outside the class they applied, so a case can be made that it does indicate application thinking. The other category that is relevant for this study are the Learning Outcomes. It is evident that most of the themes support the concept of application as a learning objective. A number of groups commented on how the simulation reinforced the concepts from the class and related directly to many of the basic concepts of the course. The simula- tion also focused on implementation of the course concepts. One group stated that “The marketing simulation was a very beneficial learning tool. All of us agree that it was excellent to have hands on experience and we will retain the information due to the fact that we actually used it in real life (sic). Because we were able to implement the things we were learning in class, it allowed us to better understand the concepts being taught.” This illustrates that the simulation helped the students to understand better the concepts of the course by applying them in this particular situation. The simulation made applying the concepts of the course a practical experience and achieved a level of application thinking for those who made a successful effort on the simulation. It was rewarding to see from the feedback that the simulation made the course material easier to understand. One group said, “We thought that overall the simulation made the course material easier to understand… The mix of the class material and the simulation seemed fairly effec- tive to us as far as understanding what goes where and what happens when you mess around with this or that such as price or product changes. Overall it helped our understand- ing of the course and was a nice way to pull different con- cepts together.” This indicates that the simulation helps facilitate understanding of the concepts through applica- tion. The simulation also showed the complexity of market- ing decisions and the interplay of multiple variables. This complexity also reflects the interdependency of the market- ing mix elements. These are all characteristic of business decision-making and are an important learning outcome for the students. These outcomes are application oriented, but also indicate that some analysis thinking taking place as well. This was not planned or expected, but it is another factor that indicates that the use of the challenging simula- tion was successful in the class. Finally, some groups who finished last believed that the simulation was all due to chance and they did not get anything out of the simulation. This sentiment should not be ignored and indicates better training and monitoring is needed for groups doing poorly. In this case it was only 2 out of 20 groups who felt this way, but it does suggest more needs to be done to engage these students. FUTURE RESEARCH For future research, adding a section in the reflective paper on how the simulation can be made more realistic would be interesting and add to the use of the simulation in future classes. Another consideration is to expand the re- search using Bloom‟s Taxonomy, developing a measure of objective learning that can capture different levels in the taxonomy. We realize that this last statement is much easier to make than actually do, given the general lack of success in this endeavor witnessed by past efforts at ABSEL. For example, the 1996 ABSEL had a special session on the measurement of learning (Gentry et al., 1996), which un- fortunately did not generate great progress in terms of how we measure learning. It would be interesting to do a quantitative study using the total population of the class to measure the level of cog- nitive skills that the students are learning from the simula- tion. We would continue to use Bloom‟s Taxonomy as the model, but include a survey, as well as a pre and post-test analysis. It would also be interesting to research overall success in the class with success in the simulation. Another avenue would be to compare student group results based on GPAs and/or majors. Page 299 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 As suggested earlier, the assessment grading should be analyzed. One method may be to look at the results as we make the grade a much larger portion of the students over- all grade. What is the impact on changes in incentives on the learning outcomes for the simulation? More specifi- cally, we think it would be interesting to investigate the influence on learning if the reward structure places undue incentive to win versus a system which encourages actions aimed at the avoidance of losing. CONCLUSIONS We have shown that the simulation supports the key learning objective of the class to get the students to under- stand how to apply the ideas and concepts introduced in the course in a business setting. The simulation helped the students reach the application level of thinking about mar- keting concepts and the simulation ties in well with other assignments used in the course. There are some interesting takeaways from this study that maybe implemented in future classes. There is a need to integrate the simulation more closely with the course material, especially early on in the class, so the students can see how the simulation works with the course material. Unfortunately it does not become evident to most of the students how the course material integrates with the simu- lation until they have been through several decision periods with the simulation, which can be almost two thirds of the way through the class. More interest in the marketing material can be gener- ated, by tying the lectures more closely to what is happen- ing in the simulation. Even though these are large classes, a debriefing session early on in each class can stimulate the students to start thinking more holistically about how the decisions they are making on the simulation relate to the course material in the book or the lecture. There is a need to do a better job of training the stu- dents for the simulation. Currently it has been: read the manual and go, with a brief overview from the instructors. From the comments on the reflections, there are some areas needing to be emphasized and tested on in the overview and training. The successful teams all commented on the amount and time and effort they put in studying the student manual and that needs to be reinforced in the classes. The students will get more out of the course and the simulations will be more competitive. More emphasis needs to be put on the competitive na- ture of the simulation to generate more interest. League leaders should be announced in each class leading up to the end of the simulation. Some internal reward system should be created that is separate from grades, such as gift certifi- cates or recognition to encourage the competition. Busi- ness students learn better if they are in competition and the class and use of the simulation should encourage that. Overall this study was insightful for the instructor, and hopefully for the reader as well. The use of reflective pa- pers is instructive for the instructor and the students, and it should be incorporated into introductory marketing classes when using a simulation game. REFERENCES Cook, R., Cook, J. C., & Cook, K. J. (2006). Mars Market- ing Management Simulation Instructor's Manual. Re- trieved May 24, 2010, from Mars Marketing Manage- ment Simulation: http://www.shootformars.com/ s e r v l e t / c o m . m a r s . D o w n l o a d S e r v l e t ? s i m - type=mms&type=prof&name=mmsInstructorManual.p df Creswell, J. W. (2007). Qualitative Inquiry and Research Design:Choosing Among Five Approaches, 2nd Edi- tion. Thousand Oaks, CA: Sage Publications, Inc. Forehand, M. (2005). Bloom's Taxonomy:Original and Revised. Retrieved May 25, 2010, from M. Orey, (Ed.), Emerging Perspectives on learning, teaching and technology.: http://projects.coe.uga.edu/epltt/ index.php?title=Bloom%27s_Taxonomy Gentry, J.W., P. Anderson, A.C. Burns, A. J. Faria, R. Fra- zer, J. Gosenpud, L. Lawton, R. Nulsen, R.Teach, and W. Wellington (1996),"How Do We Measure the 'Learning' in Experiential Learning and How Do We Best Stimulate It?" Developments in Business Simula- tion and Experiential Learning, Vol. 23, p. 197. Gentry, J.W., G. McCain, and A.C. Burns (1979), "Relating Teaching Methods with Education Objec- tives in the Business Curriculum," Developments in Business Simulation and Experiential Learning, Vol. 6, pp. 196-198. Royse, D. (2001). Teaching Tips for College and Univer- sity Instructors: A Practical Guide. Needham Heights, MA: Allyn and Bacon. http://www.shootformars.com/servlet/com.mars.DownloadServlet?simtype=mms&type=prof&name=mmsInstructorManual.pdf http://www.shootformars.com/servlet/com.mars.DownloadServlet?simtype=mms&type=prof&name=mmsInstructorManual.pdf http://www.shootformars.com/servlet/com.mars.DownloadServlet?simtype=mms&type=prof&name=mmsInstructorManual.pdf http://www.shootformars.com/servlet/com.mars.DownloadServlet?simtype=mms&type=prof&name=mmsInstructorManual.pdf http://www.shootformars.com/servlet/com.mars.DownloadServlet?simtype=mms&type=prof&name=mmsInstructorManual.pdf http://projects.coe.uga.edu/epltt/index.php?title=Bloom%27s_Taxonomy http://projects.coe.uga.edu/epltt/index.php?title=Bloom%27s_Taxonomy Page 300 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 APPENDIX A Competitive Analysis Worksheet Benchmarking Analysis for Marketing Simulation One way to compare marketing activities and decisions is to benchmark it to the competition. The following are a few analyses you might want to use when analyzing the completion and your results. You might want to do the first table for overall and by product. The second table is by product. Cumulative Profits Communication Mix ($ Invested in each element) Communication mix (% of total in each element) Cumulative market share leader Your team Difference Cumulative Profit leader Your team Difference Gross Mar- gin for Product ______ Brand Awareness for Product ___ Communication Mix for product ___________ Positioning scores in comparison to the ideal points Retailer Satisfac- tion for Product _______ Cumulative mar- ket share leader Your team Difference Cumulative Profit leader Your team Difference Page 301 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 APPENDIX B Simulation Decision Log SIMULATION GAME - DECISION LOG TEAM # _______ DECISION PERIOD: ________ What major issues and problems were discussed during your meeting(s) for this period‟s decision? For each major change in your decision variables, provide a brief justification and rationale to support the planned change. Group members present at the meeting(s) for this decision period. (Please sign your names) Page 302 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 Page 303 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 Page 304 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 Table of Contents Volume 38, 2011 Simulated Tabletop Exercise for Risk Management - Anti Bio-terrorism Scenario Simulated Tabletop Exercise From Business Games to Simulations - Simuworlds & Microworlds Demand Equation Redux: The Design and Functionality of the Gold/Pray Model in Computerized Business Simulations Managing Client-Based Learning: Insights from Successful Teaching Project Courses in Marketing Tracking Forecast Error Type, Frequency and Magnitude with the Forecast Error Package Responding to Facilitate Collaboration Simulating Sudden Change and the Value of Timely Information Managing Human Resources Simulation A Study on Collectivism and Group Decision-Making: An International Comparison of Japan, China, and Russia Using a Gaming Simulation The Use of Management Games in the Management Research Agenda Gaming On-Line: A Simulation Application Positioning and Performance in Simulated Networks Supply Chain Management: A Simulation Application Simulation as a Teaching Method in Strategic Management Distance Studies Entrepreneurship: A Game of Risk and Reward Phase II: The Start-Up Return to the Paradise Islands: From Confrontation to Cooperation Effect on Market Performance of Displaying Supply and Demand Curves in a Business Simulation Appreciating Complexity: The Chief of Staff of the Army Game Managing Organizations: Experiential MBA Course Teaches Alternatives to the Machine Model A Simulation Model for Analyzing the Night-Time Emergency Health Care System in Japan The Continuing Evoluation of Assessing Project Management as an Academic Learning Outcome (ALO) Should College Instructors Change Their Teaching Styles to Meet the Millenial Student? The Mouse Game and its Effects on Team Interdependence Learning from the Gulf Oil Spill to Prepare for a Brighter Future: A New Game Engaging Stake Holders in Triple Bottom Line Accounting & Strategic Planning Video Killed the Biblio Star: The Impact of Digital Media on Student Learning Outcomes Exploring Motivation: Using Emoticons to Map Student Motivation in a Business Game Exercise An Alternative to PC and Internet Based Simulations: The Internet Integrated Mode MiddleState University -- A Crisis in Education Complexity Avoidance, Narcissism and Experiential Learning Examining the Cognitive, Affective, and Psychomotor Dimensions in Management Skill Development Through Experiential Learning: Developing a Framework A Situational Leadership Exercise Based on the Biology of a Starfish JOGAI CEFET -- The Industrial Administration Undergraduate Game Would You Take a Marketing Man to a Quick Service Restaurant? Modeling Corporate Social Responsibility In A Food Service Menu-Management Simulation Use of a Simulation in a Large Class Environment for a Marketing Principles Class: A Qualitative Analysis of Whether Learning Objectives were Met A Team Based Information Literacy Exercise ABSEL Marketing Communications Plan An Interdisciplinary Study of the Impact of Playing a Marketing Simulation Game on Student Knowledge of Management Accounting/Finance Principles Analyzing Construction Planning of Interiro Finish Work of Apartment Building by Simulation Doing Murder One Again The Simple Business Game and Simulation Transfering the Knowledge of Middle Management to Novices Infectious Disease Simulation Model for Estimation of Spreading Understanding the Relative Influence of Several Factors in ERP Simulation Performance: An Exploration of Ecological Validity Tragedy of the Commons: An Exercise Using Clickers to Illustrate and Teach a Key Concept in Negotiations The Meaning of Firm Demand in Business Simulations If the Games Work, Why Aren't More Faculty Willing to Play? Those Who Do and Those That Don't: A Study of Engaged and Disengaged Business Game Players