GILDING THE LILY, ENHANCING THE PEDAGOGICAL EXPERIENCE, OR FIXING WHAT ISN’T BROKE: THE DILEMMA OF NEW SIMULATION VERSIONS Developments in Business Simulation and Experiential Learning, Volume 28, 2001 GILDING THE LILY, ENHANCING THE PEDAGOGICAL EXPERIENCE, OR FIXING WHAT ISN’T BROKE: THE DILEMMA OF NEW SIMULATION VERSIONS E. Brian Peach, The University of West Florida Richard G. Platt, The University of West Florida ABSTRACT The paper discusses the advantages and disadvantages of releasing new versions of a simulation with added features and capabilities. The question is raised whether the goal set of the simulation designers and the goal set of the users are entirely aligned. Some goals of the designers, such as added income, may come into conflict with the goals of the users, such as realistic scenarios and user friendliness. Irrelevant and/or unnecessary features may help marketing and sales of the simulation, but detract from rather than enhance learning. The article uses the Business Strategy Game as a discussion example. INTRODUCTION An increasing number of studies attest to the widespread use of Total Enterprise Simulations in both the corporate world and business schools. Faria (1989) estimated that more than 5,000 U.S. companies use business simulations in corporate training and development programs. Use of simulations is also widespread in Europe and other parts of the world (Joldersma & Guerts, 1998). The rise of corporate universities and the increasing sophistication of simulations available for PCs makes them increasingly attractive for use in the corporate environment. In addition to the corporate world, simulations have become endemic to business schools. In surveys of AACSB business schools, Faria estimated 95% used simulations in the mid-1980s (Faria, 1987), and ten years later he increased the estimate from 95% to 97% (Faria & Nulsen, 1996). The use of simulations does not come without cost. Rollier (1992) estimated it cost his corporation $14,000 per executive per course in executive wages, and emphasized that simulations must bring value to justify adding to this cost. In academia, the cost is less quantifiable in dollars, but simulations certainly add to the course complexity and workload for instructors and students. DESIRED LEARNING OUTCOMES To justify the cost, simulations must be perceived as adding to the desired learning and/or behavioral outcomes for the course. Although the educational benefits of simulations have been the subject of conflicting research (Biggs, 1990), the debate has centered more around the superiority of simulations relative to other pedagogical approaches (Greenlaw & Wyman, 1973; Keys, 1976; Wolfe, 1985; Keys & Wolfe, 1990), than whether learning occurs in simulations. Identification of specific desired learning outcomes by an instructor is complicated by the wide variety of possible outcomes. Dickinson & Faria (1994) cite previous research that has addressed many potential learning outcomes. Types of learning thought to be improved through simulations include: basic economic concepts (Edwards, 1987), financial analysis skills (Faria & Nulsen, 1976; Hall, 1987), goal setting and information processing (Philippatos & Moscato, 1969; Biggs & Greenlaw, 1976), organizational behavior and personal interaction (Cangelosi & Dill, 1965; Chisholm, 1979), performance on mathematical problems (Faria & Whitely, 1990; Whitely & Faria, 1989), and sales forecasting (Edwards, 1987; Hall, 1987). Teach & Govahi (1993) surveyed 602 graduates from 36 business schools and prioritized 41 managerial skills according to usefulness to students. They found simulations were best in 1) helping set and evaluate objectives, 2) solving problems systematically, 3) making decisions, 4) forecasting, 5) adapting to new tasks, and 6) managing time. Gosenpud & Washbush (1994) surveyed simulation users to assess what they believed should be learned from simulations. Their nine-point list used in the survey included: (1) Marketing mix management, (2) Production and inventory management, (3) Cash management, (4) Raising and investing capital, (5) Financial statement and cost analysis, (6) Strategic decision making, (7) Strategic management theory, (8) Group process effectiveness, and (9) Communication skills. EVOLUTION AND MARKETING OF SIMULATIONS Simulation designers face a number of objectives. Among these are designing a simulation that models relationships in the environment, is relatively easy to use, is interesting to the user, and packaged in such a way that it can capture sufficient market share to justify continued development. It may be that over time, however, as a simulation progresses through successive versions, some of these objectives may come into conflict. Laundry detergents have long practiced the “new improved” tactic 196 when there is rarely, if ever, any significant change in the formulation. On the other hand, simulations occasionally seem compelled to make actual, substantive changes as new versions are released. Often these changes are necessary to keep up with the underlying technology of the simulation (e.g., the move from DOS to Windows, to NT to Linux), or to correct major flaws. Often changes may be necessary to keep the simulation relevant to the environment it portrays. In other cases, the changes are intended to make the simulation more user friendly. Unfortunately, in some cases it appears the changes are to add features that add to complexity without adding to value or learning derived from the simulation. We believe designers should refer to lists of desired learning outcomes when adding new features to a simulation. Unless a feature is directly supportive of a learning outcome, it should not be added regardless of its gee-whiz attributes. If a feature is supportive of a desired learning outcome, designers should then ask whether the amount of increase in learning is justified by the increased complexity and workload demand on the student and instructor. Any ABSEL follower of simulations is aware of the gradual increase in complexity of simulations as computers become more powerful. Designers can add features without worrying the additional tasking will bog down the likely user’s computer resources. In the following sections, we discuss how some of these added features in one a widely used simulation seem to be counterproductive. THE BUSINESS STRATEGY GAME One of the most widely used simulations is the Business Strategy Game (BSG) by Thompson & Stappenbeck (1999), has undergone a number of revisions and it is now in Version six. The BSG is a Total Enterprise Simulation that incorporates the major variables identified by Keys & Biggs (1990) for such simulations, such as marketing, production and finance variables. For a more complete description of the game, see Snyder (1995). Versions 1-3 were DOS based; Versions 2 and 3 essentially fixed problems that were identified in previous versions. No fundamental change to the game’s underlying assumptions were made. Version 4 moved the BSG to a seemingly Windows environment. It actually remained a DOS program, but clever programming made the screens appear to be Windows based. There were also a number of claimed improvements, including adding a Celebrity Endorsements Feature to the marketing algorithm. At least one undocumented change in the underlying structure occurred when the movement of product between the Private Label and Branded markets was reversed. A new Five Year Strategic Plan Model was also introduced. In addition to various undescribed “fine-tuning” adjustments in Versions 5 and 6, Version 5 introduced a Demand Forecasting Feature and an Inventory Liquidation option, and Version 6 revised the Executive Compensation Feature. THE DOWNSIDE OF UNNECESSARY ENHANCEMENTS There are a number of drawbacks, or downsides, to unnecessary simulation features and enhancements. Any one of the following four major drawbacks should give pause to simulation designers considering adding features to their simulation. As a group, the need for caution is even more compelling. New Features Adding new features to a simulation clearly adds to the overall complexity of the simulation, increasing the total learning time and effort for users. More importantly, they detract from the time users have to put into learning and mastering the concepts that are truly relevant. Increased effort for decreased learning makes it harder for the instructor to establish the relevancy and value of the simulation to all users, and in many cases will be all it takes to alienate a portion of the users. One of the more important variable sets in a Total Enterprise Simulation is the marketing function. It is imperative that students can see the link between their manipulation of the marketing variables and resultant sales. Even students who are relatively weak at predicting results should be able to assess the results post hoc. In versions 1- 3, marketing/promotion consisted of current and cumulative advertising. Version 4 added a marketing variable called Celebrity Endorsements and combined Endorsements and Cumulative Advertising together into a component called Image. Celebrity Endorsements are clearly a major factor in the real environment of athletic shoes such as Nike and Reebok and adding Celebrity Endorsements may or may not have been popular with some users. Unfortunately, it is not clear how it added to the value of the simulation. How does adding celebrity endorsements help the student understand the true learning objective - the relationship between marketing effort and sales? In fact, there are a couple of drawbacks to adding such a feature. First, the BSG product is athletic shoes, but adding endorsements as a factor makes the simulation more industry specific. Scott & Strickland (1985) advocated using a completely generic product. Although others believe a specific product enriches a simulation, Keys & Biggs (1990) point out that the more specific a product, the more likely students are to mimic the real industry rather than the simulated industry. Second, the Celebrity Endorsements feature adds complexity to the marketing sales link making it more difficult for students to develop and assess appropriate strategies. It also appears to add complexity to the underlying marketing algorithm. The BSG provides a wealth of reports that students use to identify the relative value of various marketing inputs. In after the fact analysis, before Celebrity Endorsements, it was relatively easy to understand the reasons why some teams 197 experienced more success than others based on marketing effort. The linkage between marketing variables and sales was clear. The only difficulty was predicting future moves by competitors. The addition of Celebrity Endorsements has not only made such analysis more difficult but often provides conflicting evidence on linkages from one decision to another. Thus, the addition of Celebrity Endorsements seems counterproductive as it increases student frustration, lessens learning, and tends to focus students on the real shoe industry. Decreased Learning Cannon & Burns (1999) pointed out that player success may represent other constructs than learning. If motivating players leads to simulation performance, “. . . teachers who use simulations as educational tools will naturally structure their teaching to promote game performance.” (Cannon & Burns, 1999, p41) Thus, teachers may be tempted to focus on including or emphasizing in their courses material that is not a primary learning objective, but is important to simulation performance. This detracts from the time the instructor can spend on core concepts and can negatively impact student learning. Eliminating bugs Simulations are somewhat complex programming efforts. There will always be bugs in new versions. Subsequent releases should focus on eliminating the bugs and fine tuning the various algorithms as users gain experience. In our experience, there is significant effort spent on fixing bugs that are found in the first few months of use. As time passes, however, the programming effort seemingly shifts to the ‘next’ version. At some point, it is no longer possible to get known bugs fixed – the response is “that will be fixed in the next version.” Unfortunately, this can have effects on users ranging from significant to devastating. If students so much as even suspect that their poor performance might possibly be due to defects in the simulation rather than in their decision performance, they externalize blame for their shortcomings. A computational or similar error anywhere in the simulation puts the entire process into question. For example, the BSG added the Five Year Strategic Plan as a separate component in the decision making process. Thus, the BSG now has two decision components - the current decision and a five year projection. These are independent, and entries in one component do not affect the other. Unfortunately, students do not readily accept this. Introduced in Version 4, the Five-year Strategic Model had a number of significant bugs in its computational processes. Although many have been fixed, some minor computational errors have persisted from Versions 4 through 6. When students encounter and identify these errors (admittedly, most happen infrequently) it has created severe tension for the affected players. Assurances that the errors do not affect the actual play of the game (e.g., the current decision) ring somewhat hollow. More to the point, because these errors do affect the accuracy of the five-year projection, they greatly diminish its usefulness as a planning tool. As stated before, all new programs have bugs. The point here is that the effort that was put into designing new and possibly unnecessary features should have been put first into correcting all known bugs. Computational errors, and even misspelled words, are distracting to students and detract from their confidence in the simulation. Model accuracy The last drawback to be discussed here could be designated as “most important” except that all of the mentioned drawbacks are very important. However, the relevance of the simulation, or in other words its ability to accurately and consistently model the environment it purports to model, is fundamentally important to any simulation. At present, there is no quantitative basis for contending that the demand algorithm used in the BSG Version 6 has been adversely affected by the addition of new components, such as celebrities to the marketing mix. But at the anecdotal level, fourteen years of using this simulation leads to the conclusion that the results of well developed strategies no longer consistently lead to predictable results. The fundamental premise of our strategy course is that to succeed, firms must be able to position themselves in their environment with strategies tailored to their resources and the environment they face. In a good simulation, winning strategies are based on students being able to accurately predict the actions of their competitors, and devise a production/marketing/financial plan that effectively deals with such actions. Developing such strategies requires looking at past results to (1) assess the relative influence of various demand drivers and (2) identify competitor strategies and likely actions. Version Four introduced a Competitor Report that highlighted many of the decision inputs made by all teams in the industry. Teams could scan across and compare their various marketing decisions to determine the effectiveness of various strategies. When a team came to us for help, it was always possible to explain post hoc the success or failure of the various teams. The relationship of advertising, price, etc. of the various teams fairly clearly led to the actual sales level. With Version 5, and more so with Version 6, teams have started to get results that are occasionally inexplicable. Teams with more advertising and lower cost (strong demand drivers) lose out to teams with more dealers (a weak driver). We cannot conclusively state that the added complexity of the demand algorithm has led to occasional unpredictable results. But it certainly seems to us as experienced instructors that although the simulation generally provides results explainable by a team’s actions, there are instances where the results cannot be explained. Think of the credibility problem for a team of students that has seemingly outperformed its competitors on the critical demand drivers yet sells less product. Such events undermine the credibility of the simulation and make it very difficult for the instructor to explain to the students how to improve. 198 CONCLUSION This article was not intended as a condemnation of the BSG. The authors believe it to be an excellent simulation on balance and continue to use it. But in reviewing and using other simulations, we see a similar predilection by designers to add complexity with succeeding versions. Adding features to a simulation can be highly beneficial if they lead to increased realism without unnecessary complexity. To the extent that they increase complexity without significant learning increases, or distract designers from fixing known deficiencies, then they detract from the simulation’s value. Although a simulation that truly mirrored all aspects of environmental reality would be far too complex to learn in a classroom, one that adequately captures the relationships between production, marketing, and resource management such that students can derive a positive learning experience have been successfully developed. Bells and whistles may be seen as marketing devices, but they can detract significantly from learning and thus the overall value of the simulation. REFERENCES Biggs, W. D. 1990. Introduction to computerized business management simulations. In James W. Gentry (Ed.). Guide To Business Gaming and Experiential Learning. 23-35. East Brunswick, NJ: Nichols/GP Publishing. Biggs, W. D., & Greenlaw, P. S. (1976). The role of information in a functional business game. Simulation & Games, 7, 53-64. Cangelosi, V. E. & Dill, W. R. (1965). Organizational Learning: Observations toward a theory. Administrative Science Quarterly, 10, 175-203. Cannon, H. M., & Burns, A. C. 1999. A framework for assessing the competencies reflected in simulation performance. Developments In Business Simulation and Experiential Learning, 26, 40-44. Chisholm, T. A. (1979). An examination of the perceived effectiveness of computer simulations in a classroom setting as affected by game, environmental and respondent characteristics. Developments In Business Simulation and Experiential Learning, 6, 103-109. Dickinson, J. R., & Faria, A. J. (1994). A random strategy criterion for validity of simulation game participation. Developments in Business Simulation & Experiential Exercises, 21, 35-39. Edwards, W. F. (1987). Learning macroeconomic theory and policy analysis via microcomputer simulation. Developments In Business Simulation and Experiential Learning, 14, 50-53. Faria, A. J. 1989. Business gaming: Current usage levels. Journal of Management Development, 8(2), 207-223. Faria, A. J. 1987. A survey of the use of business games in academia and business. Simulation & Games, 18, 207- 224. Faria, A. J., & Nulsen, R. O. (1976). Student reactions and evaluations towards differing assignment levels in a marketing simulation competition. Developments In Business Simulation and Experiential Learning, 3, 26- 35. Faria, A. J., & Whitely, T. R. (1990). An empirical evaluation of the pedagogical value of playing a simulation game in a principles of marketing course. Developments In Business Simulation and Experiential Learning, 17, 53-57. Gentry, J. W. 1990. What is experiential learning? In James W. Gentry (Ed.). Guide To Business Gaming and Experiential Learning. 9-20. East Brunswick, NJ: Nichols/GP Publishing. Gosenpud, J., & Washbush, J. (1994). What simulation users think players should be learning from simulations. Developments in Business Simulation & Experiential Exercises, 21, 96-99. Greenlaw, P.S., & Wyman, F. P. 1973. The teaching effectiveness of games in collegiate business courses. Simulation and Games, 4(3). 259-293. Hall, D. R. (1987). Developing various student learning abilities via writing, the Stock Market game, and Modified marketplace Game. Developments In Business Simulation and Experiential Learning, 14, 84-87. Joldersma, C., & Geurts, J. L. A. 1998. Situation/gaming for policy development and organizational change. Simulation and Gaming, 29(4), 391-399. Keys, J. B. 1976. A review of learning research in business gaming. In B. H. Sord, (Ed.), Computer simulation and learning theory. 173-184. Keys, H. B., & Biggs, W. D. 1990. A review of business games. In James W. Gentry (Ed.). Guide To Business Gaming and Experiential Learning. 48-73. East Brunswick, NJ: Nichols/GP Publishing. Keys, J. B., & Wolfe, J. (1990). The role of management games and simulations in education and research. The Journal of Management, 16, 307-336. Philippatos, G. C., & Moscato, D. R. (1969). An empirical study of the learning aspects of experimental game playing. AIIE Transactions, 1, 343-348. Rollier, B. 1992. Observations of a corporate facilitator. Simulation and Gaming, 23(4), 442-456. Snyder, S. J. (1995). A new twist on an old game: The business strategy game: A global industry simulation. Developments in Business Simulation & Experiential Exercises, 22, 134-135. Teach, D. T., & Govahi, G. (1993). The role of classroom techniques in teaching management skills. Simulation and Gaming, 24, 429-445. Thompson, A. A. & Stappenbeck, G. J. 1999. The Business Strategy Game. 6th ed. Boston, MA: Irwin/McGraw-Hill. Whitely, T. R., & Faria, A. J. (1989). A study of the relationship between student final exam performance and simulation game participation. Simulation & Games, 20, 44-64. 199 Wolfe, J. 1985. The teaching effectiveness of games in collegiate business courses: A 1973-1983 update. Simulation and Games, 16(3), 251-288. 200 Table of Contents Volume 28, 2001 Using A Web Based Tutorial Program To Enhance Student Learning Reflecting On Reflection: An Examination Of Reflective Learning And Assessing Outcomes Overcoming Obstacles To A Cross-Global Collaborative Classroom Panel Discussion On Building And Maintaining Trust In The Abselesque Classroom Cosmopolitan-Based Cross National Segmentation In Global Marketing Simulations Strategic Management And The Case Method: Survey And Evaluation In Hong Kong Designing Interactive Self-Learning Modules Using Macromedia Director Teaching Financial Markets Using The Wall Street Journal Core Course Outcomes Assessment And Program Continuous Improvement Using An Integrative Business Plan Œ An Empirical Evaluation Using A Mock Trial Activity For Interdisciplinary Learning Fidelity, Verifiability, And Validity Of Simulation: Constructs For Evaluation 2001 International Collegiate Business Strategy Competition: 37 Years Of Collegiate Competition Motivating Students: An Initial Attempt To Operationalize The Curiosity Gap Model E-Learning: The Next Wave Of Experiential Learning Helping New Game Adopters: Four Perspectives An Initial Validity Invesitigaion of A Test Assessing Total Enterprise Simulation Learning Administering A Marketing Simulation - Common And Varied Practices Among Instructors Issues In Case Method Instruction: Class Discussion Leadership Sex Composition, Cohesion, Consensus, Potency And Performance Of Simulation Teams Integrating Management Curriculum: Linking Courses With Frontpage Absel: The Way We Talk! Teaching Strategic Management: A Case Study In The Diffusion Of Innovation In Education A Demonstration Of Bussim Strategy A Total Enterprise Simulation Teaching Methods: Etic Or Emic Antecedents Of Work Team Performance In A Business Simulation: Personality And Group Interaction When Experiential Educators Reflect Upon Their Craft: A Qualitative Research Report Strategic Sensemaking In Organizations: Model Formulation And Operationalization Panel Discussion: A Model For Initiative Development: Discussion And Ideas Of Individual Spirit Harmonics Assessing The Efficacy Of Experiential Learning In A Multicultural Environment Are Business Schools Producing 21st Century Managers? Blue Horizon Cruises: An International Simulation Exercise To Illustrate Conflicting Economic And Cultural Models Different Perceptions Of The Same Thing: An Experiential Exercise Current Use Of Agribusiness Simulation Games: Survey Results Of University Agribusiness And Agricultural Economics Programs Team Building 101 For Accountants An Online Evaluation Of The Compete Online Decision Entry System (Codes) Total Enterprise Simulation Winners And Losers: A Preliminary Study Gilding The Lily, Enhancing The Pedagogical Experience, Or Fixing What Isn™t Broke: The Dilemma Of New Simulation Versions The Absel Endnote Database: The Perfect Tool For The Bernie Keys Library Decision-Making Exercises (A), (B), And (C): Examining The Role Of Conflict In Group Decision-Making Group Decision Simulation (A)-(F): Examining Procedural Justice In Group Decision-Making Processes Evaluation Of Performance In Business Games: Financial And Non Financial Approaches A Framework For Evaluating Simulations As Educational Tools Congruence II™: A Strategic Business Board Game The Tournament Concept: Extending The Idea Of Using Simulations As Instruments Of Assessment The Economic And Political Transition Of The Mexican State In The Threshold Of Twenty First Century: From The Entrepreneurial State To The State Of Entrepreneurs. Experiential Learning For Accountants: The Not For Profit Project An Investigation Of The Environmental Awareness Attained In A Simple Business Simulation Game A Daily Exercise In Theory Application: Is This Why I Had To Take Marketing, Management, Finance –? An Alternative View to Scoring Selected Simulation Games and Experiential Exercises