The Simplicity Paradox: Another Look at Complexity in Design of Simulations and Experiential Exercises THE SIMPLICITY PARADOX: ANOTHER LOOK AT COMPLEXITY IN DESIGN OF SIMULATIONS AND EXPERIENTIAL EXERCISES Hugh M. Cannon Wayne State University hugh.cannon@wayne.edu Daniel P. Friesen Wayne State University dy8750@wayne.edu Steven J. Lawrence Wayne State University stevenjl78@gmail.com Andrew H. Feinstein California State Polytechnic University, Pomona andyf@csupomona.edu ABSTRACT Cannon sees complexity as the amount of knowledge game players need to apply – to think about – in order to connect their decisions with the impact they will have on company performance. He suggests that game designers can defeat the paradox, incorporating greater levels of complexity in the game, by evoking information processing mechanisms that increase the ability of game players to pro suggests: ABSEL scholars have a long tradition of grappling with the problem of complexity in game design and performance. The effectiveness of any solution depends on the definition of complexity. We propose a two-dimensional definition, distinguishing between complexity resulting from too much information (information overload) and complexity resulting from too little (uncertainty). This paper draws on Cannon’s (1995) theory for managing complexity and the apparent contradictory findings of Wolfe and Castrovianni’s (2006) laboratory study on the use of “strategic chunking” to illustrate the simplicity paradox: The application of simplifying mechanisms for managing information-load might actually increase overall complexity by increasing uncertainty. The paper discusses the nature of the information-load/uncertainty trade-off and its implications for game design. cess information. Specifically, he • Strategic  chunking  ‐  where  players  effectively  reduce  the  amount  of  information  they  need  to  process by grouping, or “chunking,” a set of related  ideas into a higher‐level, more abstract (“strategic”)  concepts.   • Sequential  elaboration  –  where  players  reduce  effective  complexity  by  breaking  complex  thinking  into smaller, less complex parts, spreading them out  over time.  • Organizational  specialization  and  coordination  –  where  players  reduce  individual  complexity  by  distributing  components  of  the  complex  tasks  among  different  members  of  their  a .  INTRODUCTION simulated  organiz tion • Intermediate  measures  of  performance  –  where  games  are  structured  to  reward  players  for  successful  performance  of  a  component  task  (such  as  forecasting  or  demand  creation)  in  the  overall  management of a simulated firm.  In his 1995 article on the “Dealing with the Complexity Paradox in Business Simulations,” Cannon suggested that the more accurately simulations model the kinds of situations participants would encounter in a real business organization, the more complex the simulation would be. While moderate amounts of complexity can be helpful to learning, too much can create information overload, and actually decrease the amount of learning achieved by simulation participants (Fritzsche and Cotter 1990). Hence, the paradox: In order to make simulations realistic, we risk making them too complex, thus reducing their teaching effectiveness. Cannon’s definition of complexity comes from the work of 243 | Developments in Business Simulation and Experiential Learning, Volume 36, 2009 mailto:hugh.cannon@wayne.edu mailto:dy8750@wayne.edu mailto:stevenjl78@gmail.com mailto:andyf@csupomona.edu Our departure from the information-load definition of complexity is triggered by Wolfe and Castrovianni’s findings. Notwithstanding the fact that students were primed to recognize and use the abstract (chunking) concepts, the students did not correctly perceive the simple/static or complex/dynamic environmental conditions, nor did they respond with strategically consistent sets of decisions. The chunking strategy didn’t work! Burns, Gentry, and Wolfe (1990), who see complexity as the number of decisions, the number of functions, and the abstraction of the concepts embedded in the game. An increase in the number of decisions and functions results in greater complexity. As we have noted above, abstraction (the process of “chunking”) reduces it. According to Burns, Gentry, and Wolfe (1990), “Abstraction refers to the stripping away of irrelevant details and aspects and simplifying the rule such that it characterizes the operation The villain appears to have been the simplicity paradox: The abstractions that were designed to reduce information load actually created a new type of complexity! Students would have no trouble perceiving the specific characteristics of the game through which the information-load complexity was created – the number of countries, products, and so forth. However, to make use of abstract environmental categories, the students would, first, have to recognize that abstractions are appropriate, select the proper set, and then match them to the specifics of the situation. This adds complexity by creating enormous uncertainty. of the experiential exercise.” While Cannon’s framework is useful as far as it goes, it is limited by its definition of complexity. The definition involves what we might call information load, or the amount of information game players must effectively process in order to make an optimal set of decisions. In this paper, we will introduce uncertainty as a second dimension of complexity. We argue that creating simplifying mechanisms such as the ones Cannon identifies can actually increase complexity by requiring students to select schemata by which they will organize information, and to classify data according to the abstract categories embodied in the selected schemata. While the use of simplifying mechanisms may reduce the amount of information players have to process at one time by spreading it out over manageable chunks, uncertainty regarding the correct schemata and classification required to apply them calls for internal comparison and evaluation of alternatives, thus creating a new type of information-processing problem. This is the essence of “the simplicity paradox.” Of course, “strategic chunking” is only one of Cannon’s (1995) simplifying mechanisms. Exhibit 1 portrays steps that a student would have to go through in order to select and apply one of the mechanisms. When we consider the actual steps students must go through to simplify the decision-making process, we begin to see just how complex the “simplifying” process really is. Diagramming it, even with the many oversimplifications incorporated in the Exhibit, leaves us with a new appreciation of why Wolfe and Castrovianni’s (2006) students failed in their application of the “strategic chunking” principles. THE SIMPLICITY PARADOX This is not an argument for rejecting the simplifying mechanisms. The mechanisms are central to organizations’ ability to process large amounts of information. The information-processing burden can be enormous, even in a simulation game. For instance, suppose that a game involved 30 player decisions, and that the players were presented with 60 pieces of information that would potentially be relevant to one or more of these decisions. With no simplifying mechanism, the students would have to evaluate the potential implications of (60 x 30 =) 1,800 information-decision relationships. And this does not take into account the fact that the pieces of information and decisions might work in combination with each other, creating a virtual infinity of interaction effects that must also be taken into account! Burns, Gentry, and Wolfe’s (1990) inclusion of “abstraction” in their definition of complexity provides a good point of departure for our discussion of the simplicity paradox. As we have seen, abstraction (“strategic chunking”) provides a mechanism for reducing complexity in simulations. Indeed, we could argue that all of Cannon’s simplifying mechanisms discussed in the previous section are themselves abstractions – general approaches that can be used to reduce the information load involved in successfully playing a simulation game. In order to illustrate the paradox, we can draw on Wolfe and Castrovianni’s (2006) use of Duncan’s (1972) simple/static and complex/dynamic environmental taxonomy. They investigated the way MBA students used these to enhance performance in Wolfe’s The Global Business Game. Wolfe and Castriovianni primed their students to “chunk” the myriad of specific environmental characteristics encountered in the game into Duncan’s simple/static and complex/dynamic typology. The environmental characteristics included such characteristics as the number of country markets, the number of products, product/market growth rates, accuracy of forecasts, and the rate and predictability of change in costs and prices. The students were exposed to the Duncan typology and its associated concepts through lectures, discussion, and a host of web-based support materials. Students were also exposed to the strategies that would be appropriate for each environmental condition. According to Cannon’s (1995) theory, students would apply “strategic chunking,” using the environmental typology to simplify their analysis of environmental data in the game and using strategic responses to simplify their handling of the many decisions they would have to make. In practice, simplifying mechanisms are not an all-or- nothing proposition. To survive in a simulated business environment, and indeed, to survive in life, people develop simplifying heuristics that parallel the mechanisms we have been discussing. For instance, confronted with a pricing decision, players will automatically draw on their general knowledge of pricing to chunk relevant pricing-related information. They will likely separate pricing from production decisions in some form of sequential elaboration. And so forth. The analyses Wolfe and Castrovianni (2006) were discussing are advanced concepts, related to high-level management theories that have potential for reducing large numbers of highly complex relationships into a relatively few, strategic patterns. To do this, the concepts must necessarily be very abstract, and the price we pay for the abstraction is uncertainty regarding what theory to apply and how to apply it. 244 | Developments in Business Simulation and Experiential Learning, Volume 36, 2009 CONCEPTUALIZING COMPLEXITY The simplicity paradox notwithstanding, the information- load approach to complexity is technically correct, if complexity is a function of the human information processing. In fact, “complexity” serves as an umbrella label for a number of theories, ideas and research programs that are derived from scientific disciplines such as meteorology, biology, physics, chemistry and mathematics (Rescher 1996; Stacey 2003). While the technical differences in these approaches to complexity have not yet been entirely established or reconciled, looking at the application of the general concept to simulation and gaming, we have made progress toward building a framework. We have already discussed Cannon’s (1995) theory of complexity and simplifying mechanisms. In contrast to Cannon’s cognitive approach, Wolfe (2005) points to the importance of software-based coaching materials, corresponding to decision-support systems in industry, or what Feinstein, Martin, and Ogawa (2001) refer to as “cognitive prostheses.” Searching the 2008 edition of the Bernie Keys Library, we find 179 articles addressing decision support in one form or another. Addressing what we have called the uncertainty dimension, Gosen and Washbush (2005) point to the need for reflection on the meaning of the game for abstraction to have its simplifying benefit (Gosen and Washbush 2005). Simplifying mechanisms notwithstanding, there appears to be a better learning effect in moving from a less complex to a more complex simulations (Wellington, Faria and Hutchinson 2007). In the end, the complexity of a simulation game is expressed in the amount of information processing required to make effective decisions. From this perspective, Exhibit 1 provides a useful picture of the way simplifying mechanisms create complexity. For instance, Box E suggests that players will begin by trying to classify the game situation in such a way that it can be fit to an existing knowledge structure (schema) about which we already have some information. This requires sorting through a candidate group of schemata (Box I) to find one that might apply to the game – Duncan’s (1972) environmental taxonomy, for example. The Game Decision Process with and without Evoking a Simplifying Mechanism Exhibit 1. Without mechanisms With mechanisms Box F suggests that players would search for simplifying mechanisms (Box J) that fit the game-classifying schema selected in Box E. For instance, having conceptualized Wolfe’s (2003) The Global Business Game in terms of Duncan’s environmental taxonomy, a team might choose to organize itself into specialized functional or geographic sub-units, depending on whether the game’s environment fell into the simple/static or complex/dynamic classification. Moving on to Box G, players would sort through the various characteristics of the game (Box K), seeking to classify them according to the schema selected in Box E, then applying them to the simplifying mechanisms selected in Box F. Again, using the Global Business Game as an example, players would allocate decisions to the appropriate sub-units of the team and seek to identify critical informational characteristics that the units would need to make their decisions. In the implementation stage represented by Box H, team would establish control mechanisms to ensure timely work flow and monitor decisions for quality and consistency with any broader strategic guidelines established for the company as a whole. While the analyses summarized in Boxes E through H all involve information processing, thus linking them to the 245 | Developments in Business Simulation and Experiential Learning, Volume 36, 2009 information-load approach to complexity, the information processing is driven by too little rather than too much information. They represent a second, uncertainty approach. Putting the information load and uncertainty approaches together, we can map the two dimensions as illustrated in Exhibit 2. We are reluctant to put labels in the cells of Exhibit 1, because the literature offers no clear vocabulary for addressing the distinctions the Exhibit embodies. Nevertheless, the distinctions are significant, and roughly correspond to the terms we use. The literature addresses the fact that simulations differ in their complexity. The premise of these studies has generally been the “information load” approach to complexity, focusing primarily on the number of decisions required by the simulation (Keys and Biggs 1990). Gentry (1990) appears to address the uncertainty dimension in his discussion of computer-assisted instruction (CAI) versus live-case exercises. He argues that CAI has relatively little potential for experiential learning, because it is oriented toward content-based learning and involves very little variability, uncertainty, or environmental interaction. By contrast, a live case has high potential, involving relatively little structure, high variability, uncertainty, and environmental interaction. In fact, the potential uncertainty is so great, that Gentry makes special note of how important instructor guidance is to facilitate a successful exercise. Gentry’s discussion suggests that coping with uncertainty is an important element in experiential learning. While the literature makes no formal distinction between an experiential exercise and a simulation, the general sense is that an exercise is more limited in its scope and objectives. According to Warrick, Hunsaker, Cook and Altman (1979, p. 92), “An experiential learning exercise may be defined as a task or activity involving participants that is designed to generate ‘live’ data and experiences that can be used to teach concepts, ideas, or behavioral insights.” In other words, an experiential exercise seeks to give meaning to a relatively few abstract concepts. This provides the rationale for using “experiential exercise” (cell 4 of Exhibit 2) to represent activities characterized by high uncertainty and low information load. A Two-Dimensional Model of Simulation Complexity Exhibit 2. HighLow In fo rm at io n Lo ad H ig h Lo w Uncertainty Complex simulations Simple simulations Structured exercises Behavioral exercises (1) (2) (3) (4) In contrast to experiential exercises, a “simulation places the student in a dynamic decision-making environment that forces the student to make decisions under many of the same pressures the business executive faces” (Nulsen and Faria 1977, p. 217). “Simple simulations” (cell 3) incorporate a relatively small number of relatively easy-to-conceptualize variables to represent such a decision environment. “Complex simulations” (cell 2) involve a larger number of decisions (Keys and Biggs 1990). The larger number lends itself to simplification through Cannon’s (1995) simplifying mechanisms. However, these introduce a level of abstraction, and hence, uncertainty into the game. Finally, we have used the term “structured exercises” (cell 1) to represent low-uncertainty, high-information-load activities. In fact, the classification is broad. Dougherty (1975) uses the term to represent virtually any exercise where the learning objectives procedures are specified ahead of time, contrasting it with “unstructured exercises” such as T-groups. At the extreme, a prototypic structured exercise would be the computer-assisted instruction discussed by Gentry (1990) when evaluating various types of exercise relative to the degree of experiential learning they are likely to promote. PRINCIPLES OF DESIGN Our discussion suggests an emergent theory for managing the amount of complexity designed into business simulation games and experiential exercises. The underlying theory is captured in Exhibit 2, based on the simplicity paradox, which in turn grows out of the interaction between the information load and uncertainty dimensions of complexity. Again, we are reluctant to put labels on the cells. However, the conventions adopted for Exhibit 2 are robust enough to provide a general set of guidelines. We will seek to clarify them with additional discussion and examples of what they might represent. Beginning with cell 1, we note that structured exercises 246 | Developments in Business Simulation and Experiential Learning, Volume 36, 2009 could certainly be used for low as well as advanced levels of instruction. However, the power of a structured experiential exercise is to immerse students in an information-processing experience using tools that are difficult to master, but are relatively well defined. For instance, Cannon and Alex (1990) present a method for structuring a live case using a hierarchy of strategy and objectives patterned after Colley’s (1961) DAGMAR (Defining Advertising Goals for Measured Advertising Results) system, where the strategy at one level of planning provide the objectives for the next level. By providing a series of rigorously developed planning forms and procedures, the exercises such as this focus student attention on the specific concepts they need to learn (Corner and Nicholls 1994). Within the structure of the planning system, students engage in what Kolb (1984) calls “accommodative,” and eventually, “reflective” learning, relying on concrete experience to practice applying the concepts embodied in the planning system, and eventually conceptualizing new problem-solving learning to approaches (Cormany and Feinstein 2008). Contrast this with cell 2, where a complex simulation incorporates both high levels of information and also high-level patterns of strategic response. That is, the information- processing tools involve abstract thinking, and are neither well defined nor easy to apply. We saw this in Wolfe and Castrovianni’s (2006) study of MBA students using Wolfe’s (2003) The International Business Game. The game called for environmental analyses and strategic responses that the students were unable to deliver, notwithstanding their having been briefed on the theory prior to the game. One way to conceptualize the differences in pedagogical strategy between cell 1 and cell 2 is to recognize that both require information-simplifying mechanisms. In cell 1, the mechanisms are built into the assignment. For instance, in Cannon and Alex’ (1990) system, the structure of the assignment focuses student attention on the key concepts and theories relevant to each level of planning (“strategic chunking”). The planning hierarchy provides a model for “sequential elaboration.” And the outcome of each separate assignment provides “intermediate measures of performance.” In cell 2, the mechanisms are implicit in the theoretical discussion at the beginning of the simulation game, and in the support materials provided to the students. However, there is no structure within the game to ensure that the students use the appropriate mechanism. Consider the intellectual skills required (and developed) by these two approaches. Bloom’s revised taxonomy of educational objectives provides a useful tool for conceptualizing these skills in the context of simulation games and experiential learning (Cannon and Feinstein 2005). Exhibit 3 summarizes the key concepts. While Bloom’s initial taxonomy (1956) only addressed low-level knowledge and high-level thinking processes, the revised taxonomy addresses knowledge and processes as separate dimensions. The structure in cell 1 exercises supply the knowledge regarding how to structure and solve a problem, so the students have only to apply it. In cell 2, students must use higher-level thinking skills, such as analysis and evaluation, to reduce the ambiguity. That is, they must differentiate between the various competing theories and concepts, and then judge the game against the criteria they establish .In many cases, they must go beyond this and use creative skills to synthesize data that are simply not available to players in a simulation game. For instance, they might reason that the advertising response curve for a new product would be relatively steep, as consumers absorb important, but otherwise unavailable, product information. In each case, we assume that learning takes place. That is, we assume the process students have used gets encoded in knowledge structures that are then available for future use. In the end, the differences in the approaches are reflected in the knowledge structures students develop. These knowledge structures – schemata – help students recognize and address similar problems in the future, thus reducing uncertainty. Exhibit 4 illustrates the practical impact of design strategy and learning captured in cells 1 and 2 of Exhibit 2. As we have The Structure of the Revised Technology Exhibit 3. The Cognitive Process Dimension The Knowledge Dimension 1. Remember 2. Understand 3. Apply 4. Analyze 5. Evaluate 6. Create A. Factual knowledge B. Conceptual knowledge C. Procedural knowledge ‘D. Meta-cognitive Knowledge Source: Lorin W. Anderson and David R. Krathwohl. A Taxonomy for Learning, Teaching, and Assessing. New York: Longman, 2001, p. 28. 247 | Developments in Business Simulation and Experiential Learning, Volume 36, 2009 noted, in the structured exercise, the knowledge structures containing the simplifying mechanisms are built into the assignment. This effectively decreases complexity within the exercise, first by structuring the information-processing mechanisms to reduce uncertainty, and second, by using the mechanisms to reduce information load. Students in the structured exercise still face uncertainty. It will come once they have left the class and try to apply the concrete structure they have learned from the exercise to practical business situations. This is shown on the right-hand side of the exhibit, where students are required to map the concepts they have learned onto the external environment. The structure fostered low-level knowledge structures, such as procedural knowledge of applications – applications that are generally not robust enough to be useful in an actual working environment. We see the opposite phenomenon on the left-hand side of the exhibit. Players of the complex simulation must figure out how to create knowledge and apply the simplifying mechanisms, following the process portrayed in Exhibit 1. This creates uncertainty within the game, thus making it more complex. However, such a game would be designed to develop the general knowledge structures needed to deploy simplifying mechanisms – conceptual, procedural, and metacognitive knowledge related to analyzing, evaluating, and creating business strategies. Given their higher level of abstraction, these structures would likely be more robust in their real-world application, thus reducing uncertainty in mapping concepts from the game against practical business problems. At this point, we might ask, “So why wouldn’t we all use complex simulation games?” The answer lies in the dark side of complexity, and the rationale for strategies arising from cell 3 of Exhibit 2. Too much complexity can be overwhelming. This is apparent in a series of studies by Gentry and his colleagues that investigate Loewenstein’s (1994) “curiosity gap” concept in conjunction with experiential learning motivation (Gentry, Burns, Dickinson, Putrevu, Chun, Yu, Williams, Bare, and Gentry 2001; Gentry, Burns, Dickinson, Putrevu, Chun, Yu, Williams, Bare, and Gentry 2002; Gentry and McGinnis 2008). The Theory maintains that students’ motivation for learning is influenced by the importance of the material to be learned, and by the gap between what they know and the level of learning they must achieve. If the gap is too low the knowledge to be gained by effort may not be worth the cost; if it is too high the learner may decide he or she cannot bridge it. Teach and Murff (2007) have addressed this by advocating greater use of very simple games, to deliver learning in smaller, less complex doses. In context, their definition of simple reflects a traditional information-load approach, but it also addresses uncertainty. The Shifting Role of Uncertainty Exhibit 4. Strategic chunking Sequential elaboration Organizational specialization and coordination Intermediate measures of performance Information processing mechanisms built into the game Information processing mechanisms supplied by players Map game characteristics onto strategic concepts Design appropriate sequencing Design and administer organizational scheme Design intermediate measures Map concepts onto external reality Map sequencing process onto external reality Map organizational experience onto external reality Map intermediate measures onto external reality Information processing mechanisms Source of Uncertainty Source of Uncertainty A contrasting approach would be to opt for a low- information-load exercise, but load it with uncertainty. In this context, uncertainty would likely take the form of powerful interpersonal or intrapersonal variables, where students are confronted with a need to respond to new, highly ambiguous, and often emotionally charged situations. For instance, Ettinger (2004) discusses a “needle and thread” exercise, where the entire learning experience is built around trying to thread a needle blindfolded, using verbal coaching from two colleagues as a guide. The information load is minimal, but the uncertainty is enormous. Of course, there is the superficial question of how to thread the needle, but the real impact of the exercise is in the way students deal with issues of interdependency, trust, power, and feedback. “Just as Threaders who have their eyes closed, people in organizations commonly lack vision and do not know 248 | Developments in Business Simulation and Experiential Learning, Volume 36, 2009 where they stand in any big picture sense. Like Instruction Givers who must instruct Threaders to complete the task, managers in organizations must direct subordinates who may work very diligently to accomplish a task, but can lose or lack entirely a sense of direction and purpose” (Ettinger 2004, p. 100). SUMMARY AND CONCLUSIONS The purpose of this paper has been to explore the complexity paradox in simulation game design, as articulated by Cannon (1995). Cannon suggested that, in order to be sufficiently realistic to stimulate sophisticated managerial learning, the game had to be complex. When games become sufficiently complex to represent the kinds of issues managers actually have to deal with, their complexity obscures the cause- and-effect relationships that participants must see in order to learn from the game. Cannon suggests four simplifying mechanisms that organizations use to address the paradox in real organizations, suggesting that these should be used in simulations as well. Unfortunately, the mechanisms involve the application of high- level abstract thinking, which introduces a new kind of complexity. The implications are apparent in Wolfe and Castrovianni’s (2006) study of MBA students who were briefed on high-level concepts (Duncan’s 1972 theory of environmental complexity) that should have helped them simplify their view of what was happening in a complex multi-national simulation game. The students failed to conceptualize the game in the manner required to apply Duncan’s theory. This suggests a second paradox – what we have called the simplicity paradox. The simplicity paradox says that developing the kind of simplifying mechanisms Cannon recommends can actually increase complexity. The underlying theory grows out of a two-dimensional concept of simulation-design complexity. First, the information-load dimension addresses the number of decisions and other concrete informational cue to which game participants must respond. Too much information causes cognitive overload (complexity). Second, the uncertainty dimension addresses the information game participants must supply in order to organize the informational cues they receive into meaningful (actionable) patterns, thus resolving the ambiguity inherent in the game. Whereas the first dimension addressed too much information, the uncertainty results from too little. The implications of the simplicity paradox are twofold: First, students must learn how to use the higher-level thinking skills that enable them to deploy the simplifying mechanisms needed to cope with information overload. In the end, this is the primary purpose using simulation games as a learning experience. Second, these higher-level thinking skills are learned through playing the simulation games for which they are needed. However, the complexity of the game must be staged according to the students’ ability. Lowenstein’s (1994) curiosity-gap theory suggests that students will be motivated to learn concepts that they see as relevant, providing they are not too simple or too complex. From a very practical perspective, game designers need to tailor the level of complexity of their simulations to the specific needs of their students. Game participants are dealt an array of decisions which they must make. Among these they need to decide which can be managed individually and which will need to be strategically “chunked” to simplify the decision process. The journey in Exhibit 2, from cell 3 (simple simulations), to cell 2 (complex simulations) can be managed in the design process as we introduce information load and uncertainty requirements, consciously, as discrete elements. Game designers could alternately route through cell 1 (Structured exercises) or cell 4 (Behavioral exercises) as an intermediate step between cells 3 and 2. The simplicity paradox theory has a number of research implications. The first, of course, is to validate the theory itself – to test the two-dimensional nature of complexity and validate its propositions regarding the impact of information overload versus uncertainty. From a more applied perspective, it begs an investigation of how simulation designers might measure the complexity embedded in their game, particularly complexity along the uncertainty dimension. This question will be inextricably connected with questions regarding the measurement of student knowledge structures and their ability to apply them in uncertain environments. Of course, these questions are well known in the simulation and gaming literature. REFERENCES Anderson, Lorin W. and David R. Krathwohl (2001). A Taxonomy for Learning, Teaching, and Assessing: A revision of Bloom’s taxonomy of educational objectives. New York: Longman. Bloom, Benjamin S., N. D. Englehart, E. J. Furst, W. H. Hill, and David R. Krathwohl (1956). Taxonomy of Educational Objectives – The Classification of Educational Goals, Handbook I: Cognitive Domain. New York: David McKay Company. Burns, Alvin C., James W. Gentry, and Joseph Wolfe (1990). A cornucopia of considerations in evaluating the effectiveness of experiential pedagogies. In James W. Gentry (ed.). Guide to Business Simulation and Experiential Learning. East Brunswick: Nichols/GP Publishing, 253-278. (Reprinted in the Bernie Keys Library, available from http://absel.org). Cannon, Hugh M. (1995). Dealing with the complexity paradox in business simulation games. Developments in Business Simulations and Experiential Exercises, 22, 96-102. (Reprinted in the Bernie Keys Library, available from http://absel.org). Cannon, Hugh M., James N. Cannon, and Manfred Schwaiger (2005). Simulating customer lifetime value: Implications for game design and student performance. Developments in Business Simulations and Experiential Exercises, 32, 54- 064. (Reprinted in the Bernie Keys Library, available from http://absel.org). Cannon, Hugh M. and Andrew H. Feinstein (2005). Bloom beyond Bloom: Using the revised taxonomy to develop experiential learning strategies. Developments in Business Simulations and Experiential Exercises, 32, 348-356. (Reprinted in the Bernie Keys Library, available from http://absel.org). Cannon, Hugh M. and Theodore C. Alex (1990). Structuring a live case: A method and example from an introductory 249 | Developments in Business Simulation and Experiential Learning, Volume 36, 2009 advertising class. In James W. Gentry (ed.). Guide to Business Simulation and Experiential Learning. East Brunswick: Nichols/GP Publishing, 216-230. (Reprinted in the Bernie Keys Library, available from http://absel.org). Colley, Russell H. (1961). Defining Advertising Goals for Measured Advertising Results. New York: Association of National Advertisers. Cormany, Dan and Andrew Hale Feinstein (2008). Implementation of effective experiential learning environments. Developments in Business Simulations and Experiential Exercises, 35, 34-60. (Reprinted in the Bernie Keys Library, available from http://absel.org). Corner, Lucette and J. A. F. Nicholls (1994). Enhancing a computer simulation with a structured reporting environment. Developments in Business Simulations and Experiential Exercises, 21, 40-44. (Reprinted in the Bernie Keys Library, available from http://absel.org). Dougherty, Robert H. (1975). The future potential of structured learning exercises. Business Games and Experiential Learning in Action, 2, 382-386. (Reprinted in the Bernie Keys Library, available from http://absel.org). Duncan, Robert B (1972). Characteristics of perceived environments and perceived environmental uncertainty. Administrative Science Quarterly 17:3 (September), 313- 327. Ettinger, Judith S. (2004). Needle and thread: an activity for examining various management behaviors. Developments in Business Simulations and Experiential Exercises, 31, 99- 101. (Reprinted in the Bernie Keys Library, available from http://absel.org). Feinstein, Andrew Hale, James Edward Martin, and Roy Ogawa (2001). Using a cognitive prosthesis to assist foodservice managerial decision-making. International Journal of Hospitality Information Technology. 2(1), 41-55. Fritzsche, David J. and Richard V. Cotter (1990). Guidelines for administering business games. In James W. Gentry (ed.). Guide to Business Gaming and Experiential Learning. East Brunswick: Nichols/GP, 74-89. (Reprinted in the Bernie Keys Library, available from http://absel.org). Gentry, James W. (1990). What is experiential learning? In James W. Gentry (ed.). Guide to Business Simulation and Experiential Learning. East Brunswick: Nichols/GP Publishing, 9-20. (Reprinted in the Bernie Keys Library, available from http://absel.org). Gentry, James. W., Alvin C. Burns, Sanjay Putrevu, Yu Hongyan, Laura Williams, Thomas Bare, and Ruth Ann Gentry (2001), Motivating Students: An Initial Attempt to Operationalize the Curiosity Gap Model. Developments in Business Simulations and Experiential Learning, 28, 69-75. (Reprinted in the Bernie Keys Library, available from http://absel.org). Gentry, James W., Alvin C. Burns, John R. Dickinson, Sanjay Putrevu, Seungwoo Chun, Hongvan Yu, Laura Williams, Thomas Bare, Thomas; and Ruth Ann Gentry (2002). Managing the curiosity gap does matter: What do we need to do about it? Developments in Business Simulations and Experiential Learning, 29, 67-73. (Reprinted in the Bernie Keys Library, available from http://absel.org). Gentry, James W., and Lee Philip McGinnis (2008). Thoughts on how to motivate students experientially. Developments in Business Simulations and Experiential Learning 35, 73- 77. (Reprinted in the Bernie Keys Library, available from http://absel.org). Gosen, Jerry; and John Washbush (2005). Analyzing and thinking while playing a simulation. Developments in Business Simulation and Experiential Learning 32, 141-143. (Reprinted in the Bernie Keys Library, available from http://absel.org). Keys, J. Bernard and William D. Biggs (1990). A review of business games. In James W. Gentry (ed.). Guide to Business Simulation and Experiential Learning. East Brunswick: Nichols/GP Publishing, 48-73. (Reprinted in the Bernie Keys Library, available from http://absel.org). Kolb, David A. (1984). Experiential learning: Experiences as the source of learning development. Englewood Cliffs, NJ: Prentice-Hall. Loewenstein, George (1994). The Psychology of curiosity: A review and reinterpretation. Psychological Bulletin 116:1, 75-98. Nulsen, R. O. and A. J. Faria (1977). New horizons in simulation research. New Horizons in Simulation Games and Experiential Learning, 4, 217-222 (Reprinted in the Bernie Keys Library, available from http://absel.org). Rescher, N. (1996). Complexity: a Philosophical Overview. New York: Transaction. Stacey, R.D. (2003). Strategic Management and Organisational Dynamics: The Challenge of Complexity. Harlow: FT/Prentice Hall. Teach, Richard D. (1990). Profits: The false prophet in business gaming. Simulation & Gaming 21:1 (December), 63-72. Teach, Richard D. and Elizabeth J. T. Murff (2007). The Wee Game: A pre-game. Developments in Business Simulations and Experiential Exercises, 34, 201-204 (Reprinted in the Bernie Keys Library, available from http://absel.org). Warrick, D. D. (Don), Philip L. Hunsaker, Curtis W. Cook, and Steve Altman (1979). Debriefing experiential learning exercises. Journal of Experiential Learning and Simulation 1, 91-100. (Reprinted in the Bernie Keys Library, available from http://absel.org). Wellington, William J.; Faria. A. J.; and Hutchinson, David (2007). Consistency of participant simulation performance across simulation games of growing complexity. Developments in Business Simulation and Experiential Learning 34, 24-31. (Reprinted in the Bernie Keys Library, available from http://absel.org). Wolfe, Joseph (2003). The global business game: A strategic management & international business simulation. Mason, OH: South-Western. Wolfe, Joseph (2003). Overcoming the business game complexity paradox. Developments in Business Simulations and Experiential Learning 32, 329-330. Wolfe, Joseph and Gary Castrovianni (2006). Business games as strategic management laboratories. Developments in Business Simulations and Experiential Exercises, 33, 31-40 (Reprinted in the Bernie Keys Library, available from http://absel.org). 250 | Developments in Business Simulation and Experiential Learning, Volume 36, 2009 http://absel.org/ http://absel.org/ http://absel.org/ http://absel.org/ http://absel.org/ http://absel.org/ http://absel.org/ http://absel.org/ http://absel.org/ Table of Contents Volume 36, 2009 Protecting Academic Integrity: Student Assessment In The Online Environment Pedagogical Shift: Teaching Report Writing For Accountants Online Increasing Student Learning In An Investment Management Course Through The Innovative Use Of Experiential Learning Pedagogy A Moral Development Unit For Business Courses Dividing Up Grandma’s Things: A Multifaceted Exercise In Critical Thinking Application Of Haekel’s Thesis To ABSEL Development Linking Stories With On-Line Threaded Discussions For Critical Thinking In Management Curriculum Why Have We Neglected Vicarious Experiential Learning? A Triadic Multi-Disciplinary Approach To Enhancing The Efficacy Of Experiential Learning Movement Toward Increased Student Roles In The Design Of Experiential Exercises Ready-Mix Concrete Company: An Experiential Exercise In Management Theory Using Portfolio Theory In A General Management Simulation An Experimental Analysis Of Advertising Strategies And Advertising The Relationship Between Goal Orientation And Simulation Performance With Attitude Change And Perceived Learning Beyond The Profitable-Product Death Spiral: Managing Product Mix In An Environment Of Constrained Resources Online Marketing Control With The Strategic Business Unit Analysis Package Beat The Market Game Creating Decision Support Systems In Business Simulation Games The Use Of Computer-Assisted, Interactive Role-Play Simulation In China Asia Marketing: An International Business Game An Empirical Test Of “Behavioral Immersion” In Experiential Learning Existing And Emerging Business Simulation-Game Design Movements Computerized Business Simulations: A Systems Dynamics Process Service Launch Evaluating Business Plans In A Simulation Environment Teaching Software Development By Means Of A Classroom Game: The Software Development Game Group Decision Experiments Using Business Game - Problem Solving With Conflict Luna: A Role Play Game For Learning Incoterms 2000 Entrepreneurship: A Game Of Risk And Reward Phase I -- The Search For Opportunity The Wee Game: A Pre-Game Learning Inhibitors In Business Simulations And Games New Version Of An Old Simulation Helps Build A Perfect Capstone Course Project Competitor: A Simulation Game For Project Management With 2 Models And 2 Modes Mitigating The Winner’s Curse In The Auction Market Of A Computer-Assisted Business Gaming Simulation Evaluation Of Collaborative Filtering By Agent-Based Simulation Considering Market Environment Marketing Simulation Game Decision Making Experience And Its Impact On Indecisiveness Among Introductory Marketing Students Individual And Organizational Learning In A Top Management Game In Pursuit Of Stockholder Value: Reinforcing Core Concepts In A Business Strategy Simulation With A “Shadow” Stock Market Competition The Simplicity Paradox: Another Look At Complexity In Design Of Simulations And Experiential Exercises In Search Of The Ethnocentric Consumer: Experiencing “Laddering” Research In International Advertising The Paradise Islands Revisited: Trouble In Paradise Developing And Assessing Student Information Literacy Competency Do First Mover Advantages Exist In Competitive Board Games: The Importance Of Zugzwang The Natural Debriefing Approach: A Case In A Simple Business Game Pursued For Perfect Communications New Product Development Simulation Dominance In On Line Business Games Competitions Cooperative Business Game With Framing Effect Experiential Exercises In The Online Environment Enhancing Web-Based Simulations With Game Elements For Increased Engagement The Ginseng Game Experience Business Using a Simple Business Simulation