Learning Inhibitors in Business Simulations and Games LEARNING INHIBITORS IN BUSINESS SIMULATIONS AND GAMES Richard Teach Georgia Institute of Technology richard.teach@mgt.gatech.edu Elizabeth J. T. Murff Eastern Washington University ejtmurff@gmail.com ABSTRACT Those that use business simulations in their classrooms are convinced of the educational viability of business games. However, most of the studies that attempt to demonstrate the learning that takes place in the course of playing a business games have not succeeded. In addition, most studies have shown no statistically significant relationships between a simulated firm’s performance and learning. WHY? This paper reviews some simulation research findings using Bloom’s cognitive hierarchy of learning, and links them to Maslow’s hierarchy needs. Then it considers the links between cognitive learning theory and its impact on game design and the impact of cognitive overload in complex business simulations and its impact on learning. SOME PRIOR RESEARCH FINDINGS There seems to be a great dilemma in the published literature regarding the amount of learning that takes place when participants are utilizing large-scale inclusive business simulation games. In the years since the advent of business simulations, authors have commented on the game participants’ greater attention and motivation (Shubik, 1972), depth of insight (Frazer, 1978) and speed of learning (Rogers & Freiberg, 1994) in these games. Yet in a review article, Anderson et al. (1998) reported that … “despite the extensive literature, it remains difficult, if not impossible, to support objectively even the most fundamental claims for the efficacy of games as a teaching pedagogy. There is relative little hard evidence that simulations produce learning or that they are superior to other methodologies.” Gentry et al. (2007) suggested that “Learning implies change in behavior over the course of the simulation competition.” (Quote from page .7) Ideally, teams should learn the demands of the simulation environment (Druskat & Kayes, 2002) and align members to meet those demands according to their role preferences (Kayes Kayes and Colb., 2005). Yet research has revealed that teams are unlikely to significantly change their relative performance during the game (Wellington & Faria, 1995). This suggests that significant behavior changes may not be taking place during the actual simulation and thus learning is not generally occurring. Numerous educational researchers (Burns & Burns, 1990; Cannon & Feinstein, 2005; Feinstein, 2004; Howard et al., 2006; Mujtaba & Kennedy, 2005; Schumann et al., 2001; and many more) have suggested that playing business games should produce abilities consistent with the higher levels of Bloom’s cognitive hierarchy of learning. In increasing order, these are (Bloom et al., 1956): • Knowledge -- Remembering previously learned material • Comprehension -- Grasping the meaning of material • Application -- Using information in concrete situations • Analysis -- Breaking down material into parts • Synthesis -- Putting parts together into a whole • Evaluation -- Judging value, using definite criteria Motorola University (1996) reported that “practice by doing” provides a 75% retention rate compared with the 5% retention rate of lectures, providing evidence of only the lowest level of Bloom’s hierarchy being positively impacted by these methods. Attempts to measure abilities consistent with the upper levels of Bloom’s hierarchy have not provided evidence of learning associated with large-scale business simulations as of this date. THE INTERFERENCE OF PSYCHOLOGICAL AND PHYSICAL NEEDS WITH LEARNING The taxonomies of learning have many obstacles along their paths. There are multitudes of learning inhibitors surrounding us every day and in every situation. Some of these impediments are described by Abraham Maslov as the hierarchy of needs (Maslov 1943). Consider a common occurrence for some students, the lack of sleep. When this happens, it is difficult for the participant to concentrate, remember or decide almost anything. Learning capacity suffers with sleep loss (Curcio, 2006 and Little et al. 2005). This then suggests a possible intervening variable as the participant’s cognitive level is limited to the lower stages of Bloom’s taxonomy by his unmet physiological need for sleep. As sleep is at the lowest level of Maslow’s hierarchy of needs, this suggests that these may be confounded with Bloom’s cognitive stages, at least for business simulations. In increasing 191 | Developments in Business Simulation and Experiential Learning, Volume 36, 2009  mailto:richard.teach@mgt.gatech.edu mailto:ejtmurff@gmail.com order, at least for the U.S. college students studied by Maslov, (1943) these needs are: • Physiological Needs -- Food, sleep, stimulation activity • Safety Needs -- Protection from physical or psychological harm • Belongingness -- Love, friendship, comradeship • Esteem Needs -- Self respect, personal worth • Self Actualization -- Full potential Now, consider a mid-level manager in a company sponsored executive training simulation where senior managers are carefully reviewing each participant’s performance or a student whose course grade depends upon his performance or others’ perceptions of his effectiveness. While simulations are often touted as a safe way to train, it is not very safe for these participants. Social ridicule (Cannon & Feinstein, 2007), reduced grades (Buenger et al., 2007) and missed promotions are very real safety threats. Game participants often play in a manner that minimizes these risks by avoiding experimentation with new concepts (Sujan et al., 1994). The lack of psychological safety keeps the participants from reaching the application or analysis levels of learning. Next, consider another fairly common occurrence, the dysfunctional team. Belonging and trust is necessary when the grades of every member are affected by the team’s performance (Bane, 2004; Duck, 2006; Hergeth, 2007; Mayer et al., 1995). Team formation, where the fear of stepping on each other’s toes often results in a dumbing-down effect, can also affect learning and performance (McKenny & Dill, 1966; Norris & Niebuhr (1980); Robinson, 1996). Poorly performing teams fail to respond appropriately to game feedback, resulting in mindless decisions in future inputs. As the cycle repeats itself, teams lose sight of their objectives and become de-motivated (Kayes, Kayes & Kolb, 2005). Trust leads to participative decision making, resulting in higher levels of performance (Meising, 1982). At the analysis and synthesis levels of Bloom’s taxonomy, team cohesion is needed to figure out the entire scenario. A lack of belonging and trust limits participants to the analysis level of learning, at best. Finally, consider the driven, performance-oriented participant. On the surface, it seems that this student should be well-motivated to learn at the higher levels. Yet he/she is likely to be ego-centric, driven by a fear of failure (Gentry et al. 2007) and an avoidance of loss-of-face (Meece et al., 1988). With this need for esteem driving his/her behavior; this student is unable to project his evaluations upon the entire team’s decision making processes as he is exposed to possible embarrassment if the resulting outcome is less than optimal. The evaluation level of learning is impeded by an unsatisfied need for esteem. COGNITIVE BLOCKS TO LEARNING USING COMPLEX BUSINESS GAMES These unmet needs within the simulation participant himself clearly may limit cognitive learning, yet is often not within the purview of the instructor to satisfy these needs. However, another confounding variable that may be inhibiting cognitive learning is within the control of the instructor: the complexity of the game itself. In particular, this refers to “the player’s manual, the number of decisions programmed, the internal algorithms employed and the number of supplemental materials and reports generated by the game” (Wolfe, 1985). Early in the use of computerized business simulations, it was believed that simulations needed to have face validity; they had to have the look and feel of real businesses. This drive for realism naturally results in a high degree of complexity. Yet, the problem of simulation complexity was immediately apparent as Kibbee (1961) wrote that the object of business simulations was not to reproduce a business problem, but to supply some principles that could be used in resolving the highlight problems in the simulations. Kibbee explained: “What is far more important in most management games is verisimilitude: the degree to which the players feel the simulated situation is real.” In fact, Graham and Grey (1969) pointed out the simpler simulations with fewer decisions variables were more accepted by the participants but they did not relate this acceptance to the learning aspects of playing business games. Yet the problem of simulation designers insisting that their games must have a high degree of face validity with its concomitant need for a high degree of complexity affect business simulations to this day. Greenlaw and Wyman (1973) noted that the CARNEGIE TECH MANAGEMENT GAME, a total enterprise simulation, was extremely complex with 50 to 100 hours of preparation time and between 100 and 300 decisions required for each round of play. They reported that intelligence, personality and satisfaction were not related to game performance. Furthermore, as the studied game progressed participant interest decreased. These authors also discussed an early examination of the relationship between game complexity and learning using MANSYM. No significant differences between the group using a simple version and the group using a complex version were found. It may be that playing a complex game does result in some learning, but players of a simple game are able to learn just as much. If so, the simple game is to be preferred as it imposes a lesser burden on the parts of the student, the course instructor and the game designer. Furthermore, these authors detail research on the UCLA GAME NO. 3, which is less complex than the Carnegie Tech game. In this case, the differences in team ability clearly affected performance, satisfaction and learning. This game was more likely to be understood by the participants, thus allowing for this observation of the relationship between ability and performance. This paper was an early call for game designers and users to consider the cost/benefit of using large-scale business simulations. The development of complex games require more effort, the use of complex games in the classroom require a larger commitment on the part of the instructional staff, and the playing complex games require large amounts of time and effort on the part of the players. In Wolfe and Jackson’s (1989) study involving three person teams playing the complex game THE BUSINESS MANAGEMENT LABORATORY, the students were divided into five competitions with identical parameters. However in three out of the five competitions, a program change was inserted that caused each demand function to essentially eliminate the effects of advertising. The students were then asked to evaluate the “realism” of algorithmic aspects of the 192 | Developments in Business Simulation and Experiential Learning, Volume 36, 2009  game they had just played. Those who played the games with the very unreal response function were not different in their analysis of the algorithms than the students who played the unaltered competitions. It was concluded that “players were unable to detect the error and the error’s presence had no impact upon the quality of their economic performance” (Wolfe and Jackson, 1989). This finding is staggering as it indicates that the players in complex games cannot determine important and critical relationships between demand and primary demand determining independent variables. Furthermore, this lack of understanding makes no difference upon the quality of the team’s economic performance! Could these students have possibly learned that ignoring advertising would not have any impact on the performance of their future businesses? Anderson and Lawton (1992) conducted a study to assess the relationship between financial performance on a business simulation exercise and various other measures of student learning. They used 12 decision rounds of MICROMATIC: A MANAGEMENT SIMULATION, a game requiring a total of sixty-six decisions per round in the areas of production, marketing, finance, and accounting. Little or no relationship was found between the performance score and the other measurements used to assess student learning. Hall and Cox (1994) challenged the assumption that educationally-effective business simulations needed to be complex. These authors tested a set of eleven simulations for the relationship between adult learning and the simulations’ complexity and duration. Their findings were that game complexity was not a predictor of learning. Wolfe (1985) stated that “complex games may teach more but they place dysfunctional burdens on the teaching system’s other components.” Students playing total enterprise games need a lot of explanation, not only as to the intricacies of how business functions interact but also as to how to interpret the results of their decisions and even what research questions are possible to ask. Before any game can be a real learning experience, all the participants need to have some degree of commonality in understanding the simulation and its simulated environment. In highly complex games, this understanding is difficult for even the best students. Currently, the technology has improved to the point that many total enterprise simulations have been converted to run over the Internet, yet this seems to just have ratcheted-up the level of complexity for the player even more. This drive for increasing complexity has not been followed by all game designers, however. Frazer (1980) wrote that game complexity was being driven by “a feeling that gaming isn’t quite respectable…” He suggested that game designers should “….take a realistic situation and simplify it greatly to make the situation amenable to meaningful analysis by whatever level of student the game is designed for. This will result in good analysis and a meaningful decision-making experience for the students, rather than merely subjecting them to a mass of data about which they can only guess at good answers.” He clearly understood that the quest for realism was impossible and generally without merit. If one believes that decision making and strategic choices are only a matter of logic and knowledge learned prior to game- play, then complex games should reward those participants that are the best prepared and there should be a relationship between prior grades or other indicators of prior learning and game results. But, little evidence exists of this relationship. It could be the subject of a research study where subjects were selected from executive programs, elite MBA programs that require a lot of experience as a prerequisite for acceptance, less selective MBA programs, and undergraduates. Most business games begin by having all firms have identical resources and having made identical decisions in period “0.” When this occurs, there is no information on how much to change prices (or product quality, promotions, the number of sales people, etc.) in order to increase profitability in the ensuing round. The analysis of the information provided by the firm’s internal and external reports which then supplies the means to understand and develop the best decisions and strategies is impossible. If participants are attempting to make their decisions based upon the data provided by playing the game, they cannot do it. Thus, the decisions used to make decisions and select strategies must be made by “rules of thumb” or based upon conjecture or even random guessing; this is not what was intended by the simulation designer or the instructor using the simulation. Many large-scale simulation games suggest that the participants play one to three “practice” rounds in order to see how the environment impacts the outcomes. However, often the participants are too lost to make decisions that will help them explore their environment before they have to start playing the game for “real.” This concept that some students “never have a clue is” widespread in ABSEL discussions but rarely described in its published proceedings. In an ABSEL paper Krishnamoorthy, Markulis and Strang (1987) described the value of student journals when attempting to understand why students made specific decisions. They described one finding as follows: we read entries such as: “We really did not understand why stock price or ROI (or almost any dependent variable) was acting this way, so we went ahead and changed our price or promotion (or some other independent variable).”(Quote from page 118). This is one demonstration that students, when playing business games students do not automatically understand the underlying concepts that drive these simulations. David Jordan (1998), analyzed student performances in business simulations in a capstone course and reported “…it was apparent that students were having difficulty applying previously covered material.” He went on and reported that, “What was found was the inability of the student to use the data and analysis in making effective decisions. They could “crunch the numbers,” but did not understand how to use the results” (Both quotes from page 232). Thus, the students learning when playing a business simulation was inhibited by the lack of fundamental understanding of previously taught materials. Ken Goosen (1998) reported that, “When students reach senior status and are taking their capstone courses, a common complaint is that they do not understand or have the ability to prepare cash flow statements.” (Quote from page 144) Cash flow statements are critical when playing business simulations. This same lack of understanding was reported by Wellington, Faria and Whiteley (1998). They wrote, “However, when the specific strategy variables were examined for the push and pull industries and competitor and environment industries, the findings indicated that the participants did not understand the specific nature of the marketplace environment in which they were competing” (quote from page 251). In a paper on groupthink, Jan Edman (2006) reported that the students neither 193 | Developments in Business Simulation and Experiential Learning, Volume 36, 2009  learned from their play of a simulation nor learned from the reports generated by the game when playing practice rounds. This lack of learning noticed because “the participants were committed to decisions similar to the initial decisions of their firms regardless of the performances of their firms. One explanation for this is that participants in firms with lower performance did not understand what decisions gave higher performance in the game” (Quote from page 282) It is not only the students that may not know or remember facts, techniques and methodologies, but the game designers may not understand how students learn. In an article on E- learning Edward Sawyer (2004) reported “…many content developers do not understand how people learn, measures of effectiveness are based on such easy to gather data as student throughput” (Quote from page 17). This same statement could be said of business game designers. A major trap of logic is occurring in the design of complex business games and it is shared by those that select business games for the classroom, simply put: “Mine is bigger than yours!” Game designers put more variables and more critical conditions in their business games, just to prove that their game is more realistic or more elaborate than other games. They are not considering how the participants can simultaneously consider all the nuances of the problem. Faculty members select the games to use in their classrooms based upon the desire to use the most sophisticated model possible as it indicates the knowledge of the professor, but not necessarily the most appropriate level of knowledge of the game participants. Many of the participants in complex business simulations are bounded in their Bloom’s learning capabilities by their unmet Maslow’s needs, the students are not up to the tasks being asked of them. Some are. This may result in a few students accomplishing great results, but most participants being confused or even misled. Allan Patz (1999, 2000, and 2001) and Teach and Patel (2007) reported that the capable students finish each round in the lead, decision period after decision period. But these students are starting out capable, what about the rest of the class? Education is not meant for those who already know the material and how to apply it. If one believes that an enterprise is best run by understanding and reacting to the forces that impact it, then there is a serious design problem within complex games. COGNITIVE LEARNING THEORY AND ITS IMPACT ON GAME DESIGN Cognitive learning theory focuses on how people learn and remember the things presented to them. It is concerned with the cognitive load that the presented materials place upon the learning process. Teach and Murff (2008) referred to the interaction between game design and game complexity when comparing simple versus complex games and reported; The underlying problem of complex games may lie in the limitations of the human mind. Miller (1956) reported on his research on the fundamental abilities of humans to process information. He noted that “seven simultaneous concepts (plus or minus 2) was the general limit for human comprehension.” (Quote from page 206) Compare this magical number seven to the number of decisions and interactions in large, complex total enterprise simulations being used in classrooms and training sessions. While the number of decisions alone does not necessarily define the complexity of a game, it provides a hint at the level of complexity; the variable interactions certainly provided additional strong clues to a game’s complexity. These authors, using this theory, would conclude that these complex games are beyond the scope of learning for the participants. In business games, the data are often presented in compact, easily recognized forms such as balance sheets and income statements as well as other reports. Decision analysis and decision making tools are often either recommended or provided. Instructors typically help their students by giving tutorial sessions in EXCEL or other spreadsheet tools. Why? These tools reduce the cognitive load during the learning process and to allow students to concentrate on the learning itself (Jonassen 1991). Jonassen goes on and claims that this reduced cognitive load allows students to learn by “experimentation and discovery.” However, there exists a “cognitive anomaly” in the way business games are played. Business students are often naïve; their ill-structured knowledge about how businesses really work and what one needs to know often conflicts with the data provided. In business simulations, the data are often presented in a whole host of tables and sometimes graphics because the provided data are in a form that those more experienced in businesses (and the game designers) understand. Students often fail to recognize clues of impending disaster and games are usually written to prevent disasters. For instance, most games have automatic loan features to prevent bankruptcy. There are learning costs for these preventative measures used to keep all of the students in the simulation until the end of the rounds. Sometimes students make outlandish decisions that anyone with more knowledge in how businesses work would never do. Numerous authors (Teach 1987, Fritzsche and Cotter 1990, Teach 1992, Washbush 1992) have reported that, on occasion, students drastically alter their decision process near the last rounds of simulation/game playing. This is referred to as “End- Play.” Sometimes price decisions are drastically changed in the very late rounds of their completion. One of the causes could be a complete lack of understanding of the concept of price elasticity. Another cause might be that the team thinks the computer program would not react quickly to their changes and they could gain by outsmarting the software and a third plausible reason could be that one or more of the teams want to intestinally wreck the simulation. While most students playing business simulations have taken accounting, they may not understand the accounting concepts very well. See (Huston, d’Ouville and Willis 1982, Aaidar-Sauaia 2004, Hergeth and Jones 2003).   White (1981) was concerned about the shortage of managerial skills of which accounting skills were but one. Consequently, there exists a dissonance between the students’ knowledge and the highly structured representations of the data needed to run the simulated business. Aidar-Sauaia (2001) wrote that “traditional measures of financial accounting may be quite restricted when results beyond these such as continued improvement and innovation are sought.” Quote from page 210. (Albacete and Van Lehn’s (2000a, 2000b) work with students studying physics, under circumstances that could be compared with students playing complex total enterprise games, found that a cognitive anomaly exists between “the naïve students’ ill structured knowledge and 194 | Developments in Business Simulation and Experiential Learning, Volume 36, 2009  the highly structured knowledge of experts.” (Quote from page 40) Alkhalifa (2005) reported that “when the cognitive level of the material presented to learners is high and the materials are more complex, then a retardation effect occurs to learning, while a lower cognitive level requirement achieves better results with all types of materials.” (Quote from page 40) One of Alhalifa’s concerns was that the designer of a teaching system should be knowledgeable of the concept of “cognitive load,” defined as the amount of cognitive processing required to perform an operation. Alkhalifa went on and derived seven main areas that a designer of teaching systems should consider: 1. Perception and recognition 2. Attention and memory 3. Mental representation of concepts 4. Natural language comprehension and generalization 5. Reasoning and deduction 6. Cognition and emotion 7. Cognitive learner differences These seven main areas should apply to the design of business games as well as physics lessons. The game designer needs to consider how high a cognitive load is being placed upon the students playing the simulation when the student is determining what information from the reports provided by the game is relevant to making a good decision in each of the rounds. This is a much more difficult task for the designer than just adding a few more variables to make the simulation “bigger than anyone else’s.” Yet, it is the right task for the designer who is truly interested in the learning attained by the students. Consider the following extended quotation from Jackson (2002): “To be able to unlock the thoughts of authors of texts used in the various disciplines so meaning can be constructed; students have to be guided to shift their teenage perspective (developed from their experiences) to the perspective of the author. This is an extremely difficult process for it involves so many factors: understanding the technical language the author is using; being comfortable with the organizational structure that is used for presentation of the discipline (e.g., science texts are structured different than history texts or literature); and finding personal experiences that match the frame of reference of the author. Is it any wonder that poor readers have such a difficult time constructing meaning from these texts written by experts in the discipline? As teachers of a discipline we are comfortable reading the texts we are using because years of experience with the presentation style of the discipline texts have molded our patterns of thinking, our discipline "lingo", and our frames of reference. To many students, so much of the language and context of the texts are alien and therefore meaningless. What also has to be recognized is that as teachers of a discipline, most of us have only one discipline to focus on. But our students have several disciplines they are trying to construct meaning about. To be competent learners in several disciplines requires real mental dexterity. Shifting from one discipline to another is like using a kaleidoscope. The student has to refocus for each discipline, and to get a clear picture; a different set of tools must fall into place. “Luckily the brain is like a muscle. When given the right exercises (mediated learning experiences) to develop the needed tools and a nutritional diet of opportunities to make conceptual connections that build the relevant context to comprehend the text, struggling readers become independent readers. Independent readers have the competence and confidence needed to morph from failing students to motivated high achievers.” CONCLUSIONS In short, those of us designing simulations and using them as instructional tools need to remember that students do not understand the material being presented in the way that we do. Increasing complexity is for experts, so that we perceive it at closer to reality. But learning is not about the designers or the teachers, it is about the students receiving and applying the information, both the high achievers and the failing students. Clearly the conclusions drawn from this paper is that more, no, much more research needs to be conducted in the “what is learned by playing business games” genre and more specifically the issue of the link between the complexity of a task and learning. Pearson Hunt (1964) emphasized this needed research by commenting that in order to learn how to teach, teachers need to learn how students learn. REFERENCES Aaidar Sauaia, Antonio Carlos (2001), “Evaluation of performance in business games: Financial approaches,” Developments in Business Simulation and Experiential Learning, Volume 28 pp 210 -214 Aaidar Sauaia, Antonio Carlos (2004), “Individual achievement versus team performance,” Developments in Business Simulation and Experiential Learning, Volume 31 pp 154 - 158 Albacete, P. L, and K. VanLehn (2000a), “The conceptual helper: An intelligent tutoring system for teaching fundamental physics concepts,” Lecture notes in Computer Science, Vol. 1839, pp 564-573 Albacete, P. L, and K. VanLehn (2000b), “Evaluating the effectiveness of a cognitive tutor for fundamental physics concepts,” In L. R. Gleitman and A. K. Joshi (Eds), Proceedings of the 2nd Annual Meeting of the Cognitive Science Society, Mahwash, NJ: Erlbaum, pp 25- 35 Alkhalifa, Eshaa M. (2005), “Effects of the cognitive level of thought on learning complex material,” Educational Technology and Society, Vol. 8, No, 2, pp 40 - 53 Anderson, P., Cannon, H., Malik, D. & Thavikulwat P. (1998), “Games as instruments of assessment: A framework for evaluation,” Developments in Business Simulation and Experiential Learning, Vol. 25, pp 31-37 Anderson, P. & Lawton, L. (1992), “The relationship between financial performance and other measures of Learning on a Simulation Exercise,” Simulation & Gaming, Vol. 23, No. 4 pp326-340 195 | Developments in Business Simulation and Experiential Learning, Volume 36, 2009  Bane, K. (2004), “Avoiding catastrophe: The role of individual accountability in team effectiveness,” Developments in Business Simulation and Experiential Learning, 31, pp 130- 131 Bloom, B., Englehart, M., Furst, E., Hill, W. & Krathwohl, D. (1956), Taxonomy of Educational Objectives: The Classification of Education Goals, New York: David McKay Company, Inc Buenger, C., Forte, M., Boozer, R. & Maddox, E. (2007), “A study of the applicability of the perceptions of organizational politics scale (POPS) for use in the university classroom,” Developments in Business Simulation and Experiential Learning, 34, pp 294-301 Burns, J. & Burns, A. (1990), “An exposition of Guilford’s SI model as a means of diagnosing and generating pedagogical strategies in collegiate business education,” Developments in Business Simulation & Experiential Exercises, 17, pp34- 37 Cannon, H. & Feinstein, A. (2005), “Bloom and beyond Bloom: The revised taxonomy to develop experiential learning strategies,” Developments in Business Simulations and Experiential Learning, 32, pp348-356 Cannon, H. & Feinstein, A. (2007), “The canary principle: An alternative model for coaching an on-line discussion environment,” Developments in Business Simulation and Experiential Learning, 34, pp 266-275 Curcio, G., Ferrara, M. & Genera, L. (2006), “Sleep loss, learning capacity and academic performance,” Sleep Medicine Reviews, 10, pp 323-337 Druskat, V. and Kayes, D. (2002) “Learning Versus Performance in Short Term Project Teams,” Small Groups Research, 3, pp 328-353 Duck, J. (2006), “Making the connection: Improving virtual team performance through behavioral assessment profiling and behavioral cues,” Developments in Business Simulation and Experiential Learning, 33, pp 358-359 Edman, Jan (2006), “Group composition and groupthink in a business game,” Developments in Business Simulation and Experiential Learning, Volume 33, pp 278 - 283 Feinstein, A. (2004), “A model for evaluating online instruction,” Developments in Business Simulation and Experiential Learning, Vol. 31, pp 32-39 Frazer, R. (1978), “Educational values of simulation gaming,” Exploring Experiential Learning: Simulations and Experiential Exercises, 5, pp 269-275 Frazer, R. (1980). Some issues on game design. Experiential Learning Enters the 80s, 7, pp 184-186. Fritzsche, David J. and Richard V Cotter (1990), “Guidelines for administering business games, Chapter 6 in Guide to Business Gaming and Experiential Learning, Gentry, J., Dickinson, J., Burns, A., McGinnis, L. and Park, J. (2007), “The role of learning versus performance orientations when reacting to negative outcomes in simulation games: Further insights,” Developments in Business Simulation and Experiential Learning, Vol. 34, pp 4-10 Greenlaw, P. and Wyman, F. (1973), “The teaching effectiveness of games in collegiate business courses,” Simulations and Games, Vol. 4, No. 3, pp 259-294 Goosen , Ken (1998), “Cash Flow Statements: Are they Important in Business Simulations?,” Developments in Business Simulation and Experiential Learning, Volume 25, pp 244 - 245 Hall, J. & Cox, B. (1994), “Complexity: is it really that simple,” Developments in Business Simulation & Experiential Exercises, Vol. 21, pp 30-34 Hergeth, Helmut D. and Michelle R. Jones (2003), “Board games and teaching textile marketing and finance,” Developments in Business Simulation and Experiential Learning, Volume 30, pp126 - 128 Hergeth, H. (2007), “Team behavior and team success: Results from a board game simulation,” Developments in Business Simulation and Experiential Learning, Vol. 34, pp 190-196. Howard, B., P. Markulis, D. Strang, and J. Wixom, 2006), “Simulations and experiential exercises: Do they result in learning? Have we figured it out yet?” Developments in Business Simulation & Experiential Exercises, Volume 33 pp 100 - 106 Hunt, Pearson (1964), “A Professor Looks at Teaching and Himself,” Harvard Business School Bulletin, January-February 1964, pp. 1-5. Huston, C. Richard, Edmond L. d’Ouville and T. Hillman Willis (1982), “A pedagogical approach to business gaming for the computer student,” Developments in Business Simulation & Experiential Exercises, Volume 9 pp 153 - 155 Jackson, Yvette (2002), “Comprehension and discipline literacy,” New Horizons for Learning, (December) A quarterly E-Journal found at www.newhorizons.org Jonassen, D.H. (1991), “Objectivism vs. constructivism: Do we need a new philosophical paradigm shift?” Educational Technology: Research &and Development, Vol. 30, No. 3, pps 5-14. Jordan, David (1998), “An integrated business instructional system,” Developments in Business Simulation and Experiential Learning, Volume 25, pp 232 - 233 Kayes, A., Kayes, C. & Kolb, D. (2005), “Experiential Learning in Teams,” Simulation & Gaming, 36, 330-354 Kallás, David and Antonio Carlos Aidar Sauaia (2006), “Balanced scorecard implementation for strategy management: Variations of manager opinion in real and simulated” Developments in Business Simulation and Experiential Learning, Volume 33, pages 197 - 203 Kayes, A., Kayes, C. & Kolb, D. (2005), “Experiential Learning in Teams,” Simulation & Gaming, Vol. 36, No. 3, pp330- 354 Kibbee, J. (1961). “Model building for management games,” Simulation and Gaming: a symposium, 8-15. Krishnamoorthy, Ganesh, Peter M. Markulis and Daniel R. Strang (1977), “Meaningful notebooks and enhanced learning using a totally menu driven data base and decision support system (DSS) program, Developments in Business Simulation & Experiential Exercises, Volume 14, pp 118 - 122 Little, Milton T.M. Michael D. Scanlon, Monique T. Barakat, Theo D. Palmer, Robert M. Sapolsky, and H. Craig Heller (2005), “Sleep restriction suppresses neurogenesis induced by hippocampus-dependent learning," Journal of Neurophysiology, Val 94, pp 4224 - 4233 Maslov, A. (1943), “A Theory of Human Motivation,” Psychological Review, 50, pp 370-396 Mayer, R., Davis, J. and Schoorman, F. (1995), “An integrative model of organizational trust,” Academy of Management Review, 20, pp 709-734 196 | Developments in Business Simulation and Experiential Learning, Volume 36, 2009  McKenny, J. & Dill, W. (1966), “Influences on learning in simulation games,” American Behavior Scientist, 10, pp 28- 32 Meece, J., Blumenfeld, P. & Hoyle, R. (1988), “Students’ Goal Orientations and Cognitive Engagement in Classroom Activities,” Journal of Educational Psychology, 80, pp 514- 523 Meising, P. (1982), “Qualitative determinants of team performance in a simulation game,” Developments in Business Simulation & Experiential Exercises, Vol. 9, No. 2, pp 228-231 Miller, George A. (1956), “The magical number seven, plus or minus two: Some limits on our capacity for processing information,” The Psychological Review, Vol. 63, No. 2, pp 81 - 97 Motorola University (1996), “Creating Mindware for the 21st Century,” Corporate University Xchange, 2, pp 212-219 Mujtaba, B. and Kennedy, J. (2005), “Facilitating through collaborative reflections to accommodate diverse learning styles for long term retention,” Developments in Business Simulations and Experiential Learning, Viol.32, pp 224-232 Norris, D. and Niebuhr, R. (1980), “Group Variables and Gaming Success,” Simulation and Games, 11, 301-312. Patz, Alan, L. (1999), “Overall dominance in total enterprise simulation performance,” Develops in Business Simulation and Experiential Learning Vol. 26, pp 115- 116 Patz, Alan, L. (2000), “One more time: Overall dominance in total enterprise simulation performance,” Develops in Business Simulation and Experiential Learning Vol. 27, pp254- 258 Patz, Alan, I. (2001), “Total enterprise simulation winners and losers: A preliminary study,” Develops in Business Simulation and Experiential Learning Vol. 28, pp192- 195 Robinson, S. (1996), “Trust and the breach of the psychological contract,” Administrative Science Quarterly, Vol.41, pp 574-599 Rogers, C.R. & Freiberg, H.J. (1994), Freedom to Learn (3rd Ed), Columbus, OH: Merrill/Macmillan Sawyer, Edward A.(2004), “Economic evolution, human capital investment and adult distributed electronic learning: A literature review,” Developments in Business Simulation and Experiential Learning, Volume 31, pp 14 - 21 Schumann, P., Anderson, P., Scott, T., & Lawton, L. (2001), “Framework for Evaluating Simulations as Educational Tools,” Developments in Business Simulation and Experimental Learning, vol. 28, pp 215-220 Shubik, M. (1972), “On the scope of gaming,” Management Science, 18(5), pp 20-36 Sujan, H., Weitz, B. & Kirmalya, K. (1994), “Learning Orientation, Working Smart, and Effective Selling,” Journal of Marketing, 58, pp 39-52 Teach, Richard (1990) "Profits: The False Profit in Business Games," Simulation and Gaming: An Interdisciplinary Journal of Theory, Practice and Research. Vol. 21, No. 1, 1990, pages 12-26 Teach, Richard, (1992) “Using forecasting accuracy as a measure of success in business simulations, Developments in Business Simulation and Experimental Learning Vol. 3, pp Volume 19, pp 103 to 107. Teach, Richard and Vishal Patel (2007), “Assessing participant learning in a business,” Developments in Business Simulation & Experiential Exercises, Vol. 33, pp 76 - 85 Teach, Richard and Elizabeth Murff (2008), “Are the business simulations we play too complex?” Developments in Business Simulation and Experimental Learning Vol. 35, pp 205-211 Washbush, John B. (1992), “Personality characteristics and group performance in total enterprise simulations”, Developments in Business Simulation & Experiential Exercises, Volume 19, 1992 pages 176 – 179 Wellington, W., & Faria, A. (1995), “Are good simulation performers consistently good?,” Developments in Business Simulation & Experiential Exercises, Vol. 22, pp 5-11 Wellington, William, A.J. Faria, and T. Rick Whiteley (1998), “Holistic cognitive strategy in a computer-based marketing simulation game: An investigation of attitudes towards the decision making process,” Developments in Business Simulation and Experiential Learning, Volume 25, pp 246 - 252 White George H. (1981), “Meeting the managerial skill shortage – Is academia up to it?”, Developments in Business Simulation & Experiential Exercises, Volume 8 pp 210 - 213 Wolfe, J.(1985), “The teaching effectiveness of games in collegiate business courses: a 1973-1983 update,” Simulations and Games, Vol. 16, No. , pp 251 – 288 Wolfe, J. & Jackson, R. (1989), “An investigation of the need for valid business game algorithms,” Development on Business Simulations and Experiential Exercises, Vol. 16, pp 31-36 197 | Developments in Business Simulation and Experiential Learning, Volume 36, 2009  Table of Contents Volume 36, 2009 Protecting Academic Integrity: Student Assessment In The Online Environment Pedagogical Shift: Teaching Report Writing For Accountants Online Increasing Student Learning In An Investment Management Course Through The Innovative Use Of Experiential Learning Pedagogy A Moral Development Unit For Business Courses Dividing Up Grandma’s Things: A Multifaceted Exercise In Critical Thinking Application Of Haekel’s Thesis To ABSEL Development Linking Stories With On-Line Threaded Discussions For Critical Thinking In Management Curriculum Why Have We Neglected Vicarious Experiential Learning? A Triadic Multi-Disciplinary Approach To Enhancing The Efficacy Of Experiential Learning Movement Toward Increased Student Roles In The Design Of Experiential Exercises Ready-Mix Concrete Company: An Experiential Exercise In Management Theory Using Portfolio Theory In A General Management Simulation An Experimental Analysis Of Advertising Strategies And Advertising The Relationship Between Goal Orientation And Simulation Performance With Attitude Change And Perceived Learning Beyond The Profitable-Product Death Spiral: Managing Product Mix In An Environment Of Constrained Resources Online Marketing Control With The Strategic Business Unit Analysis Package Beat The Market Game Creating Decision Support Systems In Business Simulation Games The Use Of Computer-Assisted, Interactive Role-Play Simulation In China Asia Marketing: An International Business Game An Empirical Test Of “Behavioral Immersion” In Experiential Learning Existing And Emerging Business Simulation-Game Design Movements Computerized Business Simulations: A Systems Dynamics Process Service Launch Evaluating Business Plans In A Simulation Environment Teaching Software Development By Means Of A Classroom Game: The Software Development Game Group Decision Experiments Using Business Game - Problem Solving With Conflict Luna: A Role Play Game For Learning Incoterms 2000 Entrepreneurship: A Game Of Risk And Reward Phase I -- The Search For Opportunity The Wee Game: A Pre-Game Learning Inhibitors In Business Simulations And Games New Version Of An Old Simulation Helps Build A Perfect Capstone Course Project Competitor: A Simulation Game For Project Management With 2 Models And 2 Modes Mitigating The Winner’s Curse In The Auction Market Of A Computer-Assisted Business Gaming Simulation Evaluation Of Collaborative Filtering By Agent-Based Simulation Considering Market Environment Marketing Simulation Game Decision Making Experience And Its Impact On Indecisiveness Among Introductory Marketing Students Individual And Organizational Learning In A Top Management Game In Pursuit Of Stockholder Value: Reinforcing Core Concepts In A Business Strategy Simulation With A “Shadow” Stock Market Competition The Simplicity Paradox: Another Look At Complexity In Design Of Simulations And Experiential Exercises In Search Of The Ethnocentric Consumer: Experiencing “Laddering” Research In International Advertising The Paradise Islands Revisited: Trouble In Paradise Developing And Assessing Student Information Literacy Competency Do First Mover Advantages Exist In Competitive Board Games: The Importance Of Zugzwang The Natural Debriefing Approach: A Case In A Simple Business Game Pursued For Perfect Communications New Product Development Simulation Dominance In On Line Business Games Competitions Cooperative Business Game With Framing Effect Experiential Exercises In The Online Environment Enhancing Web-Based Simulations With Game Elements For Increased Engagement The Ginseng Game Experience Business Using a Simple Business Simulation