Business Simulations: Reality .......... And Beyond! Page 84 - Developments in Business Simulation and Experiential Learning, volume 42, 2015 ABSTRACT The importance of reality is often cited in the literature and promotional materials as essential to quality learning from business simulations. But, commonly, these assertions do not define reality and do not explain why it is good. Two definitions of quality for business simulation are discussed - one measures is how accurately the simulation replicates the real world (reality) and the second how well the business simulation improves real-world performance. Measuring simulation effectiveness is reviewed and two measures of simulation efficiency are proposed. Effectiveness and efficiency are explored in terms of how meta-composition impacts cognitive processing and cognitive load. Meta-compositional elements are the design aspects that are independent of simulation subject and take design beyond reality! The discussion is illustrated using an actual simulation. Key Words: Meta-Composition, Reality, External Validity, Psychological Fidelity, Simulation Effectiveness, Simulation Efficiency INTRODUCTION Dittrich (1977) investigate realism but did not define what he meant by it. Likewise Frazer discusses the desirability of realism (1980) and "a great temptation endeavor to approximate reality" (1982). but did not define realism or reality explicitly. Ben-Zvi and Carton (2007) described a business game (simulation) as "highly realistic" but did not explain what they meant by realistic. Cadotte (2005) cites realism "as his number one goal" but again does not define what he means by realism. These are just a few examples of the assertion that realism is good where realism is not defined. Because of this and because there are two possible definitions realism - External Validity and Psychological Fidelity - it behooves to define these. From a design viewpoint a key measure of simulation quality is External Validity how "the simulation model represents actual external phenomena" (Cook and Campbell, 1979) and, by implication, that providing real-world experience is central to a business simulation. Feinstein and Cannon (2001) explore the fidelity and validity of business simulations describing fidelity as "the level of realism that a simulation presents to the learner". Decker and Adler (1987) state that "..... all have the common objective of making their model as realistic as possible" and Chiesel (1979) states that “Ideally, all gaming techniques strive to obtain a 100% realistic copy of the objective system being simulated”. An alternate definition of simulation quality is Psychological Fidelity where the " training environment [simulation] prompts the essential underlying psychological processes relevant to key performance in the real-world setting" (Kozlowski and DeShon, 2004). Here the quality of the simulation is based on its ability to develop and challenge metacognitive business skills. These two definitions represent two viewpoints. External Validity suggests that quality derives from replicating the real- world experience as exactly as possible. In contrast, Psychological Validity suggests that quality derives from the cognition caused by the simulation, the relevance of this to the participants' work and how this improves on-the-job performance. This leads to a proposition that the primary focus of External Validity is the simulation models (content) and Psychological Validity focuses on the way participants interact with the simulation model (decisions and results) and the issues explored by the business simulation. (process) - a content- process divide that was explored by Gentry et al (1992). A BEYOND REALITY SIMULATION An actual business simulation is used to illustrate and explain the beyond reality design conceit, its issues and meta- compositional elements. The simulation was designed for the North American division of a leading manufacturer of power and control equipment that sell through independent electrical distributors. The company wanted to improve their sales engineers knowledge about their customers and their customers' industry. They felt that their sales people needed to know about their customers' problems and the issues they face as this was particularly important as they sold to businesses where the product they sold was critical to the customers' success. There was an existing Distribution Business Simulation that could be customized but this required major customization to incorporate all the issues that needed to be explored. The simulation needed to be run by company staff and last no more than a day. The company staff requirement would not be a problem and in fact a benefit (as, in the words of the client "training by company employees was more about local market knowledge than cost" - something that ensured transfer. The existing simulation lasted 6 hours 45 minutes but the additional issues would increase the number of decisions by 67% and the duration could only be increased to 8 hours 30 minutes (26%). This mismatch between complexity increase and duration meant that the design presented problems - problems that were addressed through beyond reality meta-composition. BUSINESS SIMULATIONS: REALITY .......... AND BEYOND! Jeremy J. S. B. Hall Hall Marketing jeremyhall@simulations.co.uk Page 85 - Developments in Business Simulation and Experiential Learning, volume 42, 2015 BEYOND REALITY, EFFECTIVENESS & EFFICIENCY Thavikulwat (2004) suggests that, commonly, the design purpose for a business simulation is reality and purpose is discovered later. This conceit is supported by the many papers in the BKL library that mention External Validity (more than 120 papers) and multiple papers about simulation model design. However, one can posit that a design focus on replicating reality is ineffective and inefficient and should be replaced with a Beyond Reality conceit. The Beyond Reality conceit is concerned with how a business simulation can be designed to enhance cognitive learning (effectiveness) and shorten duration (minimize cognitive load and hence improve efficiency). SIMULATION EFFECTIVENESS Simulation Effectiveness represents an attempt to measure the extent to which the simulation meets learning needs. One can take the view that a simulation is effective if it is realistic (Content Effectiveness) or that it is effective if it develops business skills, challenges understanding and reforms mental schema (Cognitive Effectiveness) Where the business simulation is used to train business people, Cognitive Effectiveness is arguably the best measure. Cognitive Effectiveness can be measured from course reviews or Kirkpatrick's training evaluation model (Kirkpatrick & Kirkpatrick, 1994). Course reviews focus on the learners' perceptions of benefit and learning and are reasonable based on the learners experience and knowledge and their need for intrinsic payoff (Knowles et al, 1998). In other words during the simulation adult learners are asking themselves "will this learning help me do my job better?" and these perceptions will be articulated on the course review form. Kirkpatrick's model extends this to evaluate post program changes to behavior, transfer and actual business results (Schumann et al, 2001). These Simulation Effectiveness measures provides a way of assessing the way the simulation prompted the essential underlying psychological processes relevant to key performance in the real-world setting (Psychological Fidelity). SIMULATION EFFICIENCY Simulation Efficiency consists of Focus Efficiency and Duration Efficiency. Focus Efficiency can be explored from two overlapping sets - learning needs and the business simulation (Figure 1) where the simulation is areas A+B and learning needs are B+C. Area B represents the decisions, models and results built into the simulation to provide relevant learning. Area A is irrelevant learning - the decisions, models and results that are included because they are real but not because they address required learning needs. Focus Efficiency measures the ability of the simulation to deliver relevant learning by measuring the relationship of B to A+B. Hall and Cox (1994) explored how the number of decisions are highly correlated with simulation duration and this means that a measure of focus efficiency can be expressed as the number of relevant decisions as a percentage of the total number of decisions. Not only does this measure help design focus effort but helps when choosing a simulation off-the-shelf. As the case study simulation was based on a distribution business simulation for specialist staff and junior management focus efficiency was high as it meant that the decisions were highly relevant. However, to make the simulation more relevant, one decision was removed and several added. Obviously, the required duration precluded exploring all the issues facing an electrical equipment distributor and some of the participants would have differing learning needs but, arguably, the simulation has a focus efficiency close to 100%. Duration Efficiency is a measure of how well the simulation packs learning into a budgeted duration. A possible measure of Duration Efficiency is the worst case duration forecast (based on the Hall and Cox (1994) formula) expressed as a percentage of the simulation's actual duration. The case study simulation involved 15 different decisions and, based on the Hall and Cox formula should have had a duration of 725 minutes. The actual duration was 510 minutes, leading to a Duration Efficiency of 142%. Duration Efficiency is influenced by Composition Efficiency (explained next) and Delivery Efficiency (where this is impacted by the user interface, facilitation, time-table etc). Duration Efficiency is important because of the way learning programs are becoming condensed and shortened (Austin & Gustafson, 2006). FIGURE 1 LEARNING AND SIMULATION SETS Page 86 - Developments in Business Simulation and Experiential Learning, volume 42, 2015 META-COMPOSITION Developing business simulations has a significant artistic component (Bellman et al, 1957; Thavikulwat, 2004 and Bott & Daalen, 2007). In the visual arts, composition is the structural arrangement of the visual elements distinct from the subject of the work of art. Dow (1913) describes it as an "approach to art through structure [that] is absolutely opposed to the time- honored approach through imitation". Later he states that "good drawing results from trained judgment not from making facsimiles". For simulations meta-composition relates to design elements independent of the situation modeled. Likewise it can be argued that good business simulations come from judgment rather than creating exact replicas. Meta-composition involves the purposeful design for cognitive processing (and through this Cognitive Effectiveness) and for cognitive load (and through this Composition Efficiency). It encompasses the learning journey, ambiguity management, simplification, stylization, structural relationships and reflection triggers. Although discussed separately these aspects are not independent of each other, rather they interact and collaborate. LEARNING JOURNEY Beyond the models is the way simulation progresses over time. Hall & Cox (1993) suggested a systems dynamics model that consisted of two dynamics - cognition and affection. Hall (2009) revisited this model to add a third dynamic - workload (cognitive load). These papers advocated a beyond reality design approach where the learning journey was consciously planned and consisted of a natural response (designing into the simulation) and a managed response (provided by the facilitator when the simulation is used). Natural response - the simulation's Temporal-Topical System (Hall, 2008) - is the way issues and challenges are purposefully designed into the simulation rather than just relying on the reality provided by the model. The design of the Temporal-Topical System provides a way to focus on different issues and cognitive inquiry as the simulation progresses. Figure 2 shows the way decisions were introduced in the Distribution Business Simulation and the cognitive inquiry associated with each. Percent Markup, Inventory Purchase and Marketing (Sales Support) decisions were made throughout the simulation but the other decisions were introduced progressively. This spread cognitive load and focused cognition as each new decision was introduced. Introducing topics as the simulation progresses improves Composition Efficiency as it reduces the total number of decisions made during the business simulation and hence the amount of cognitive processing. For the Distribution Business Simulation and using the Hall & Cox formula the phased introduction of decisions reduced the expected duration by just FIGURE 2 DECISION - TIMING & IMPACT Decisions Periods Cognitive Inquiry Percent Markup (per market) 1-8 Profit/Demand impact Inventory Purchases (per market) 1-8 Demand Forecasting impact Marketing (Sales Support) 1-8 Client Communication impact Staff Numbers 2-8 Resource forecasting impact Training Days 3-8 Resource improvement impact Number of Products 4-8 Client need - inventory impact Receivable Days 4-8 Balance Sheet impact on demand Electronic Linkage 5-8 Inventory system cost/asset impact Demo Equipment 5-8 Sales support demand impact Demo Room 5-8 Sales support demand impact Small Project Initiative 5-8 Supplier partnership impact FIGURE 3 AMBIGUITY SPECTRUM Too Little Too Much Learning Importance Time Page 87 - Developments in Business Simulation and Experiential Learning, volume 42, 2015 over three hours. It also impacts engagement as illustrated by feedback from the Distribution Business Simulation facilitators who stated "the continuous introduction of new ideas kept everyone interested ..... Throughout the training, there were never problems with people checking email, voicemail and so on. Most would voluntarily work through lunch on their (virtual) business". Further, working through lunch added to efficiency by shortening the scheduled classroom time by about an hour. The purposeful design of the learning journey distorts reality by addressing learning process issues to improve cognition, affection and workload. AMBIGUITY MANAGEMENT Ambiguity is important as it determines the amount and depth of cognition and exposes the links between decisions and results and thus is treated separately from simplification and stylization. In the real world the impact of decisions are ambiguous and this ambiguity is uncontrolled and for many decisions is high (figure 3). The level of ambiguity of decisions, their impact and the results needs to be established based on meta-cognitive needs, cognitive load and learning purposes. Hall (2008) suggested that because ambiguity directly impacts cognitive load (thinking time) the degree of ambiguity should be based on the learning importance of the decision or result. The Distribution Business Simulation was deterministic rather than stochastic and hence participants did not have to deal with the ambiguity caused by uncertainty. For example the amount purchased was the same as the amount delivered (suppliers did not miss deliveries, deliver the wrong materials or deliver at the wrong time). This move away from reality reduced the purchase decision ambiguity as when making their purchase decisions, participants only needed to forecast demand and take into account current inventory. Ambiguity design involves deciding the importance and contribution to learning and setting an appropriate level of ambiguity. SIMPLIFICATION The real world is messy, complex, confusing, chaotic, uncontrolled and uncontrollable. Simplification overcomes these problems and defines the extent to which the simulation (Hall, 2008) clearly incorporates relevant issues rather is merely a replica of the real world. For the Distribution Business Simulation, it was decided to remove the Accounts Payable decision as this was seen as irrelevant (or inappropriate). Another issue to explore was the range of equipment (number of different products) to offer. This would lead to participants discussing the trade-offs between working capital, cash flow, sales, profits and profitability. However, instead of deciding ranges on a market sector basis, participants decided it singly for the whole business. This simplification was done to reduce cognitive load and thus shorten duration. Making the range decision across the whole company (instead on a market-by- market basis) saved about an hour without impacting cognitive processing. Simplification involves a purposeful focus on learning needs and issues (Cognitive Effectiveness), removing irrelevant decisions (Focus Efficiency) and shortening duration (Composition Efficiency). Additionally, simplification reduces confusion resulting from an over complex simulation and this helps with engagement. STYLIZATION Stylization defines the extent to which the simulation model (Hall, 2008) moves in a purposeful away from behaving Price Marketing Product Availability Staff Availability Staff Skills Product Range Receivables Improvements Sales Demand FIGURE 4: STRUCTURAL RELATIONSHIPS Time Activity 08:00 - 08:20 Simulation Briefing 08:20 - 10:00 Familiarization & Preparation 10:00 Submit Decisions for Quarter 1 10:40 Submit Decisions for Quarter 2 10:40 Additional Decisions 11:20 Submit Decisions for Quarter 3 11:50 Submit Decisions for Quarter 4 11:50 - 12:00 Brief of additional Decisions 12:00 - 13:00 Lunch 13:50 - 14:30 Meetings with Bank 14:30 Submit Decisions for Quarter 5 15:00 Submit Decisions for Quarter 6 15:30 Submit Decisions for Quarter 7 16:00 Submit Decisions for Quarter 8 16:00 - 16:30 Prepare for Review 16:30 - 17:30 Simulation Review FIGURE 5 TIME TABLE Page 88 - Developments in Business Simulation and Experiential Learning, volume 42, 2015 exactly like the real world. For the Distribution Business Simulation a key client need was to reduce profitability to the levels of real-world distributors. Although it was possible to do this it was felt that low levels would be demotivating and, although initial profitability matched the industry norms, it was possible for participants to be grow profitability significantly. This was a stylization to ensure engagement that did not impact verisimilitude (the perception of reality). One participant who left to work for a distributor said "the class [simulation] helped him understand and prepare for the job and that he had real benefit understanding the business from the simulation approach".) Stylization helps cognition by clarifying (amplifying) the impact of decisions and having an appropriate level of challenge ensures engagement. Thus stylization improves Cognitive Effectiveness and Composition Efficiency STRUCTURAL RELATIONSHIPS Just as fine art composition involves the arrangement of visual elements, simulation meta-composition involves the structural relationships between the models, decisions and results (Hall, 2008). Structural design involves the purposeful arrangement of the ways decisions, the model and results relate to each other to clarify causal links and ensure appropriate cognitive processing. Over complex structures are difficult or impossible to unravel and hence reduce Composition Efficiency and Cognitive Effectiveness (as participants become confused and cannot unravel the impact of their decisions). Too simple structures lead to little cognitive processing and reduce Cognitive Effectiveness. For the Distribution Business Simulation the structural relationship that determined sales demand was complex and non-linear. Eight different factors (Figure 4) impacted sales and this meant that it might be difficult for participants to unravel the links between individual decisions and sales demand. This problem was overcome by phasing the introduction of the decisions that impacted the factors (Figure 2) with the first three (price, promotion and product availability) from the first period, staff availability becoming an issue from period 2 onward, staff skills from period 3 and so on. Also, qualitative feedback occurred when a decision or decisions began to have a major impact on results. Finally, facilitators had special reports that separated out the impact of individual factors that they could use to coach and challenge. In reality, qualitative feedback might be available but neither would decisions be phased nor would one be able to separate out impacts. Structural design led the client to observe that "Each decision needed to be accounted for by another to maximize impact. The company has been trying to teach thinking through the process for years – this class helped them understand". Structural design involves deciding which structure is appropriate to provide suitable cognition and cognitive load. REFLECTION TRIGGERS Central to the use of business simulations is the Kolb Experiential Learning Cycle (1984) and simulation design must accommodate this. Active Experimentation and Concrete Experiences are explicit aspects of reality (dialectic modes of grasping experience (Kolb et al, 2000)). Beyond this there is a need to incorporate Reflective Observation and Abstract Conceptualization (dialectic modes of transforming experience (Kolb et al, 2000)). Reflective Observation and Abstract Conceptualization are activities that are undertaken in the real world rarely. Of particular importance is reflection since as Moon (2004) states "We reflect in order to learn something, or we learn as we reflect". Elements of the learning journey, ambiguity, simplification, stylization and meta-composition trigger reflection. But beyond this, there are needs for reflection triggers and a suitable time- table. Reflection triggers are devices that cause participants to step back, discuss and review their actions, They may be triggered by the simulation, be pre-planned or initiated by the facilitator proactively. For example, during decision-making, unusual decisions (such as a very high price) can be flagged. Similarly, results can be analyzed and strengths and weaknesses reported. For the Distribution Business Simulation, reflection was triggered by comments from "staff", "bankers" and "customers" about the financial, marketing and operational situation to suggest issues that needed to be discussed. Secondly, the simulation produced special reports for the facilitators to allow them to question, challenge and coach participants proactively Besides the usual review and providing appropriate time for decision-making, about half-way though the simulation (Figure 5) participants had to meet with the bank to discuss borrowing needs. With the facilitators role-playing bankers this meeting forced participants to review their progress and articulate their plans. The timing of the bank meeting directly after lunch was Reality Factual Knowledge beyond Reality Business Ability Academic Education Adult Learning FIGURE 6 PURPOSE AND PARTICIPANTS SPECTRUM Page 89 - Developments in Business Simulation and Experiential Learning, volume 42, 2015 deliberate and as mentioned earlier "Most would voluntarily work through lunch on their (virtual) business". Purposefully design of reflection involves a clash between Cognitive Effectiveness (learning) and Composition Efficiency (duration) as providing reflection time increases duration. CONCLUSIONS There are several issues associated with the beyond reality conceit - the simulation's purpose and its participants; the measurement of Composition Efficiency and the expanding the knowledge of meta-composition. SIMULATION PURPOSE AND PARTICIPANTS A question is whether External Validity or Psychological Fidelity is the best goal. It is possible that the appropriateness depends on learning purpose and participants. Anderson and Lawton (2009) provide a list of the reasons for using business simulations. Many of the items on this list are concerned with gaining business knowledge (understanding terminology, concepts and principles, etc.). In contrast, Dulewicz (1982) provides an alternate competency assessment based list for business simulations that focus on key performance capabilities (analytical ability, helicopter ability, business sense, etc.). Further Haynie et al (2010) in the context of entrepreneurs, suggested the need for higher-order cognitive strategies for business leaders. Although there are overlaps between these two lists one can posit that they illustrate two different viewpoints - pedagogic (academic) education and andragogic (adult) learning. These differences suggest different fidelity/validity needs. For pedagogic (academic) education it is possible that External Validity is an appropriate measure of quality. But, to make business people more successful, Psychological Fidelity may be key. Arguably (and, perhaps, controversially), the two represent the ends of a spectrum (Figure 6) that extends from developing (factual content) knowledge to developing business ability (competencies) with academic education purpose towards the knowledge end and the adult learning purpose towards the business ability end. The position of a business simulation on this spectrum determines how one measures quality and this depends on it's purpose and participants. A business simulation designed to be used by business undergraduates is likely to need to replicate the real world (External Validity) and teach content. In contrast, a business simulation designed to be used to develop business leaders is likely to need to help them improve performance (Psychological Fidelity) - with a design focus on process and meta-composition. But what of a business simulation for executive MBAs? Such a simulation would probably lie in the middle of the spectrum and, consequentially require both External Validity and Psychological Fidelity - a content and process focus. As it is unlikely that purpose will be at one or the other end of the spectrum simulation, quality needs to be measured in terms of both External Validity and Psychological Fidelity. The relative weighting of these quality measures will depend on the purpose (academic education or business learning) and participants (students or business people). To the left of the mid-point the key measure would be External Validity with Psychological Fidelity secondary. To the right of the mid-point the key measure would be Psychological Fidelity with External Validity secondary. MEASURING COMPOSITION EFFICIENCY Here Composition Efficiency was measured based a formula from a paper from 1994 (more than two decades ago). A formula that does not take into account how cognitive processing changes if a decision is made repeatedly, the level of ambiguity, structural relationships, reflection triggers, etc. These are factors that are likely to an impact on cognitive load and thus on duration. But, it was possible to quantify the impact of the progressive introduction of decisions (learning journey) that changed duration from 725 minutes to 532 minutes a figure that is reasonably close to the actual duration of 510 minutes. This may be because of the way issues were compartmentalized and of similar importance (Figure 2). The Hall & Cox paper assumed that cognitive processing time for a decision remained reasonably constant throughout the simulation. With the Distribution Business Simulation this would be true for the purchase decision as each period participants would have to forecast demand and take into account inventory levels when deciding how much to purchase. However, other decisions (such as setting up a demo room) would require significant cognitive processing when first making the decision but virtually none subsequently. Thus cognitive processing time depends on the type of decision. Other factors (ambiguity etc.) are likely to impact cognitive processing equally throughout the simulation but differently for different decisions. It may be possible to empirically research how these impact duration and through this understand how to increase Composition Efficiency (reduce duration or pack more learning into the available time) without effecting Cognitive Effectiveness. That is to say to reduce cognitive load without impacting cognition (learning). Arguably this is important because of the need to reduce course durations and because it allows one to estimate duration while the simulation is being designed, rather than during piloting and early use. In the interim, forecasting duration is based on judgment, tacit knowledge and experience. META-COMPOSITION ISSUES The paper discussed several elements of simulation composition but there is a need to critique and add to these - expand the business simulation knowledge base to cover the elements of design that cause appropriate cognitive processing (and so improve Cognitive Effectiveness) and that impact cognitive load (and so impact Composition Efficiency). There is an opportunity for an in depth review of the simulation design literature and perhaps existing simulations for this critique and expansion. Poore (1903) suggests that "without composition, there can be no picture" and it can be argued that "without meta- composition there can be no simulation - low effectiveness and efficiency" - the elements that take design beyond reality!. Page 90 - Developments in Business Simulation and Experiential Learning, volume 42, 2015 REFERENCES Austin, Adrian M. & Leland Gustafson (2006) Impact on Course Length on Student Learning Journal of Economics and Finance Education Volume 5,Number 1 Anderson, Philip H. & Leigh Lawton (2009) Business simulations & cognitive learning: Developments, desires and future directions, Simulation & Gaming April 2009 Volume 40 no. 2 Bellman, Richard, Charles Clark, Cliff Craft, Don O. Malcolm & Franc Ricciardi (1957) On the construction of a multi-stage multi-person Business Game, The RAND Corporation. Ben-Zvi, Tal & Thomas C. Carton (2007) From Rhetoric to Reality: Business Games as Educational Tools INFORMS Transactions on Education 8(1) Bots, Pieter & Els van Daalen (2007) Functional Design of Games to support natural resource management policy development Simulation & Gaming Dec 2007, Volume 38 Chiesel, Newell E. (1979) The Dynamic Aspects of Interactive Gaming puts the Realism into Gaming, Developments in Business Simulation & Experiential Exercises, Volume 6 Cadotte, Ernest R (2005) Working to Enhance the Business Simulation Pedagogy: Goals, Strategy, Features and Benefits Marketing Instruction July 2005 Cook, T. D. & D. T. Campbell (1979) Quasi-experimentation: Design and analysis issues for field settings Houghton Miffin, Boston MA Decker, Ronald James LaBarre, & Thomas Adler (1987) The Exponential Logarithm Function as an Algorithm for Business Simulation Developments in Business Simulation and Experiential Exercises Volume 14 Dittrich, John E (1977) Realism in Business Games: A Three Game Comparison Developments in Business Simulation and Experiential Exercises Volume 3 Dow, Arthur Wesley (1913 COMPOSITION A series of exercises in art structure for the used of students and teachers Doubleday, Page & Company, New York Dulewicz, Victor (1982) The application of assessment centres Personnel Management, September 1982 Feinstein, Andrew Hale & Hugh M. Cannon (2001) Fidelity, Verifiability and Validity of Simulation,: Constructs for Evaluation Developments in Business Simulation and Experiential Learning, Volume 28 Frazer, J. Ronald (1980) Some Issues in Game Design Developments in Business Simulation and Experiential Learning, Volume 7 Frazer, J. Ronald (1982) Realism and Learning: The Evolution of a Management Game Developments in Business Simulation and Experiential Learning, Volume 9 Gentry, James W, Jeffrey J Stoltman & William W. Curtis (1992) What is it what we want students to learn: Process or Content Developments in Business Simulation and Experiential Exercises Volume 19 Hall, Jeremy & Benita Cox (1993) Computerised Management Games: the feedback process and servo-mechanism analogy, Simulation & Gaming Y earbook 1993 eds Fred Percival & Danny Saunders, Kogan Page London Hall, Jeremy J. S. B. & Benita M Cox (1994) Complexity is it really that simple, Systems Developments in Business Simulations and Experiential Exercises Volume 21 Hall, Jeremy J. S. B. (1994a) Computerised Tutor Support Systems: the tutor's role, needs and tasks, The Simulation & Gaming Yearbook Volume 2 eds. Roger Armstrong, Fred Percival & Danny Saunders Kogan Page London. Hall, Jeremy J. S. B. (2008) Corporate Cartooning: The Art of Computerized Business Simulation Design Developments in Business Simulation & Experiential Exercises, Volume 35, Hall, Jeremy J. S. B. (2009) Computerized Business Simulations: A Systems Dynamics Process Developments in Business Simulation & Experiential Exercises, Volume 36 Haynie, J. Michael, Dean Shepherd, Elaine Mosakowski & P. Christopher Earley (2010) A situational metacognitive model of the entrepreneurial midnset Journal of Business Venturing 25 Kirkpatrick, D.L. & J.D. Kirkpatrick (1994) Evaluating Training Programs, Berrett-Koehler Publishers, San Francisco, CA Knowles, M. S., Elwood F. Holton III & Richard A. Swanson (1998), The Adult Learner Butterworth-Heinemann Woburn, MA Kolb, David A. (1984) Experiential Learning: Experience as a source of learning and development Prentice Hall, Englewood Hills, NJ Kolb, David A, Richard E. Boyatzis & Charalampos Mainemelis (2000) Experiential Learning Theory: Previous Research and New Directions Perspectives on cognitive learning and thinking styles Lawrence Erlbaum, NJ Kozlowski, Steve W.J. & Richard P. DeShon (2004) A Psychological Fidelity Approach to Simulation-Based Training: Theory, Research and Principles in SCALED WORLDS: Development, Validation and Applications Ashgate Publishing Company, Aldershot. Moon, J (2003) PDP Working Paper 4: Reflections in Higher Education Learning http://www.heacademy.ac.uk/ resources/detail/id72_ Reflections_in_Higher_Education_Learning Poore, Henry Rankin (1903) Pictorial Composition and the Critical Judgement of Pictures G.P. Putnam's Sons, New York and London Schumann, Paul L., Philip H. Anderson, Timothy W. Scott & Leigh Lawton (2001) A Framework for Evaluating Simulations as Educational Tools, Business Simulations and Experiential Exercises Volume 21 Thavikulwat, Precha (2004) The architecture of computerized business gaming simulations Simulation & Gaming Volume 35 Number 2 June 2004 Papers published in Developments in Business Simulation & Experiential Exercises, are reprinted in the Bernie Keys Library, [Available from http://absel.org] http://www.heacademy.ac.uk/resources/detail/id72_%20Reflections_in_Higher_Education_Learning http://www.heacademy.ac.uk/resources/detail/id72_%20Reflections_in_Higher_Education_Learning http://www.heacademy.ac.uk/resources/detail/id72_%20Reflections_in_Higher_Education_Learning