COMPLEXITY: IS IT REALLY THAT SIMPLE Developments In Business Simulation & Experiential Exercises, Volume 21, 1994 30 COMPLEXITY: IS IT REALLY THAT SIMPLE Jeremy J. S. B. Hall, Hall Marketing. London Benita M. Cox, Imperial College, London ABSTRACT This paper challenges the assumption that complexity is necessary for educationally effective computerized business simulations by demonstrating, for a sample of proven simulations, there is a correlation between complexity and the duration of the simulation. It discusses complexity in terms of two mechanisms. The first mechanism gives rise to the assertions about complexity by suggesting that realism is a key determinant of educational effectiveness and that realism is produced through complexity. However, there is a second mechanism where the amount of cognitive processing performed by participants relates to the simulation’s complexity. In turn, the simulation’s duration relative to cognitive processing produces cognitive pressure that may lead to role overload. With role overload producing a negative influence on andragogic effectiveness. When combined, these mechanisms produce a peak in the function linking complexity with andragogic effectiveness and, the position of this peak is determined by the simulation’s duration. The peak in effectiveness is shown using data from several business simulations that have been used extensively on short courses by practicing executives. Finally the findings are discussed in terms of the design and use of business simulations for Executive Short Courses. INTRODUCTION The need for business simulations to be realistic and the link between realism and complexity has been widely discussed. While reviewing the validity and effectiveness of business simulations, Miller and Leroux- Demers (1992) state “Management simulations are valid pedagogical tools provided they are complex and realistic”. Later in the same paper it is stated “To conclude, complexity in management simulation was found to be a necessary ingredient for participants’ learning. Complexity i.e. the number of decision variables, and interactions between model parameters, is necessary to approximate business reality and provide the impression of realism” Thorne (1992) in an article entitled “New stimulus for those simulations” stated “Naturally any simplification, however advanced, will ultimately be a simplification of reality. But the latest, most sophisticated simulations can now recreate, through complex mathematical models, the dynamic effect of realism”. These comments reflect a commonly held view that complexity is “a good thing”. Yet, one benefit of using models to represent the real world is that, through the abstraction and simplification of reality, it is possible to focus on a specific problem. To plan a journey, a road map is far preferable to an aerial photograph though the latter is a far more accurate replica. It is the very complexity of the aerial photograph that is its weakness and it is the simplicity and focus of the road map that is its strength. Springer et al (1965), in the context of the general use of models by management, state “The power of a model in solving a problem comes precisely from its not corresponding to reality except in those details pertinent to the problem at hand”. Where simulation models are used for management education “the problem at hand” is meeting learning objectives. This view is reinforced by Norris (1986) who discusses the design conflict where enhanced realism (through increased complexity) leads to a peak in the teaching effectiveness -reality function and Nebenzahl (1984) suggests that a game is too complex if it does not permit identification of the impact of important decisions. Beer (1967) discusses model complexity in terms of “cones of resolution” suggesting “Clearly, the most economic way to do management science is to stay as high up the cone of resolution as one can”. Where simulation models are used in management education by practicing executives the main economic measure is time. The time spent by the executives in reaching the required learning objectives. This suggests that complexity must be considered in the context of duration. For short courses lasting a matter of days duration is a critical factor affecting the choice of whether to use a simulation or not. The time required for cognitive processing relates to complexity. If this time need differs from the time available (duration) then Role Overload can occur and the stress and strain produced have a negative effect (French & Caplan, 1972). Thus “Another major factor that affects the realism of the game is the time pressure on the participating teams. The time pressure must relate to the complexity of the model. “ (Hall 1971). The focus in the literature on striving for realism was questioned by Anderson (1987) who discussed “the counterpointing of “simple” and “more complex” and the inference that the latter are an improvement on the former”. Elgood (1984) reinforces this stating “Generally, it is safe to say that far more errors are made by game writers trying to be too clever than by keeping their material simple and elegant”. One must ask the question “Why are realism and complexity so often cited as a key design requirement?” and suggest that this emphasis is philosophical. This is supported by Kotler (1984 p731) who suggests that people with a research and development (scientific) background tend to emphasize intrinsic quality, functional features and, based on a pride in scientific curiosity and detachment, strive for technical perfection. This basic “product orientation” (Kotler p17) therefore focuses on internal product features that, by their nature, can be better controlled and rigorously evaluated. In contrast “marketing orientation” (Kotler p20) approaches design from a holistic customer satisfaction viewpoint. THE MECHANISMS A link between complexity and duration to andragogic effectiveness is suggested by the diagram shown in Figure 1. This diagram shows two mechanisms - REALISM and COGNITIVE PRESSURE. Although it may not be possible to separate and quantify these mechanisms it is possible to discuss their shapes. REALISM - as illustrated in the introduction, it is generally held, that effectiveness is positively correlated with realism and that realism is a function of complexity. The shape of the complexity-realism function can be discerned from the multi- dimensionality of business models. For instance a general management simulation will involve multiple markets, products, operation levels etc. each representing a different dimension. Therefore moving from a simple model to a more realistic one involves a power function (as illustrated in Figure 2a). By impli- FIGURE 1 COMLEXITY/DURATION/EFFECTIVENESS COMPLEXITY REALISM COGNITIVE PROCESS DURATION COGNITIVE PRESSURE ANDRAGOGIC EFFECTIVNESS Developments In Business Simulation & Experiential Exercises, Volume 21, 1995 31 cation, the strongly held view of the link between realism and effectiveness, suggests that this is at least a linear function although realism has been discussed in terms of verisimilitude (Meier et al, 1969) and it has been suggested that it is necessary for the simulation to have a minimal level of reality. This is illustrated in Figure 2b in terms of a saturating curve. Combined, it suggests that, the complexity-realismeffectiveness mechanism is concave and complexity is positively correlated with effectiveness. COGNITIVE PRESSURE - the amount of cognitive processing, required by simulation participants, depends on complexity, previous knowledge, experience, group size and team working skills and can be viewed as the process of creating learning. By their nature, simulations require participants to make decisions at intervals. Thus simulations are time mediated. The relationship between the necessary amount of cognitive processing and the time available produces stress and strain. Work pressure of this nature can produce role overload (French and Caplan 1972). This can be either a quantitative overload where the amount of work to be done is either too much or too little or a qualitative overload where the work is too easy or too difficult. French and Caplan (1972) also suggest that “Too little to do” or “too easy work” are also undesirable. For simulations this will occur when there is insufficient complexity. As shown in Figure 3a & b this suggests that although cognitive pressure is positively correlated with complexity it can be negatively correlated with andragogic effectiveness. This negative correlation occurs if cognitive pressure is too high. Vet, to make most efficient use of executives’ time it is necessary reduce simulation duration’s and hence increase cognitive pressure towards the point of role overload. COMPLEXITY-DURATION-EFFECTIVENESS MODEL - combining the two mechanisms, considering their shapes and slopes, suggests that for any simulation there is an optimum duration based on its complexity. This hypothesis can be tested through an analysis of simulations exhibiting a wide range of complexities and durations. Provided the simulations exhibit andragogic validity the hypothesis is proved if there is a significant, positive correlation between complexity and duration. METHODOLOGY To test the hypothesis, duration, decision-complexity and usage data was gathered from thirteen simulations. Because two pairs of simulations were consanguineous, to preserve degrees of freedom, the actual number of data sets reduced to eleven and the usage figures of the consanguineous pairs were aggregated. The data exhibited a wide range of complexity (as measured by the number of decisions made each period). This range of complexity ensured statistical reliability. The simulations are described later in the section entitled “The Simulations” and the measures discussed, separately, in sections entitled “Andragogic Effectiveness”, “Complexity” and “Duration”. A simple regression and correlation analysis was performed on the data with the number of decisions per period as the independent variable and the simulation duration as dependent variable. Based on the regression line, outliners were analyzed and further data gathered. The significance of the correlation was tested comparing the ratio of the Mean Square (regression) and the Mean Square (residual) against the F-statistic. FIGURE 2 A) COMPLEXITY -REALISM b) REALISM - ANDRAGOGIC EFFECTIVENESS ANDRAGOGIC EFFECTIVENESS REALISM FIGURE 3 a) COMPLEXITY - COGNITIVE PROCESSING b) COGNITIVE PRESSURE - EFFECTIVENESS COGNITIVE PRESSURE COMPLEXITY ANDRAGOGIC EFFECTIVENESS COGNITIVE PRESSURE Developments In Business Simulation & Experiential Exercises, Volume 21, 1995 32 The regression model set of simulations, Management College) actual duration’s was evaluated using a separate, control from a different source (Ashridge to forecast durations and compare with The Simulations Eleven simulations (listed in Figure 4) used in the regression and correlation analysis, were chosen with duration’s from two hours to just over two days and complexity ranging from simple to complex. This choice was made to ensure that the hypothesis could be demonstrated with sufficient confidence. FIGURE 4 SIMULATION DATA SIMULATION Duratio n Decisio ns Date Runs 1986 Management Challenge 7.0 9 1986 »24 A Management Experience 10.0 13 1976 >30 Global Operations 12.0 15 1981 >80 INTEX 16.0 19 1983 17 Operations 2.2 3 1977 >45 Product Launch 2.0 3 1988 »218 Reserve 11.5 14 1987 8 Retail Challenge 7.0 9 1983 >19 Sales Mix 3.0 4 1989 »100 Service Challenge 9.0 11 1989 >20 ‘Technique 20.0 25 1989 8 NOTES Duration is hours to brief, simulate eight periods and review. Decisions is the average number of decisions made each period. Runs represents the known ruins of thc simulation. Where widely used by clients the runs are understated and the degree is indicated by > or ». Where two versions of a simulation exist the runs are aggregated. Service Challenge usage includes that of the Casino Challenge. Sales Mix usage includes that of the Retail Mix. The three shortest duration simulations (PRODUCT LAUNCH. SALES MIX and OPERATIONS) were extremely simple with very limited learning objectives addressing specific management concepts. PRODUCT LAUNCH focussed on the product life cycle concept; SALES MIX focussed on meeting budgetary targets and group working: OPERATIONS focussed on simple production planning, scheduling and control. Next, in terms of duration, were three simulations (1986 MANAGEMENT CHALLENGE. RETAIL CHALLENGE & SERVICE CHALLENGE) designed with learning objectives covering basic business appreciation (marketing, finance & operations and their interactions). They were aimed at junior management and management trainees but each addressed a different industry sector and the mix of decisions and the models reflected the issues facing those sectors. Beyond these, in terms of duration, were two simulations (A MANAGEMENT EXPERIENCE & RESERVE) that addressed tactical issues with A MANAGEMENT EXPERIENCE concentrating on forecasting and financial control and RESERVE focussing on the link between commercial success and research and development. Finally, three simulations (GLOBAL OPERATIONS, INTEX and TECHNIQUE) addressed strategic management issues and were aimed at middle to senior management. These strategic simulations differed in terms of duration and scenarios. GLOBAL OPERATIONS covered basic strategic portfolio development. INTEX simulated the management of a consumer product business in an industrializing nation. TECHNIQUE simulated the management of a high technology electronics company with a product design and creative advantage dimension Andragogic Effectiveness Validity of simulations can be tested at three levels (Rolfe 1991) - content, internal and external. Content validity addresses how well the simulation replicates the real world, internal validity addresses how well it meets the course (learning) objectives and external validity addresses the question of whether learning was transferred and improved executive performance. CONTENT VALIDITY - by emphasizing the way the simulation replicates the “real world” content validity reinforces the view and implies there is a causal relationship between realism and andragogic effectiveness without discussing the need to fulfil learning objectives nor the duration of the simulation. In the context of this paper, this is a circular argument analogous to describing automobile quality, solely, in terms of top speed. INTERNAL VALIDITY - for the simulations under investigation internal validity was demonstrated because they all had been extensively used by many experienced trainers over many years and all are still in use. Kotler (1984) explains that the probability of repeat purchase relates to consumer satisfaction and, in turn, satisfaction is a function of the closeness of the consumer’s product expectations and the product’s perceived performance (p157). Figure 4 shows the date of origination and an estimate of usage in terms of total number of runs (The usage is based on known use and therefore understated. This is shown by using the greater than (>) and much greater than (») symbols to suggest the probable degree of understatement.) EXTERNAL VALIDITY - the simulations were used by practicing executives with, often, considerable business experience. Further, it is universal practice on short courses, for the participants to “score” each session on its benefits. Thus, it is reasonable to believe that the executives’ experience allowed them to forecast benefits accurately when scoring the session. In the long term, the continuing use of the simulations is a further indicator of external validity. Complexity Complexity can be measured extrinsically using the number of decisions made (Keys), documentation length (Nebenzahl (1984)) reports produced or intrinsically with software size indicating model complexity (Wolfe. 1978; Keys. 1980; Pray and Gold, 1982; Gold and Pray, 1984). For this analysis the number of decisions made each period was chosen since the decision- making nature of a business simulation means that the number of decisions were an indicator of the amount of cognitive processing and therefore deemed to be a more direct and reliable measure of simulation complexity. However, although the number of decisions was used as an indicator of complexity, there are several considerations. First, a simple sum of decisions implies that each are of equal importance and complexity. With the simulations chosen this was, reasonably, true but, it would not have been true for functionally specific simulations that address operational, tactical and strategic issues. Secondly the number of decisions made each period may not be fixed. This was true for the three longest simulations (that addressed strategic issues) where the number of decisions made each period depends on a team’s strategy and business development. For these simulations (GLOBAL OPERATIONS, INTEX and TECHNIQUE) a typical number of decisions per period was chosen. The actual figure chosen was based both on experience running the simulations and the design of the simulation where (for GLOBAL OPERATIONS) only some possible markets are attractive, where (for INTEX) only some possible products are viable and (for TECHNIQUE) only some product/market combinations are viable for a particular strategy. Developments In Business Simulation & Experiential Exercises, Volume 21, 1995 33 Duration Duration was the total time taken for the simulation and therefore included the time for briefing, for preparation, to make eight sets of decisions and to review the simulation. If, normally, less than eight periods were simulated then the rate of decision making at the end of the simulation was used to compute the eight period equivalent. Similarly, if the normal number of periods simulated was more than eight then the duration was reduced to the eight period equivalent. Since the simulations were run as a single session on a short course their durations could be assessed precisely. If they had been used with weekly decisions on an academic programme then the assessment of duration might not have been possible. Some groups would use “spare time” to work on the simulation, others would not. Further, cognitive processing would continue, subconsciously, even when students were working on other tasks. The durations shown in Figure 4 are actual running times rather than design times. In two simulations (SERVICE CHALLENGE and RESERVE) role overload had meant that durations had to be extended. (In SERVICE CHALLENGE this was only discovered when the original regression model suggested an unusually large difference between the actual and predicted durations - discussions with the main user of the simulation revealed that the normal duration was nine hours rather than the designed seven!) RESULTS A plot of duration against the number of decisions made each period (Figure 5) suggests a linear relationship between durations and number of decisions. Correlation analysis demonstrated significance above the 99% level (Figure 6). FIGURE 5 SCATTER DIAGRAM: DURATION & COMPLEXITY FIGURE 6 REGRESSION MODEL & MEASURE OF CORRELATION DURATION (hours) = 0.829* NUMBER OF DECISIONS - 0.355 F - RATIO = 3437 The linearity and degree of confidence in the correlation was surprising especially since the simulations chosen not only addressed a wide range of durations but varied in terms of target audience (from junior manager/specialist to middle/senior manager), learning objectives and date of construction. This led to gathering data on further simulations from a separate source and the regression model was used to forecast durations for these simulations (Figure 7). The control set of simulations covered a similar range of complexity and durations ranging from MSX that lasts three and a half hours to SMASH that takes two full days (twenty hours). SPITFIRE (one day) and GSIM (a day and a half) covered the middle of the range. All these simulations had proven validity with MSX in use from the late 1960s (with versions on Computer Time Sharing, Mini and Micro Computers). GSIM and its precursor in use for at least a decade and a half and the others have been used extensively for several years. SIMULATION DECISIONS DURATION ACTUAL FORECAST MSX 5 3.5 3.9 SPITFIRE 10 9.0 7.9 GSIM 17.5 15.0 14.4 SMASH 25 20.0 20.4 Initially the data received on the Ashridge control set showed MSX and SMASH within one standard error of the estimate. SPITFIRE and GSIM also seemed on the regression line but fell, symmetrically, about six standard errors either side of this. Further discussions with Ashridge showed that the data supplied for GSIM gave the duration for the simulation and review stages but omitted briefing and preparation. Further, the number of decisions represented a maximum rather than the typical. The corrected data produced a forecast that fell within three standard errors of the actual. Initial information on SPITFIRE showed nine decisions per period. Further investigation showed that the actual number of decisions made each period was ten. This halved the difference between actual and forecast. DESIGN AND USE IMPLICATIONS TIME CRITICALITY: The time available for simulations, used on short courses, is extremely restricted. This, time criticality, is illustrated from the prospectii of post experience course providers. In 1992, a leading European post experience course provider (Ashridge Management College) offered sixty- four percent of courses with a duration one week, twenty-four percent of two weeks and the balance evenly spread between three and four weeks. An analysis of the Chartered Institute of Marketing’s 1993 Residential Courses showed that, of over one hundred different courses offered, half were of one week duration. Of the remainder approximate one quarter were of three days and one quarter of two days! Only three courses were longer than a week and the longest of these had a duration of 13 days. The modal one week course duration matches the recommendation that leading (UK) corporations should be encouraged to set a public standard of five days off- the-job training per year per executive - as in the USA and the big West German companies (Handy 1987). Hall (1972) suggests “This overall time constraint places a severe limitation upon the degree of the distillation of the real world situation both in the provision of information and in the complexity of the model”. The demonstration that duration and complexity correlate means that simulations can be developed to match short course needs. SIMULATION FOCUS: Time criticality also has implications in terms of simulation focus and learning objectives. In particular cognitive processing can be viewed as the prime learning process and to make efficient use of participants time this must relate to learning objectives and extraneous processing should be avoided. This means that complexity should be directed towards learning objectives. Rollier (1992) illustrated this need with this cri de cour “Is it feasible to establish educational objectives first, and then to design games that fit those objectives?” This leads to a modified causal analysis diagram that views the DURATION (HOURS) 0 30 24 DECISIONS 0 * * * * * * * * Developments In Business Simulation & Experiential Exercises, Volume 21, 1995 34 process from that of the simulation designer. Here learning objectives are an independent variable that directly affects cognitive processing and through it to the required complexity (Figure 8). In this diagram, complexity and realism, are internal product features. CONCLUSIONS The significant relationship demonstrated between duration and complexity shows that complexity is not an independent predictor of andragogic effectiveness. As a result complexity is only relevant in terms of learning objectives and a broad spectrum of complexity (from simple to complex) are viable simulation (product) attributes. The introduction of duration into the model and its relationship with learning objectives counterpoints andragogic effectiveness with andragogic efficiency and makes complexity an internal feature of the model. Ultimately the need for efficient use of executives’ time on short courses and the problem of developing simulations that meet this need means that “Simplicity: That really is complex!” ACKNOWLEDGEMENTS I thank Jack Hardie and Lin Cawdron for supplying data on Ashridge Management College’s range of simulations, Robert Macgregor for supplying further information on GSIM and Julie Gray for supplying further information on SPITFIRE. REFERENCES Anderson, R.J. (1987) in Simulation - Gaining in the Late 1980’s eds Crookall, D. Greenblat, C. S. Coote, A. Klabbers, J. H. G. & Watson, D. R. Pergamon Press Oxford p43-SO Butler, Richard J., Thomas F. Pray and Daniel R. Strang (1979) An extension to Wolfe’s study of simulation game complexity, Decision Sciences 10, 480-486 Beer, Stafford (1967) Management Science - The Business of Operations Research, Aldus Books, London p111-115 Elgood, C (1984) Handbook of Management Games (3rd Edition), Gower, Aldershot. French, Jr., J.R.P., and R.D. Caplan (1972) in The Failure of Success ed Marrow, A.J AMACOM American Management Association, Inc., New York. Fritzche, D.J., and R.V. Cotter (1990) in Guide to Business Gaming and Experiential Learning ed Gentry, J.W., Nichols/GP Publishing, East Brunswick / Kogan Page, London Gold, Steven C., and Thomas F. Pray (1984) “Modelling Non-Price Factors in the Demand Function of Computerised Business Simulations,” in Developments in Business Simulations and Experiential Exercises, Vol. 11, eds David M. Currie and James W. Gentry, Stillwater: Oklahoma State University, 101-106. Hall, J.J.S.B. (1971) Management Game System - Administrator’s Guide, Honeywell l.S. Ltd., London p55. Hall, J.J.S.B. (1972), Whats in a Game Data Systems, September 1972 p15-17 Handy, C (1987) The Making of Managers The National Economic Development Office (NEDO), London Keys, J. Bernard (1977) “Total Enterprise Games: Computerised” in The Guide to Simulations/Games for Education and Training, ed R,E. Horn Cranford, NJ: Didactic Systems Inc. Kotler, P. (1984) Marketing Management: Analysis, Planning and Control Prentice/Hall International, Inc.. London. Meier, R.C., W.T. Newell and H.L. Pazer (1969) Simulations in Business and Economics, Englewood Cliffs. NJ: Prentice-Hall Miller, R and T. Leroux-Demers (1992) Business Simulations: Validity and Effectiveness Simulation/Games for Learning 22 4:261-285 Nebenzahl. l.D. (1984) Simulation & Games. Vol 15 No 4 December 1984 445-466 Sage Publications Norris, Dwight R (1986) External Validity of Business Games Simulation and Games, Vol 11 No.4, December 1986 p447-459 Sage Publications Pray, Thomas F. and Steven C. Gold (1982) “Inside the Black- Box - An Analysis of the Underlying Demand Functions in Contemporary Business Simulations” in Developments in Business Simulations and Experiential Exercises, Vol. 9, eds. David J. Fritzsche and Lee A. Graf, Normal: Illinois State University, 110-115 Rolfe. J (1991) SAGSET 1990 - The Proof of the Pudding: Effectiveness of Games and Simulations, Simulation/Games for Learning 21 2:99-117 Rollier, Bruce (1992) Observations of a Corporate Facilitator Simulation and Games. Vol 23 No 4 December 1992 442-456 Sage Publications Springer. C.H.. R.E. Herlihy and R.I. Beggs (1965), R.I Advanced Methods and Models, Richard D. Irwin, Inc. Homewood, Illinois Thorne, J (1992) New stimulus for those Simulations Management Today December 1992 Wolfe, J (1978) The effects of game complexity on the acquisition of business policy knowledge. Decision Sciences J9 143-155 SIMULATIONS GSIM, Berkhamsted England: Ashridge Management College MSX, Berkhamsted England: Ashridge Management College SMASH. Berkhamsted England: Ashridge Management College SPITFIRE, Berkhamsted England: Ashridge Management College Hall. J.J.S.B. (1986). 1986 MANAGEMENT CHALLENGE, London England: Hall Marketing Hall, J.J.S.B. (1976). A MANAGEMENT EXPERIENCE. London England: Hall Marketing Hall. J.J.S.B. (1992), CASINO CHALLENGE. London England: Hall Marketing Hall, J.J.S.B. (1981), GLOBAL OPERATIONS, London England: Hall Marketing Hall, J.J.S.B. (1984), INTEX - Industrializing Nation, London England: Hall Marketing Hall, J.J.S.B. (1983), OPERATIONS, London England: Hall Marketing Hall. J.J.S.B. (1977), PRODUCT LAUNCH, London England: Hall Marketing Hall, J.J.S.B. (1988). RESERVE, London England: Hall Marketing Hall, J.J.S.B. (1987). RETAIL CHALLENGE, London England: Hall Marketing Hall, J.J.S.B. (1983) RETAIL MIX, London England: Hall Marketing Hall, J,J.S.B. (1983), SALES MIX. London England: Hall Marketing Hall. J.J.S.B. (1989) SERVICE CHALLENGE. London England: Hall Marketing Hall. J,J.S.B. (1989), TECHNIQUE, London England: Hall Marketing FIGURE 8 MODIFIED CAUSAL ANALYSIS DIRGRAM LEARNING OBJECTIVES DURATION COGNITIVE PROCESSING COGNITIVE PRESSURE COMPLEXITY REALISM ANDRAGOGIC EFFECTIVENESS ANDRAGOGIC EFFICIENCY Table of Contents Volume 21, 1994 ABSEL: The Way We Were and Need to Be The Intellectual Structure of ABSEL: A Bibliometeic Study of Author Cocitatons Over Time Activity-Driven Time in Computerized Gaming Simulations An Assessment Framework for Determing the Effectiveness of Total Enterprise Simulations Attributes of Learning Organizations: Simulating the Relationships Business Policy/Strategy Case Extension using Pro-Forma Planning: A Computer Based Model Complexity: Is it really that Simple? A Random-Strategy Criterion for Validity of Simulation Game Participation Enhancing a Computer Simulation with a Structured Reporting Environment Experiencing a Foreign Culture: A Cross-Cultural Simulation Group Cognitive Style and Computerphobia in Functional Business Simulations Human Issues in Technology Implementation Management Simulator Incorporating Advertising Strategy into Computer-Based Business Simulations: A Validation Study Increasing the Effectiveness of Performance Evaluation Through the Design and Development of Realistic Finance Algorithms The Packer-Feeder Game: A Commodity Market Simulator Relationships Between R&D and Profitability: An Exploratory Comparison of Two Business Simulations with Two Real-World, Technology Intensive Industries Simulation of the Predictive Value of Mammography Simulation Performance and Learning Revisited Strange Bedfellows: Competency Models and ACBSP Accreditation Standards Using a Business Simulation to Study the Determinants of Ethical Behavior What Simulation Users Think Players Should be Learning from the Simulations ADA and its Implications for Experiential Training Boss/ Subordinate Perceptions of Instrumental and Supportive Leadership Behaviors in Relation to Myers-Briggs Thinking Type Cluster Analyses of American Universities' Business Core Curricula Structures Utilized to Satisfy Fifteen Curriculum Areas Cooperative Learning or Learning to Cooperate Experiential Learning: Constraining Students with Time Budgets How Different Workplace Experiences Affect Different Worker Values Implications of the Trend Toward Relationship Marketing for Experiential Learning The Increasing Cultural Diversity of the American Workforce: Management's Challenge of the 21st Century Information and Uncertainty as Strange Bedfellows: A Model and Experiential Exercises Leadership as a Medium: It's Emergence and Effect on Performance in Small Leaderless Groups Speed, Depth, and Breadth: Assessing Learning in Learning Organizations Teaching Strategic Planning, Problem Solving, and Decision Making with Envisionary Experiential Exercises Validating an Instrument for Student Evaluation of Teachers: Some Noteworthy By-Products Don't Teach Ethics to Business Students Emotional Reactions Toward a Simulated Layoff: Before and After the Manipulation Enhancing Communication Using a Presentation Package Implementing Marketing Policies through a Business Management Simulation Integrating Action-Based Learning into Executive Development Programs On the validity of using the Microsegmentation Principle in Media Simulationsm Participatory Systems Analysis Some Relationships between Cultural, Organizational, and Educational Experience and Perceptions of Influence The Use of Decision Support Systems with a Marketing Simulation: The Future is Now Astute Business Policy: A Simulation of the Automobile Industry The Business Policy Game CEO II: A Gaming Simulation for Assessment Computer Paced Project Management Simulation Computerized Tutor Support Systems DEAL & GEO: Progressively Integrating Gaming Simulations for Entrepreneurship and International Business An Interactive Simulation Game for Competitive Decision-Making International Operations Simulation/Mark 2000 (INTOPIA) Multimedia Simulation Cuts Training Costs for Anderson Consulting Concepts of Total Quality Management: An Active Learning Exercise Cooperative Learning: The Extended Jigsaw Managing Diversity--Values and Attitudes: An Experiential Exercise in Awareness Navigating the Shoals of International Management Development Evaluating Student Performance in the Use of Computer Simulation Entrepreneurial Simulation Program