EXPERIENCE IN THE USE AND ASSESSMENT OF SIMULATION IN MANAGEMENT EDUCATION Insights into Experiential Pedagogy, Volume 6, 1979 132 EXPERIENCE IN THE USE AND ASSESSMENT OF SIMULATION IN MANAGEMENT EDUCATION Warwick Savage, The New South Wales Institute of Technology ABSTRACT A model is developed which can be applied in pre-service programmes to assess the progress of students and simulation-based courses towards achieving prescribed educational objectives. INTRODUCTION Many fields of education have recognised the value of simulation, and growth of its usage has been extremely rapid. In management education, the fastest growth has been in the introduction of gaming. A Study of gaming bibliographies (7) reveals a wide variety of game designs and simulated situations. It may be assumed that an extensive range of educational objectives has provided goals for game designers, and that each game IS intended to provide a means of achieving some specific educational objectives. These range from reinforcement of one theoretical concept (e.g. Price elasticity of demand, or the effect of communication in intergovernmental relations), to demonstration of complex inter-relationships (e.g. international finance transactions, and labour/ management negotiations). Having chosen simulation as the teaching technique, an educator faces the dual problem of assessment of student performance, as well as the assessment of the selected game itself in terms of its effectiveness in achieving course objectives. The aspect of student performance evaluation has been the subject of discussion by a number of authors (1;4;5). However, to date, little emphasis has been placed on the different purposes for which a particular game may be used. As a result, a reading of the literature suggests an implicit assumption that there is some universal objective, and some all- inclusive measure of performance which is applicable to participants in every game. Much of the discussion is centred on a comparison of the relative values of objective and subjective criteria, unidimensional and multi-dimensional criteria, or relational and ranking approaches. Publications relating to goal-oriented behaviour highlight the need to ensure that measures of performance are derived from objectives (h;2). In view of the wide range of educational objectives nominated for learning situations in which gaming is used, it would appear unlikely that a universal measure of performance would be appropriate beyond some lowest common denominator of objectives. Such a level of assessment is often totally inadequate for the needs of the assessor. A more practical approach would seem to be to discuss specific measures of performance in terms of particular educational objectives. When an educator has defined objectives, the problem is then to identify measures of performance, and develop a quantifiable score which reflects it. This is a general problem, and this paper will be confined to the use and assessment of simulation in management education courses at the N.S.W.I.T. It is felt that the discussion will have some relevance beyond these confines, particularly when simulation is used in “pre-service” courses, since, in many cases, the educational objectives will be comparable. ASSESSMENT TECHNIQUES The Business Studies Degree course at N.S.W.I.T. includes four fields of concentration; Accounting, Marketing, Operations Management, and Public Administration. Gaming was introduced into the course In 1974 as the ‘capstone’ subject, Business Simulation. It is a compulsory subject for all undergraduate students, and has as its primary function the promotion of integration of students’ understanding of all four fields of concentration. Initially, Business Simulation consisted solely of a management game, ‘The Business Policy Game’ (3). Since its introduction, a number of associated activities have been progressively added. Today, the subject is a tightly structured experiential learning experience in which the major knowledge and behavioural aspects of business are simulated. Around the core of a management game are built exercises in the development and use of management information systems, preparation of written reports and major plans, industrial relations negotiations, and executive performance appraisal interviews. In addition, less tangible, but equally important, features of business are superimposed. These include a competition to replicate the conflict between personal and group goals and to develop realistic competitive relationships between the functional executives. Furthermore, role- playing by both staff and students, through a simple organisation hierarchical model, introduces power and authority relationships, with their attendant pressures on individuals. One result of these developments of the subject has been to alter significantly the techniques required for performance evaluation. During the early stage, game-based assessment as discussed by Hand and Simms (5) and Biggs (1) was adequate. Most students adopted a game role which was in accordance with their field of concentration. It was held that performance in the game, as measured in ‘output’ or objective terms, e.g. inventory costs, sales, R.O.I., etc., indicated the degree of integration achieved by a student. More recent study of the educational objectives resulted in the introduction of concepts of vertical and lateral integration. By these are meant respectively, integration of the knowledge components of a students own field of concentration together with an understanding of its relationship to others; and by ‘lateral integration’, a student’s understanding of other fields of concentration and their relationship to each other. A further change was widening the definition of “simulation of business” to include the salient ‘non task-specific’ features such as power and authority relationships outlined briefly above. Initially, to determine student performance, various components of the subject were graded, and a final assessment score determined from their weighted aggregation. As revised objectives were developed, earlier measures of performance were found to be less than adequate. Clearly lacking was any evident basis Insights into Experiential Pedagogy, Volume 6, 1979 for evaluation of component parts, and selection of weighting of each. In addition, there was no reliable measure of the impact of various structural changes on the effectiveness of the subject in achieving educational objectives. Although replacement of simulation as the most applicable technique for presentation of Business Simulation is not at issue, subsequent development of the subject has introduced the necessity to review methods of assessment both of student performance, and of appropriateness of the subject structure in achieving the educational objectives. SUBJECT STRUCTURE Outlined below are the components of Business Simulation, and a description of their contribution to the educational objectives, which are to: (i) Increase students’ understanding of their own field of concentration, and its relationship with others. (ii) Increase students’ understanding of other fields of concentration. ‘The Business Policy Game (3) is used as a basic core. This game is a moderately complex simulation of a manufacturing/marketing organisation. Decision areas include Production, Marketing, Finance, Planning and General Management. Decision periods simulate one quarter’s operations, and eight quarters of historical data is available at game-start. Groups of six teams form one industry, and within this they are competitively interactive. Each company is autonomous, although 51% of the shareholding ownership is deemed to be owned by a Holding company. For each industry, a lecturer represents the Managing Director of the Holding Company. Authority and power relationships flow from internal organisation within companies, and external contacts with a lecturer. It was found that Interaction between companies generated a very high degree of competitiveness, which promoted strong motivation. However, it also promoted strong de- motivation in “losing teams. As a result this competitive aspect was moderated, maintaining an adequate, high level of motivation, while almost eliminating negative de-motivation. Whilst the game provides a vehicle for exercises in vertical and lateral integration, it also gives continuity to associated activities. For each of the first four quarters of operation, a detailed report of company performance is required by the Holding Company. Each company report is composed of contributions from all executives (students) in that company. These reports provide assessable assignments indicating students’ understanding of the four fields of concentration in the course, the quality of their written expression, and reaction to time-pressure task requirements similar to those they will meet in business. Feedback to students on these reports is extensive, and timed to allow sequential improvement of company analysis and report-writing skills. At a later stage, a major report is required from each company detailing its plans, objectives and strategies for the coming five years. As well as giving experience, and providing an indication of students’ ability in lateral integration, this report enables inter-company competitiveness (with its double edge of motivation/de-motivation) to be replaced by cooperative group goal setting. Assessment of a company’s performance is made on the basis of the goals specified in its particular major report. Preparation and presentation of this report in the face of severe time constraints provides further experience in the pressures associated with subordinate/peer/superior relationships, and in getting things done through other people. One other major report is required after the game has concluded. This is final review, and complete audit of company and management performance. Taken with the previous major report, it ensures that students’ exposure to vertical and lateral integration is wide and thorough. During the game, a competition is held to identify the “best” executive in each functional area. Students are ranked according to several objective factors which have been previously explained to them in detail. Further, this competition serves two major functions: it promotes a realistic degree of conflict between personal and group goals, and develops some inter-industry competition which is vital for the success of a subsequent industrial relations negotiations simulation. Within business, compliance-gaining in a group situation is a necessary individual skill. Examples of the need for this may be found in committees, task- force activities, contract bargaining, consulting, and union/management negotiations. Relevant aspects of the above situations are simulated in an industrial relations exercise, in which students role-play employees and management to reach agreement on a ‘log of claims’. The negotiated terms determine the basic production costs in each industry for subsequent decision periods. Usually the teams are drawn from different industries, and performance in the competition provides motivation for active participation. Interpersonal and verbal presentation skills are also developed and tested through this exercise. A one-day session, during which students make several successive decisions under conditions of frustration- induced tension, is a further activity. This simulates a common business situation, wherein the group is subjected to severe constraints which aggrevate inter-personal conflicts. RESEARCH METHODS It is evident from the preceding discussion that evaluation of both students and subject structure must take account of a range of activities, each of which makes some contribution to one or more of the educational objectives. The variety of activities, and their interdependence, preclude the adoption of a single game-based evaluation method such as those suggested by Biggs, or Hand and Simms. If such a method is to be used, it must be in conjunction with others. The purpose of this paper is to relate activities to educational objectives, and to propose means of developing methods of assessment of both student performance, and efficacy of the simulation approach, in achievement of those objectives. Since some activities contribute to the achievement of more than one objective , there will be some duplication. However, each activity has a primary area of impact, that is, a relatively greater part of its total contribution is made towards reaching one objective, and a lesser part towards others. Table I shows how activities and educational objectives are related in Business Simulation. Without quantifying actual 133 Insights into Experiential Pedagogy, Volume 6, 1979 contribution levels, they can be categorised as primary and secondary. It can be seen from Table I that the design of activities is governed by their function in Business Simulation. The function also governs development of criteria for evaluation of both student performance, and subject efficacy. As an example, objective criteria, based on decision outputs (e.g. Profit, R.O.I., Market Share, etc.) can be applied to determine Executive Role performance, but these criteria should be given a greater weighting in the assessment of the progress which has been made towards the first of the two objectives, and a lesser weighting in relation to the second objective. Uni-dimensional or multi-dimensional criteria can be adopted for each activity. It is important that the weighting for an activity is allocated such that a higher proportion is given to areas of primary contribution. Again using Executive Role as an example, the same objective measures may be used, but weighted 0.6 for their contribution to ‘understanding own field of concentration, and 0.4 for ‘understanding other fields of concentration’. Initial weightings are essentially subjective, but they can be modified as required. It is suggested that input from practitioners in the field of the simulation be utiljsed to develop a “base line for use in subsequent adjustments of initial weightings. From a knowledge of educational objectives, and experience of the content of each activity, a basis of assessment of student performance can be developed. Since it is clear that all activities directed primarily at a particular objective do not contribute equally to its achievement, the initial basis of assessment will need to be reviewed. Extending the previous example, the contribution of ‘Executive Role in the Game’ is probably greater than that of “Competition to reaching “understanding of own field of concentration’, even though the same objective is specified as the primary direction of impact for both activities. An additional series of weighting factors is required, to reflect the relative contribution of each activity to a particular objective. In the example used above, ‘Executive Role in the Game” may be given a weighting of 0.5, while that for “Competition’ could be 0.1. This second series of weightings can be summed for each objective, and the totals adjusted to reflect overall policy of the educational institution. At N.S.W.I.T., both objectives are regarded as equally important for the subject Business Simulation, and total column weightings are adjusted accordingly, by making them equal. Because weightings have a direct effect on evaluation of both students and subject structure, initial emphasis was given to identifying a means of determining them. Since the objectives are concerned with “understanding” rather than “knowledge”, appropriate evidence of progress towards them is best obtained from examples of application of knowledge. Testing levels of theoretical capability would not be relevant, given the particular objectives. Examples of applications of the theoretical knowledge relevant to all four fields of concentration of Business Studies undergraduates were found to proliferate in the area of management. There is an abundance of literature pertaining to management, and the functions of managers. From a selection of the literature, and interviews with a range of oractising managers, about 50 aspects of a manager’s function were identified. These were grouped into Major and Minor aspects. Only those 36 items generally recognised as being major aspects of a manager’s function were considered. These were then further categorised, firstly be degree of commonality, and secondly by functional specialisation (e.g. Finance, Marketing, etc.). From the first sorting process, two groups emerged; Basic Skills, which were common to almost all managers; and Role Functions which were generally unnecessary at low levels of responsibility, but which became more important as responsibility increased. However, during the total categorisation process, it became clear that decision- making skills were regarded by practising managers as being extremely crucial to any management function. Decision making was added as a third category, along with Basic Skills, and Role Functions. The succeeding task, that of ranking various aspects of management in terms of importance, had been simplified through categorisation. It was expected that ranking of categories would be relatively simple, particularly since Decision- making had already been identified as critical. However, the ranking of Basic Skills and Role Functions was inconclusive, due to an even distribution of opinion as to the relative values of universality of Basic Skills, and impact on business of Role 134 Insights into Experiential Pedagogy, Volume 6, 1979 Function attributes since these were defined as being associated with higher levels of responsibility. One clear indication was that Decision-making was at least as important as both other categories together. As a result, initial weightings allocated were 0.5 for Decision-making, and 0.25 each for Basic Skills and Role Functions. Initial weightings were reviewed in terms of the second categorisation, that by functional specialisation. Whilst some modification may be warranted in particular educational situations, it was considered that equal weighting for each function was appropriate at N.S.W.I.T., given that the students represented all four fields of concentration. To this point, a measure of effectiveness (management ability) has been determined which indicates progress towards educational objectives. Within this measure of effectiveness, the major components (specific aspects of a manager’s function) have been identified. A simple weighting was assigned to these, reflecting their relative importance in the aggregate measure. Determination of the measure of effectiveness, and its major components, is a vital stage in the development of a meaningful assessment. Activities within a simulation course can be analysed in terms of their relevance to the major components of the measure of effectiveness. This analysis not only enables determination of a relative ranking of the importance of each activity in achievement of educational objectives, but also provides the basis for assessment of students, and the subject structure. This process requires grading of students’ performance for each activity. Individual grades, taken with the appropriate weighting, represent a quantified measure of individual student performance. Aggregate student grades for each activity, weighted, would provide assessment of overall progress towards educational objectives. Successful use of the process outlined above relies on accurate measurement of student performance. This requires detailed grading guidelines for each activity. Whilst aspects of Basic Skills, and Role Functions can be defined and measured relatively accurately within Business Simulation, Decision- making poses problems, particularly of definition. Further, there is no stable and fully accepted scale for quality of decision making. In business, the relative importance of elements of decision-making are subject to change over time, and this adds further complication. The model proposed for use in assessment of subject structure takes into account the features of measurement outlined above. At N.S.W.I.T., a questionnaire has been developed for use with students of Business Simulation. It is designed to indicate students’ abilities in Basic Skills, Role Functions, and Decision-making. Tentative rankings of importance of elements of Basic Skills, and Role Functions, were derived from the literature relating to functions of a manager. These were converted to a ‘profile’ of abilities. A similar profile was obtained for the Decision-making category by applying the relevant section of the questionnaire to a sample of practising managers. In designing the sample, emphasis was placed on features of organisations, and hierarchical level of managers. Organisations were selected from those which employ N.S.W.I.T. students or graduates. This ensured that any bias in the initial survey was towards the immediate relevance of Business Simulation, regardless of future changes which may require a modified subject structure. Other organisation features included success, as determined from at least a three-year history of increasing profits; size, based on number of employees; and category of business. The sample proportions reflected the distribution of size and business category in the Sydney city metropolitan area, the primary catchment area for N.S.W.I.T. students. For the sample design, equal proportions of managers were selected from functional areas of Finance, Production, Marketing and Public Administration. These reflect the four fields of concentration in the Business Studies course at N.S.W.I.T. Salary ranges were used to indicate hierarchical levels, and proportions selected to represent a broad triangular organisational structure. RESULTS AND DISCUSSION Overall, results obtained indicated some directions of change in students’ understanding of their own and other fields of concentration. Although some aspects of the test instruments used require modifications, the tentative model proposed still appears valid. Further validation of the model will follow modification of the test instruments, but these modifications may result In some existing data being inappropriate for inclusion in later research. In the model, progress towards educational objectives is determined from measures of performance derived from students’ abilities in three categories of activities - Basic Skills, Role Function, and Decision making. The test instruments used were designed to indicate students’ abilities in these areas, and results obtained will be presented using the same groupings. Basic Skills: Because these include a wide range of activities (e.g. verbal skills, influencing others, managing time, etc.), performance is usually evaluated over a period of time. However, the intent of the model is to provide performance assessment data at two discrete times; before and after a specific learning experience. An instrument to achieve this was developed based on self-evaluation. Students were asked to rank their confidence in their ability to perform a range of activities requiring basic skills. Following post-test application of this instrument, estimates submitted by a sample of consenting students were compared with relevant aspects of their activities during the course of Business Simulation. Objective tests were used wherever possible, being supplemented with discussions with their lecturers in a range of subjects. Close agreement was found between students’ post-test estimates of their abilities, and actual ability level evidenced in their work. Continued validation will be necessary before such a test can be applied with confidence, particularly for pre-test results. It is possible that one outcome of Business Simulation is an increase in students’ ability to recognise their strengths and weaknesses. Further validation is required of the instrument for both pre- test and post-test use. Application of the self-evaluation instrument to students during 1978 indicated a general improvement in Basic Skills ability. Even with the doubtful validity of the test, the indication was sufficiently strong to warrant some confidence, at least in the direction of change, if not the magnitude of it. Comparison of post-test results and a base-line determined from the literature and practising managers indicated sizeable gaps in areas of Taking Responsibility, Dealing with People, and Managing Time. 135 Insights into Experiential Pedagogy, Volume 6, 1979 Role Functions: As with Basic Skills, a test instrument was required to measure students’ level of understanding at particular times. For this purpose, a questionnaire was used in which students were required to identify which items on a list of executive functions should be the responsibility of various functional executives. In spite of careful pretesting of the instrument, it was found that ten of thirty-two questions were seemingly still subject to a range of interpretations. The remaining twenty-two questions indicated a strong trend towards the base profile. Areas apparently needing greatest emphasis, as shown from the results, were General Management and Finance. Understanding of the Production, and Economic Planning role functions seemed to be increased most strongly during Business Simulation. Decision-making: Two separate target audiences needed to be considered in selecting an appropriate instrument to measure ability in this area. It was to be applied not only to students, within the same constraints as discussed for previous instruments, but also to managers. This imposed additional limitations on the design to meet such requirements as interest-generation, to ensure a high response rate; self- contained, in order that respondents would give consistent replies in the absence of a verbal introduction, and a ready source of further information; and brevity, since managers were being asked to donate some of their time. Rather than introduce an instrument containing major innovations, currently-used and widely-accepted instruments were investigated. Of the range available, the most appropriate format was that found in decision-making exercises frequently used as introductory tools in group activity courses. These instruments are usually brief situation descriptions, followed by a short ranking exercise requiring a series of decisions to be made. However, they are also generally content-free. To measure business decision- making ability, an instrument was developed having similar characteristics of format, but being content- specific. In it, a business situation was outlined briefly and respondents were asked to rank a given set of possible management actions in order of appropriateness to that situation. The concepts could not be replicated precisely due to the introduction of additional variables (e.g. time), and interdependencies arising from individual characteristics (e.g. management style). From the responses of management a profile was constructed against which students’ responses were to be compared. Whilst there was general agreement between post-test results and the basic profile, the instrument did not provide sufficient sensitivity to allow clear delineation between aggregate responses. However, the fact that profiles could be generated, although admittedly not sharply defined, suggests that the fault lies In the situation description and management action list, rather than in the basic instrument design. From the results and in the discussion, areas requiring modification have been identified. As stated in the preamble to the results, the basic conceptual model appears valid, what has become clear is the need to modify measuring instruments. In the Basic Skills instrument, the primary requirement is validation. Continued field testing, and simultaneous application of a range of alternate measuring instruments will be necessary. The problem of a variation of perceptive ability during Business Simulation experience will still remain. However, whilst a highly accurate measure of the degree of change in Basic Skills ability is desirable, it is not essential, given that it is one of a number of measures of performance, and that a base line can be developed for use in evaluating results obtained after Business Simulation experience. Areas of student weakness can be identified, and the assumption made that similar areas will also be in evidence in the following course session. Subject structure can be modified to apply effort to offset weaknesses, drawing resources from areas where performance is unduly high. Results obtained from pretesting in the following sessions can be used to confirm the course structure selected. Also arising from the results and discussion is the need to review questions used in the Role Functions performance measuring instrument. Again, continued field testing will result in development of a suitable instrument. Subject structure is already being reviewed in order to put more emphasis on areas of weakness identified in the initial study. In summary, initial application of a new set of instruments has shown the need for some modifications within them and further validation of each. Making modifications will probably limit the extent to which 1978 data can be used in future. It will also require large samples for testing in 1979, both to provide adequate validation, and to ensure rapid development of sufficient field experience to support wider use of the instruments. CONCLUSION Arising from several years of experience in the use of simulation in management education, a tentative model has been proposed for use in the assessment of the efficacy of simulation, and to provide a basis for the evaluation of student performance. The model takes account of the range of both educational objectives, and available simulation packages. A variety of relevant examples of applications of a particular knowledge base is derived from the objectives. Elements of the applications are identified, and used to select appropriate simulations. The same elements provide part of the means for calculating relative weightings used in assessment of progress towards educational objectives, and evaluation of performance of students. Remaining factors required to develop relevant measures of performance are derived from people with real-world experience in the field of the simulation. Linking educational objectives, students, course structure and practitioners ensures that measures of performance will be appropriate, and course structure flexible, facilitating the maintenance of course relevance, while promoting development. In addition to the model, Instruments have been designed for use in performance measurement and assessment. Initial concentration has been on course assessment, rather than student performance evaluation. Further research in this latter area, and on instrumentation, will widen considerably the educational application of simulation. 136 Insights into Experiential Pedagogy, Volume 6, 1979 REFERENCES (1) Biggs, W. D., A Comparison of Ranking and Relational Grading Procedures in a General Management Simulation”, Simulation and Games, Vol. 9, No. 2, June 1978. (2) Chase, R. B. and N. J. Aquilano, Production and Operations Management, a Life Cycle Approach, Irwin, Illinois, 1973. (3) Cotter, R., The Business Policy Game, Appleton- Century-Croft, New York, 1973. (4) Embry, D. H., A. J. Strickland III, and C. R. Scott, Jr., “Instructor’s Manual -Tempomatic IV : A Management Simulation’, Houghton, Mifflin, Boston, MA, 1974. (5) Hand, H. H. and H. P. Simms, Jr., ‘Statistical Evaluation of Complex Gaming Performance’, Management Science, Vol. 21, No. 6, Feb. 1975. (6) Humble, J. W., Improving the Performance of the Experienced Manager, McGraw Hill, London, 1973. (7) Zuckerman, D. W. and R. E. Horn, The Guide to Simulation/Games for Education and Training, Information Resources Inc., Massachusetts, 1972. 137 Table of Contents Volume 6, 1979 The Use of Cases with Role Plays in a Research Methods Course Debugging and Implementing the Live-Case Approach to Marketing Research in the Australian Environment Using a Case as the Basis for a Modified Debate Using Cases in Business Communication Classes A Structured Approach to Case Analysis and Reporting Intercollegiate Case Analysis Competition as an Experiential Learning Experience Attributes Germane to Student/Live Case Situations Understanding Dispute Resolution Through Experiential learning Personality Development and Conflict Dynamics: An Experimental Design to Study the Effects of Teaching Methodologies on Conflict Resolution Interpersonal Competence and the Digital PDP-11:40 The process of Writing a Collective Bargaining Simulation: A Case Study in Practical Pedagogy Simulating Negotiations in an Educational Context Simulations as a Technique for Teaching Collective Bargaining Experiential Processing of Differing Managerial Perspectives: The Use of a Game Show Format Individual Self Report vs. Group consensus in Small Decision-Making Groups Effects of Sex-Role Stereotypes on Promotion Decisions: An Exercise Computer Aids to Planning: The Budget and Forecasting Module Use of a Microcomputer in a Decision Analysis for Investment Portfolio Selection A Game of Investment Strategy: Description, Use, Criticism and Modification Coping with Future Uncertainties Through Probabilistic Budgeting Administration and Design of Simulation Materials for a Specialized Management Training Program Managing Stress in Organizational Life Banksim: The Bank Management Simulation Issues in the Organizational Application of Simulation and Experiential material Games Within Games: The Role of the Glitch Experiential Learning from Classroom to Business An Evaluation of the Small Business Institute Program as an Experiential learning Exercise Conversations with Top Management (Simulated) Experience in the Use and Assessment of Simulation in Management Education A Statistical Analysis of Simulation Users: Relationships between Stress and Human Subject guidelines Group Decision Making in a Computer Game Analysis of Demographic and Psychosocial Variables Comparing Performance During Three Managerial Accounting Simulation Schedules Business Policy Simulation and the Intense Course Structure The Use of Intensive Simulation in Executive Development and Academic Settings The Use of Experiential Exercises in the Undergraduate Consumer Behavior Course A Personal Marketing Strategy Approach: Framework and Application Teaching PERT Experientially in Marketing Research Discovering the Majority Fallacy An Experiential Approach to Studying International Business A Demonstration of the Business Simulation Game as a Curriculum Assessment Device Relating Teaching Methods with Educational Objectives in the Business Curriculum The Junior Achievement Applied Management Program as a Compromise Situation Between the Simulation and the Internship Who is Using Computerized Business Games?: A View from Publishers' Adoption Lists Trials and Tribulations in Testing Educational Innovations An Examination of the Perceived Effectiveness of Computer Simulation in a Classroom Setting as Affected by Game, Environmental and Respondent Characteristics Game Administration: A Life Cycle Analysis Research on the Effectiveness of Using a Computerized Simulation in the Basic Management Course Gaming and Attitudinal Change The Teacher-Student Relationship in Experiential Classes and the Student's Perception of Course Effectiveness An Exploratory Study of Student Characteristics and Educational Processes in Programmatic Experiential learning General Incongruity Adaption Level (GIAL) as a Predictor of Risk Preferences in a Simulated Management Game The Cap-Stone Opportunity: Combining Business Simulation and Experiential Learning Incorporating MIS/DSS into Policy Courses Via Simulation Who Benefits Most from Participation in Business Policy Simulations: An Empirical Study of Skill Development by Functional Areas The Use of Program NAMEX in Teaching the Accounting for Nonmonetary Assets The Use of Program CVP in Teaching Cost-Volume-Profit Analysis Experiential vs. Traditional Classroom Approach in the Basic Management Course: A Puerto Rican Experience A Guide to the Successful Use of Business Simulation Games The Dynamic Aspects of Interactive Gaming Puts the Realism into Gaming System Representation of SIMORG The Design of a Database System to Support Business Simulation and Experiential Learning A Data Entry and Retrieval System for a Computer Simulation (DERS) Simulation - An Advantage for Accounting Research Computer-Aided Project Performance Control Simulation: An Interactive Experiential Gaming Technique for Managerial Decision Making Utilization of Manual Simulation Games to Develop Scenarios of Future Events - An Exploratory Study Foundry: A Foundry Simulation Customized Debriefing: The Achilles Heel of Experiential learning? Computerized Business Simulations and Experiential Learning Exercises: An Instructional Interface The Baseball Game: A Group Role Observation, Problem Solving Experience A Practical Design for Experiential Learning Exercises: Roles, Technical Equipment and Alternative Debriefing Formats