AN ASSESSMENT OF SIMULATION USAGE IN MANAGEMENT ACCOUNTING COURSES Developments In Business Simulation And Experiential Exercises, Volume 19, 1990 58 AN ASSESSMENT OF SIMULATION USAGE IN MANAGEMENT ACCOUNTING COURSES Kenneth R. Goosen, University of Arkansas at Little Rock ABSTRACT Studies of simulation usage normally involve respondents who use different simulations. This paper presents the results of a survey of the users of the same simulation, The Management/Accounting Simulation. One objective of the study wag to determine the degree of variation in the administration of the simulation among the various users. The findings regarding variation in various aspects of administration and performance evaluation are presented in this paper. INTRODUCTION The Management/Accounting Simulation, a simulation designed specifically for courses in management accounting, has been in collegiate use since 1974. Originally, written in FORTRAN and designed for use on main frame computers, the simulation wag rewritten and published in 1988 exclusively for use on micro computers. Since the original publication date, nearly 20,000 students have participated in the simulation. Until this study, no formal survey of users had been made. Since a significant number of professors and instructors have used the simulation, the time seemed right to do an in-depth study of usage and learning benefits. Consequently, a questionnaire was designed and mailed to 110 users and non-users. Questionnaires were not sent to purchasers of the software in foreign countries including Canada. Thirty-two usable responses (29%) were received, Of this number 15 were from non users. This study presents the results of a survey of users of the same simulation. The value of this particular study is that results may be evaluated without any distortion from users who use different simulations. If significant variations in perceived learning and benefits are found in situations, as in this case, where the simulation used is a constant, then studies based on one simulation under the control of the same user may require that generalization of results be withheld pending additional further research involving other users. Simulations are flexible educational tools. Considerable variation can exist regarding administration, objectives, grading, and performance evaluation (Wolfe, 1985). A major purpose of the survey was to learn how usage varied in the following broad areas: Management accounting courses Goals and objectives Simulation administrative procedures Perceived learning and educational benefits. Evaluation of performance. MANAGEMENT ACCOUNTING COURSES This study revealed that the Management Accounting Simulation was being used in all levels of management accounting courses and in different types of schools. The simulation has been used in sophomore, junior and senior, and graduate level management accounting courses. Also, the simulation was being used in several community colleges. Fifty percent of the users (Table 1) indicated they were using the simulation in the graduate management accounting course. This finding is somewhat surprising in that the simulation was primarily designed for accounting majors. The fact that the simulation allows the use of management tools in the functional areas of marketing, production, and finance probably accounts for the use of the simulation with MBA students. Also, the maturity of graduate students may be another reason. EDUCATIONAL.. GOALS AND OBJECTIVES The Management/Accounting Simulation, even though heavily accounting oriented, is still a general or business enterprise oriented simulation. The goals or objectives and procedures in using this type of simulation can vary significantly. Respondents were given a list of some possible objectives (Table 2) and were asked to check as many as apply. The degree to which these objectives were accomplished is presented in Table 4. All user respondents except one checked choice number 1; that is, a primary goal was to use the simulation as a type of practice set. All users (100%) as shown in Table 4 believed that their objective in using the simulation as a practice set was accomplished or that their expectations were exceeded. There is virtually unanimous agreement by Simulation theorists that simulations have a high potential to integrate concepts of the same discipline or from different disciplines. Seventy one percent of the user respondents (Table 2a) reported that one reason for adopting the simulation was to give the course an integrative multi disciplinary element. Since business simulations in general have been recognized for their integrative potential it was not surprising, to learn that 76% of the user respondents (Table 4) reported their expectations on this objective were achieved or exceeded. Other simulation studies have reported that a high percentage of students consider business simulations to be fun. In this study, 65 per cent of the user respondents indicated that a major purpose of adoption was to introduce an element of fun in management accounting courses. Surprisingly, however, only 45% of the respondents (table 4) felt that there was significant accomplishment of this objective. The “fun” element of the simulation may have been negated somewhat by the fact that the simulation was challenging and required a significant amount of time and effort in order to achieve good results. Respondents were also asked to select the most important reason for using The Management/Accounting Simulation. Choice number 1 (ace Table 2b), giving students an opportunity to practice the use of management accounting techniques, was selected most often by user respondents (29%) as the main reason for adopting. Other reasons listed as most important are shown in table 2a. When asked about their student’s reaction to the simulation, 44% of the respondents reported that 50% or more of their students (Table 3) considered the simulation to be too difficult. However, Table 3 also reveals that 95% of the users felt that 50% or more of their students considered the simulation to be a positive learning experience. Although a high percentage of students seemingly prefer a less difficult simulation, it is possible that use of a simpler simulation might have significantly decreased the percentage of students who believe the simulation to be a positive or rewarding learning experience. When asked about whether the simulation was a positive learning experience, 50% of the respondents reported that 90% of the their students felt this way. Another 14% of respondents indicated that at least 60% of their students believed the simulation was a positive experience. Even though the simulation was regarded as difficult and challenging, it is clear from Table 4 that the simulation had been regarded by the majority of students as a positive learning experience. Another measure of learning benefits was obtained by asking users whether they plan to continue using the Developments In Business Simulation And Experiential Exercises, Volume 19, 1990 59 Developments In Business Simulation And Experiential Exercises, Volume 19, 1990 60 simulation. When asked this question, 16 users of 17 user respondents reporting reported plans to continue using the simulation again. On the part of user respondents it is apparent The Management/Accounting Simulation was perceived to make a significant contribution to attaining course objectives. In this study 15 respondents stated that they had not used the simulation. However, 7 of these non-user respondents reported they planned to use the simulation at a future date. The major obstacle to implementation by non-users appeared to be the perception that the simulation would require considerable startup time SIMULATION ADMINISTRATIVE PROCEDURES The use of a simulation requires the planning and execution of a series of administrative procedures. The more important of these procedures include: Determination of number of periods of play Assignment of students to teams Determination simulation grade percentage Selection of method recording decisions by students Selection of method for printing simulation results In this study, two-thirds of the user respondents (Table 6) reported assigning 3 students to a team. This number is consistent with other studies have that found that the optimal number of participants on a team to be 3(Wolfe, 1985). Regarding the number of decision periods, 60% of the respondent indicated that their students were required to make decisions for three periods. However, 3 respondents reported that they required their students to make 6 or more sets of decisions. It is not uncommon for users of other types of simulations, e.g., business policy simulations, to require 12 or more periods of play. Because The Management/Accounting Simulation is more oriented to the analysis of data, the number of periods of play was expected to be considerably lese. Because The Management/Accounting Simulation is geared towards the use of specific management accounting tools, it would seem unwise to require the first set of decisions to be submitted early in the course. The inability to analyze critical decision data because an important tool had not yet been taught could create frustration in some students. on the other hand, a benefit of an early start might be that those tools learned later in the course will be appreciated more. The majority of the respondents (63%) required the first set of decisions to be made between the fourth and sixth week. The Management/Accounting Simulation provides the students with two methods for submitting decisions to the simulation administrator: (1) A decision form (manually completed) and (2) an input computer program (Student Input Program) which creates decisions file on a floppy Disk. If submitted on a decision form then the instructor must assume the responsibility for keying in decisions on a computer. Because most teachers are highly adverse to performing unnecessary busy work one might think that all users of The Management/Accounting Simulation would require students to input their own decisions. In fact, 65% of the respondent did require their students to input their own decisions. However, the unavailability of computers or the desire to minimize input errors may account for the fact that 35% of user respondents were willing to input decisions for their students. Among some potential users there is the perception that simulations require considerable administrative effort on the part of the instructor. The time factor was the major reason mentioned by non-user respondents for not using the simulation. An analysis of the responses concerning time (Table 9) reveals that the total time devoted to the administration of the simulation varied between 20 to 40 hours. Assuming a 15-week semester, then the time devoted per week varied from 1 to 3 hours. Consequently, the total reported time does not indicate that The Management/Accounting Simulation required excessive administrative time. With repeated usage of the simulation, the amount of time devoted to administrative details should become significantly less. One of the more important expenditures of time appears to have been made consultation with students. From an educational point of view, this increased interaction with students should be considered as a desirable by- product of using the simulation. PERCEIVED LEARNING AND EDUCATIONAL BENEFITS The effect of using simulations on learning has been the study of much research. The findings on this point have been somewhat contradictory. In a few studies, the use of a Simulation did slightly increase test scores slightly. However, it is believed that simulations have benefits beyond mere acquisition or memorization of basic knowledge. While this study does not provide a quantitative measure of benefits, it is apparent from the responses to certain questions that the users felt that The Management/Accounting Simulation did significantly help achieve the learning objectives of their courses. In the introductory chapter of Management/Accounting Management/Accounting Simulation two of the more important objectives are stated. Students are told that the two basic objectives are: 1. To increase your understanding of the importance of managerial accounting as an aid to decision-making. 2. To be an instructional device for increasing your ability to use managerial accounting techniques and concepts. Perhaps the most important question underlying the survey is: Did students understanding and ability to use and understand management accounting techniques increase as a consequence of being required to play The Management/Accounting Simulation? From a careful examination of Table 10, it is apparent that the simulation is perceived to have more impact on some tools than others. All users agreed that the simulation had considerable impact to high impact on the students understanding of budgeting. The next management accounting tool receiving a beneficial impact from using the simulation was direct costing. Sixty four percent of the user respondents reported the simulation had considerable to high effect on the understanding of direct costing. The effect of the simulation on students understanding of cost-volume-proof it Management/Accounting was also reported as favorable. Fifty five percent of the respondents reported that the simulation had considerable impact on the understanding of cost-volume-profit analysis. An unexpected benefit of the using the simulation can be deduced from the responses concerning financial statement preparation. Only 31% of the respondents (Table 2) said that one of the goals in adopting the simulation was to increase student ability to understanding financial statements. However 45% of the respondents perceived that their students ability to prepare financial statements had considerably increased. Another 36% reported some effect on the ability of their students to prepare and understand financial statements. This result is highly significant, particularly, in light of a recent survey of controllers concerning important areas of accounting specialization. This survey revealed that 72% of the respondents considered the ability to prepare and analyze financial statement to be very important to career advancement. This skill was number 1 on the hat of 20 possible areas of specialization. The management accounting techniques reported by respondents to be the least affected by the use of The Management/Accounting Simulation was ratio analysis and capital budgeting. This low impact on students understanding from the use of the simulation is not too surprising since ratio analysis is generally treated by management accounting instructors as a financial accounting topic. The understanding of capital budgeting techniques by students was considered by the user respondents to have been minimally affected. This low perception rating i somewhat unexpected in that the simulation provides several excellent opportunities for present value analysis. In addition two cost analysis problems appear in the simulation that expressly required significant use of capital budgeting techniques. Developments In Business Simulation And Experiential Exercises, Volume 19, 1990 61 Developments In Business Simulation And Experiential Exercises, Volume 19, 1990 62 Developments In Business Simulation And Experiential Exercises, Volume 19, 1990 63 EVALUATION OF PERFORMANCE With the use of a simulation comes the unavoidable requirement to assign a grade value to performance. The difficulties and complexities of evaluating simulation performance has been the focal point of considerable simulation research. Measures or criteria of performance must be selected. The most commonly used measure is net income, but it is not uncommon for users to measure performance using multiple criteria of performance. Sales, market share, return on investment, market and market value of stock are frequently used in the same simulation play. This approach requires that the multiple measures be assigned weights. Another technique for evaluation of simulation play is to put considerable grade value on enrichment techniques. Written and oral reports concerning performance are frequently required. Under this grading scheme the important grade factor is the students ability to communicate the results of performance. Another means of indirectly addressing the evaluation problem is to assign analytical problems such as preparing budgeted financial statements and cash budgets. Also, documentation of analyzes underlying certain decisions may be required. Consequently, the evaluation of performance can involve some or all of the following: Selecting performance measures Assignment of a grade weight Use of Enrichment techniques Evaluation of enrichment techniques In this study the respondents were asked to specify the percentage that simulation performance counted towards the course grade. The grade weight given to the simulation varied considerable as is evident in Table 12. this table shows that some users counted the simulation as less than 10% and others more than 50% of the final course grade. Research into the use of simulation in other academic areas also showed considerable variation in the grade value assigned to performance. Simulations provide a rich and dynamic foundation for innovative teaching. The range of opportunity for innovative enrichment techniques is quite large. The opportunity for case type analysis, planning of strategy, use of budgeting, written and oral reports, etc. abound. An important finding of this study was that many users were using enrichment techniques. In Table 11, a list of some possible enrichment techniques are presented. The Management/Accounting Simulation student manual has built into it several optional enrichment techniques. In the student manual are presented 24 cost analysis problems. These problems contain information extracted from the simulation and require the use of specified tools. Sixty five percent of the user respondents reported assigning these cost analysis problems. In addition 35% of the users requested that their students support their decisions with budgets. Of some interest is the fact that 29% of the user respondents required their students to make oral presentations on simulation performance. SUMMARY OF SURVEY RESULTS Findings: Approximately 50% of the usage is in graduate level courses The major purpose for using The Management/Accounting Simulation according to 94% of the users was to use the simulation as a practice set. Simulation users reported that they believed at least 82% of the students found the simulation to be a challenging learning experience. In addition, the vast majority of users (88%) perceived that more than 50 of their students felt the simulation was a positive learning experience. Eighty-one percent of the respondents believed that the use of the simulation resulted in moderate to significant improvement in students understanding and ability to prepare financial statements. All users stated that the simulation had considerable to high impact on student’s ability to use and understand business budgeting. The simulation had the highest impact on business budgeting as a management accounting tool. The tool next in importance in which understanding was increased was direct costing. Sixty-four % of the respondents reported considerable to high impact. Teams typically consist of three students. Several users permitted teams to consist of one student. Students were typically required to make decisions for three periods. The first set of decisions were generally required during the 4th to 6th week of a semester. The majority of instructors (users) required student teams to use the Student Input Program. Sixty-five percent of users reported assigning some or all of the Cost Analysis Problems. About 1/3 of the users required their students to prepare a budget with each set of decisions The amount of time users devoted to the administrative details of the simulation varied from 1 to 3 hours per week. Sixteen of the seventeen users respondents reported that they planned to continue using The Management/Accounting Simulation. The simulation was assigned a grade value of between 10% to 30% of the course grade. This study reveals that considerable variation exists in the administrative style and perceived learning benefits The Management/Accounting Simulation. Also considerable variation exists in the perceived success of using the simulation. This study of the same simulation by a variety of users indicates that simulation research based on performance of students under the supervision or control of a single user may be insufficient to make generalizations. In order to obtain valid research data, future simulation research may require participation by many users of the same simulation. REFERENCES Goosen, Kenneth R.(1988), The Management/Accounting Simulation, Micro Accounting Publications Wolfe, Joe (1985), “The Teaching Effectiveness of Games in Collegiate Business Courses: A 1973-1983 Update,” Simulation & Games, pp. 251- 288. Table of Contents Volume 19, 1992 The Pedagogical Utility of a Management Simulation Game in a Business Policy Course Modeling Economic Environments in Business Simulations: Some Comparisons and Recommendations Experiential Learning and TQM Principles: Teaching Behavioral Science in a Business School The Use of Poster Presentations as a Final Project in the Business Policy Course The Device Business: A Management Simulation for Strategy Formulation The Advantages of Experiential learning in the Auditing curriculum A Framework for the Identification of Moderated and Mediated Performance consequences of Pedagogical Alternatives Simulating Qualitative Research Relating to Values and Lifestyle Segmentation A New Market Demand Model for Business Simulators Interactive Optimization Using the Method of Relative Improvement Preferences: Methodology and Empirical Evaluation The Service Trainer Simulation Benefits of Internet Computer Networks for ABSEL Members How Should We Measure Experiential Learning? An Assessment of Simulation Usage in Management Accounting Courses The Influence of Myers-Briggs Type and Group Dynamics on Simulation Performance Effective Leadership Behavior in the Desert Storm Arena: An Application of the Vroom-Yetton-Jago Model Installing and Consolidating Work-Team Values: The Effects of a Multicultural Outdoors Experiential Program Key Determinants and Decision performance in a Business Simulations and Experiential Learning environment Multi-Cultural Experiential Learning: A Computer Simulation in Indonesia Attitudes Toward and Emotions Related to Women as Managers: A Replication and Beyond Scenario Approach to Simulating Consumer Expenditures: A Cross-Cultural Analysis Insights into Ethical Decision making Activities and Organizational Performance: A Management Simulation Analysis of College Students and Managers Evaluating a Business Simulation Program for Joint Venture Negotiation and Management Teaching Business Interviewing Strategies with an Experiential Approach Cooperative Learning Across the Business Curriculum Modeling Total Quality Elements into a Strategy-Oriented Simulation Teaching Business Decision making using a Simulator Extending the Educational Utility of a Simulated Competition within the Confines of an Established Undergraduate Marketing Curriculum Expert Systems Versus Traditional Methods for Teaching Accounting Issues Confidence Extremes Diminish Quality Performance in a Total Enterprise Simulation Can Ethics Be Taught? A Simulation Tests a Traditional Ethics Pedagogy Through The Looking Glass, Inc.: Organizational Climate Research as Experiential Pedagogy The use of Cluster Analysis for Business Game Performance Analysis Power and Ethnicity: An Experiential Learning Exercise (How to Sensitize Students to Diversity) Directed Development of Critical and Creative Thinking Skills for Case Analysis Implementing Total Quality Management in a Computerized Business Simulation Product Quality in Business Simulations Use of Simulation for Ethics Education in Management Satisfying the University's Customers through Total Quality Management Instruction: A Case Study Personality Characteristics and Group Performance in Total Enterprise Simulations Concepts of Interval Estimation and Quality Control Charts via Computer Simulated Sampling An Examination of the Effect of Team Cohesion, Player Attitude, and Performance Expectations on Simulation Performance Results The Effectiveness of Inventory Management and Production Scheduling Training in a Total Quality Management Environment Peer Group Indicators of the External Validity of Business Games: A five-year Longitudinal Study Political Strategies and Personal Actions Does Practice make Perfect? Observations on Simulation Trial The Use of A Non-Business Computer Simulation to Teach Marketing Management BankPro Commercial Bank Simulation The Production Game Pursuing Excellence: Work Strengths, Assets and relevant Values (An Exercise) Test of a Short Outward Bound Experience for College Students What is it that we want Student To Learn? Using Two TQM Philosophies when Playing Blackjack The Lagged Effects of Decision Variables on Financial Performance Measures Used in Two business Simulations The Development of an Experiential Exercise for Career Planning and Effective Job search Performance Measuring Quality in Management of Business Pedagogy Exploring Quality and Productivity Improvement: Using and Experiential Process Toward a Generalized Architecture for Intelligent Reactive Management Systems The Quality Game TQX: Using Expert Systems to Improve Training in Total Quality Management A Simulation of the Effect of the Medicaid Payment Lag on the Financial Position of Community Pharmacies The Use of a Board of Directors to Evaluate and Validate Decisions in a Competitive Graduate Management Simulation Course The Impact of Academic Dishonesty on Business Simulations and Experiential Learning Activities Picture Project: An Experience of Icebreaking and/or Decision-Making An Effective Role-playing Exercise for Teaching Requisite TQM Supervisory Attitudes/Behavior Computerized Management Simulations and Some Correlates of Students' Satisfaction The Timing and Stability of Reactions to Market Structure in a Single Player Simulation Environment A Graphics Application Extension for a Simulated Decision Support System Environment Aspects of a Group Project Utilizing Actual Business data and a Computerized Accounting System The Role of Universities' Extended Learning Department in Assisting Organizations Implement Total Quality Management A three-dimensional Learning Experience to Develop Total Quality Management Skills Estimating Quality Costs by Computer Simulation Applications and Examples of Quality Control Software Assessing Business Pedagogy A Demonstration of an Experiential Process for Exploring Quality and Productivity Improvement Processes An Action-Ethics Dilemma: A Demonstration Org Sim Jr.: A 2-3 Hour Version of the 2-Day Blanchard/Murrell Organization Simulation Building Buildings: An Experiential Exercise in Organizational Structure, Communication, Leadership and Group Dynamics Expert Systems for Organization Design: A Demonstration A Demonstration of Product Quality in a Business Simulation: Version 2 of CEO Two Revolutions, Total Quality Leadership, and the Baldy: The Story of Milliken Total Quality after the Award - The Xerox Story Building a Competitive Advantage through Customer Satisfaction and Reengineering Human Resource Planning: Managing the Only Renewable Resource for a Competitive Advantage Using Simulations to Teach International Issues: An analysis of the Multinational Management Game's Learning Environment Quality Function Deployment: A Tutorial Care and Nurturing of Teams Making a Good Thing Better: Adding TQM to Participative management Empowering Organizations to Redesign and Transform Themselves - Eastman Chemical Using Simulations in Field Management Development of an International Life Insurance Company Reinforcing the TQ Environment via Simulation Transforming a Business College into a Total Quality College