THE EFFECTIVENESS OF PROGRESSIVELY COMPLEX ACCOUNTING SIMULATIONS IN INCREASING DECISION-MAKING PERFORMANCE Exploring Experiential Learning: Simulations and Experiential Exercises, Volume 5, 1978 60 THE EFFECTIVENESS OF PROGRESSIVELY COMPLEX ACCOUNTING SIMULATIONS IN INCREASING DECISION-MAKING PERFORMANCE Clayton H. Walker Winona State University INTRODUCTION When a business simulation is first encountered, it represents a myriad of new rules and relationships for the player to learn. Students at universities frequently express dismay and hopelessness during the first several decision-periods but learning is so rapid that by the sixth or seventh periods, they begin to lose interest. It seemed to me that researching the effect of using a simple simulation and then a more complex one would reveal the value of using simulations to prepare people for decision-making in more complex environments. Secondly, the value of designing a simulation with increasing complexity might be inferred from this research 12, page 190). Definition of Terms: The idea of simulations being “simple” or “complex” is not clearly defined in the literature so I will define a complex simulation in terms of some attributes which make decision-making require more preparation and understanding in order to be effective. Simple, relatively speaking, is a lack of those attributes which make a simulation complicated. The most complicating factor in business simulations is interactivity which upsets predictability by introducing conflicting, competitive game strategies. A second-most complicating factor is variability and alternative choices in the input markets for the factors of production (e.g. labor, materials, overhead cost.) Thirdly, alternative products and marketing areas with advertising and research and development decisions increase the complexity of a simulation. Finally, production line-balancing, transportation and warehousing, and alternative production methods for operational leverage or financial alternatives for financial leverage are used. Between the two simulations used in this research report, FINANSIM [4) contains fewer and V.K. GADGET [3] contains more of the complicating factors. A third simulation, the EXECUTIVE SIMULATION [53 is currently being used and data from it should be available at the convention. The EXECUTIVE SIMULATION is interactive and is considered to be more complex even though less decisions are made each period. Primarily, the extra decisions in the V.K. Gadget Company are production decisions in balancing a three-department factory. That was an excellent experience for the students but the external environment which is emphasized in an interactive simulation requires so much time that students would probably not be able to continue with the internal complexity while learning the remaining considerations. Purpose: The purpose of this study is to objectively determine the value of using one or more computerized accounting simulations to prepare people for decision-making in a complex business environment. Exploring Experiential Learning: Simulations and Experiential Exercises, Volume 5, 1978 61 Significance of this study: This study provides information to determine whether or not a simple simulation should precede the use of the V.K. Gadget Company simulation, whether or not a more complex simulation should be developed, and will provide a basis for deducting the relative effectiveness of the use of simulations to prepare people to make business decisions. Need for This Study: The author has proposed the development of a complex simulation which would enable students to begin with a simple model that would grow in complexity with succeeding time periods. The need for such a model has been implied by prior studies which indicate student apprehension and difficulty in mastering more complex models when they begin using simulations. Scope of This Study: This study uses three simulations in a three-course managerial-cost accounting sequence to determine the difference in student performance using the following three sequences of simulation use: GROUP MANAGERIAL ACCTNG COST ACCTNG I COST ACCTNG 2 1 No Simulation No Simulation FINANSIM 2 No Simulation. FINANSIM V.K. GADGET CO. 3 No Simulation V.K. GADGET CO. EXECUTIVE SIMULATION 4 FINANSIM (Two different administrators) Limitations: This study is limited and biased by evaluating only those students at Winona State University, Minnesota, during the 1977-78 period. The only simulations used are FINANSIM, the V.K<, GADGET CO. and the EXECUTIVE SIMULATION. (The EXECUTIVE SIMULATION is not reported in this paper because the simulation will not be complete until March 1978.) The difficulty ranking is subjectively determined by the researcher. Summary: This study presents the results of experiments with progressively complex sequences of simulation usage to determine the effectiveness of the use of a simulation in preparing for a more complex simulation. DISCUSSION OF RELEVANT LITERATURE Brenenstuhl ~[1 p. 81] cited research and the need for objective validation of the effectiveness of the use of simulations. He conducted an experiment with a control group and an experimental group and administered a pre-test, post-test which showed no significant difference between classes which used and did not use the Executive Game Simulation. His evaluation device was not similar to the activities learned with the simulation. This research evaluates the learning from a simulation with a simulation which should overcome the problems that he had. Exploring Experiential Learning: Simulations and Experiential Exercises, Volume 5, 1978 62 Roberts and Field [7, p. 92] had an excellent literature review which further identified a need for objective evidence for the value of simulations. They had an opinion survey of 84 students who had an overwhelmingly positive attitude toward the simulation in relation to other, traditional forms of education. My own experience confirms the strong response of students. Wolfe and Byrne [8, p. 325) asked their students how much cognitive and non-cognitive learning occurred in classes with a simulation as opposed to classes which used the case method and got stronger results in the classes with simulations. Absolutely no numbers are involved in this research and it is apparently inconclusive. In summary, the problems of researching the effectiveness of the use of simulations are: (1) lack of a quantifiable objective, (2) insufficient classes for conducting experiments, (3) a strong preference of students to take the class with a simulation when it is offered, (4) lack of an objective measurement device, (5) absence of a description of the expected results of the use of simulation, and (6) acceptable standards for defining effective research in simulation. DESCRIPTION OF THE RESEARCH The design of this research does not follow classical or pure research patterns but rather reports an implementation of simulations at an institution that had not used simulations previously. Thus, the research was performed retroactively to an actual application and had no bias to the outcome introduced during the simulations. Winona State University has a three-quarter sequence to prepare students for managerial and cost accounting functions and the Certificate in Managerial Accounting. Because of limited data processing resources it was feasible to implement one accounting simulation per quarter and therefore the most senior class was selected for the first simulation. Since students, professors, and data processing personnel all had to be introduced to simulation simultaneously, it seemed logical to implement a simple simulation first. Therefore, the implementation followed this schedule: (The number of students in each class is in parenthesis.) Exploring Experiential Learning: Simulations and Experiential Exercises, Volume 5, 1978 63 The number of students available in each class was minimal and no effort was made to control the quality in any class. They represent the entire population of students at Winona State University taking these courses and not a random sample. Decisions were made at the end of each week throughout each course after the game was introduced. Stock Price Objective: All of the students in all of the classes managed their own individual companies and were told prior to the first decision that their objective was to have the maximum stock price after their ninth period decision. Though other objectives could have been used, it seemed that this was the most all-inclusive measure of performance. Furthermore, it was quantifiable, complex, and readily identifiable with real-world criteria. All students were encouraged to use sensitivity analysis of all variables early in the game to develop decision-making criteria in addition to the traditional managerial accounting techniques. Mother professor at this institution gathered data on many alternative criteria and compared it to stock price and found no significant difference though that finding has not been formally reported in the literature. Statistical Analysis: Data were loaded into a computer using the Statistical Package for the Social Sciences (SPSS) system [6.) The data consisted of 110 cases with the stock price after the ninth decision and identified the simulation used and the class it was used in. Administrator Differences: For the first test, two sections of managerial accounting which were taught by two different accounting professors who both used FINANSIM were analyzed with the SPSS package. Regression analysis, discriminant analysis and analysis of variance found no significant difference at the .025 level between students taught by one professor and those taught by the other. Prediction Ability: When all classes which used FINANSIM were analyzed together, discriminant analysis could group cases to classify only 45% of the cases correctly. Even more important is the group to which they were misclassified: Academic Course Differences: The SPSS package could not classify the student’s progress in academic preparation of managerial or cost accounting, but it predicted that 78% of the students were in the beginning course of the sequence. Furthermore, the students which were classified into the cost accounting classes were beginning managerial accounting students usually. Exploring Experiential Learning: Simulations and Experiential Exercises, Volume 5, 1978 64 Prior Simulation Experience Differences: A Cost Accounting 2 class which performed the FINANSIM simulation when they took Cost 1 was compared with a Cost 1 class which had no prior simulation experience. Both classes used the more complex V.K. GADGET CO. An F-test of the variations of the two groups makes the statistical inference that we are 97% sure that the variations of the two groups are different. T-tests could not establish that the means were different at the 50% level of confidence. No significant difference was found in any of the more complicated SPSS analyses which were performed but the means and standard deviations showed: Rechecking the plots of multiple regression residuals for both FINANSIM and the V.K. GADGET CO., the distribution of cases was found to be narrower with more experienced students as indicated by the statistical analyses. The mean and standard deviation of the three FINANSIM applications were reviewed and found to be: Further analyses were unable to discriminate between the groups because some options of SPSS were not developed at the time the analyses were performed and the discriminate matrix could not be inverted by the Winona State University computer. SUMMARY AND EXPLANATION There was no significant difference between the results obtained when different administrators used the same game in the same course. There was no significant difference when the same administrator used the same game in different courses. And there was a reduced mean and standard deviation when a simple simulation preceded a more complex simulation. The reduced variability expressed by a lower standard deviation is expected from education generally. In finance, it is called risk reduction. Through learning, the number of unknowns is reduced, more accurate expectations are developed, and a greater certainty of goal achievement is attained. This “learning” also tends to reduce “beginner’s luck” as gamblers call it. Beginner’s luck also explains the higher mean scores among newer participants in a business simulation. The surveys of new millionaires in popular magazines frequently discover Exploring Experiential Learning: Simulations and Experiential Exercises, Volume 5, 1978 65 less well educated people who tried a new idea and were successful. Obviously, the probability of being a millionaire is minute and therefore education is useful because it decreases the probability of failure and reduces uncertainty with some reduction of risk- taking that might have inadvertently led to success. CONCLUSION The use of a business simulation reduces the risk of decision-making performance in subsequent business environments. If we accept the witness of experts which have been reported in the literature who tell that simulation is the most realistic and first all-encompassing business experience they have encountered in school, then we can feel reasonably certain that the use of a simulation will provide experience that will transfer to their future jobs. We can also be assured that there will be less risk in their performance and that there will be an improvement in the reliability of business judgments made by persons with business simulation experience. RECOMMENDATIONS Based on the obvious weaknesses of this study in scope, depth, extensiveness, and analytical rigor, it should be replicated with larger classes with control groups and designed research which includes more and different simulations. Secondly, an attempt is needed to identify the point of diminishing returns from the use of increasingly complex simulations. Thirdly, research is needed to identify the validity of final stock price as a measure of performance in simulations. And finally, a simulation is needed which begins at a basic level for the neophyte but grows and expands in complexity, concurrent with growth in the student’s ability. REFERENCES 1. Brenenstuhl, Daniel, “An Experiential Study of Performance in a Basic Management Course,” reported in Simulation Games and Experiential Learning in Action, proceedings of the 1975 Annual conference of the Association for Business Simulation and Experiential Learning (ABSEL), printed and distributed by the Bureau of Business Research, The University of Texas at Austin, Austin, TX 78712. (1976) 2. Brooks, Leroy D., “Flexibility in Simulation Design for Continual Student Motivation,” reported in Simulations, Games and Experiential Learning Techniques: On the Road to a New Frontier, Proceedings of the First National ABSEL Conference, Published by The Center for Economic and Management Research, College of Business Administration, The University of Oklahoma, 307 W. Brooks, Norman, OK 73069. (1974) Exploring Experiential Learning: Simulations and Experiential Exercises, Volume 5, 1978 66 3. Goosen, Kenneth R., Introduction to Managerial Accounting: A Business Game, General Learning Corporation, Morristown, NJ 07960. (1973) 4. Greenlaw, Paul S. and M. William Frey, FINANSIM: A Financial Simulation, International Textbook Company, 666 Fifth Avenue, New York, NY 10019. (1967) 5. Keys, Bernard and Howard Leftwich, The Executive Simulation (Second Edition), Kendall/Hunt Publishing Company, 2460 Kerper Boulevard, Dubuque, IA 52001. (1977) 6. Nie, Norman, C. Hadlai Hull, Jean G. Jenkins, Karin Steinbrenner, and Dale H. Bent, SPSS: Statistical Package for the Social Sciences (Second Edition), McGraw-Hill Book Company. (1975) 7. Roberts, Ralph M. and Steven E. Field, “Using Student Opinions in Evaluating Results with a Business Game,” reported in Simulation Games and Experiential Learning in Action, Proceedings of the 1975 Annual Conference of the Association for Business Simulation and Experiential Learning (ABSEL), printed and distributed by the Bureau of Business Research, The University of Texas at Austin, Austin, TX 78712. (1976) 8. Wolfe, Douglas E., and Eugene T. Byrne, “Research on Experiential Learning: Enhancing the Process,” reported in Simulation Games and Experiential Learning (ABSEL), printed and distributed by the Bureau of Business Research, The University of Texas at Austin, Austin, TX 78712. (1976) Table of Contents Volume 5, 1978 Debriefing: The Key to Effective Experiential Learning A Kiss Before Debriefing Debriefing with Serialized Theory Development for TaskTeam Learning Insights Into Debriefing Experiential Learning Exercises The Two-Step Flow of Experiential Learning: A Preliminary Investigation Programmatic Experienced-Based Learning in an MBA Program The New Research Focus: An Analysis of the Simulation Game User An Analysis of Performance in Simulation Games Compared to Performance on Structured Course Criteria: A Case Study Le's Simplify the Administrative Requirements of Computerized Educational Simulations Assessment of Sex Stereotypes Within Task Group Simulation The Use of Program BAYAUD in the Teaching of Audit Sampling The Effectiveness of Progressively Complex Accounting Simulations in Increasing Decision-Making Performance The Natural Learning Project The Application of a US-USSR Trade Simulation to the Reaching of Business Russian The Use of Semi-Autonomous Work Group in Graduate Management Education: An Australian Experience Evaluation of an Experimental Course in Organizational Behavior for Managers of Japanese Banks Berkshire II: An Experiential Decision Making Exercise: An Exercise in Hegelian Inquiry Creating a Responsible Managerial Experience A Framework for Determining the Pedagogical Value of Simulation Gaming: Implications for Future Simulation Gamin Research Emergent Simulations in Administration Courses Measurement of Administrator Role for Feedback on Structure and Goals An Experiment in Assessing the Theoretical Recognition and Application Leaning Skills of Student Case Writers Some Game Information Systems Experience Using Job Control Language (JCL) for Group and Individual Simulation Considerations for the Use of Computerized Business Simulations Assessing the Effectiveness of Learning Styles as Predictors of Performance within Three Distinct Pedogogic Methodologies Learning Style and Performance A Psychometric Analysis of Kolb's Learning Styles Inventory Learning to Identify and Satisfy consumer Wants: A Classroom Game An Experiential Exercise in Product Benefit Segmentation Sales and Sales Management: A Case/Simulation Approach The SBI Case as Experiential Learning The Use of a Live Case in Teaching Organization Theory and management Principles to Graduate Students The Residency in Hospital and health Administration as Experiential Learning The Extended Live Case Approach to Teaching Marketing Research Experiential Learning in Marketing: Student Consultants The Brazil-Columbia Coffee Marketing Negotiations: An Experiential Exercise Aimed at Highlighting the Dynamics Involved in the International Business Negotiation Process Educational Values of Simulation Gaming Business Simulation Re-Revisited Simulation and Experiential Learning as Human Subject Research An Experiential Evaluation of a Didactic - Experiential Apporach for Teaching personnel Management Experiential Learning in Wage and Salary Administration Integrating Structured Experiences into Personnel Simulation Locus of Control and Performance in a Management Simulation Correlations Between Academic Achievement, Aptitude, and Business Game Performance Suggestions for Student Input of Decisions in General Management Simulations An Exploratory Investigation of Student Perceptions of Computer Simulation as an Educational Tool Time to Sell?: An Experiential Learning Approach to Stock Market Decisions through Interactive Gaming