PERFORMANCE AND ATTITUDINAL AFFECTS OF A DECISION SUPPORT PACKAGE IN A BUSINESS GAME Developments in Business Simulation & Experiential Exercises, Volume 13, 1986 221 PERFORMANCE AND ATTITUDINAL AFFECTS OF A DECISION SUPPORT PACKAGE IN A BUSINESS GAME Bernard Keys, Georgia Southern College Max Burns, Georgia Southern College Tom Case, Georgia Southern College Bob Wells, Georgia Southern College ABSTRACT Traditional wisdom from the MIS/DATA Processing field suggests that Decision Support Systems (EWS) improve corporate decision making and planning. This paper investigates this question utilizing four business policy sections, two with EWS packages and two without. It was found that the use of EWS produced a more positive overall attitude toward game play, a greater willingness to perform planning, and a better understanding of game play and planning, than handscored work sheets. It could not be determined that a significant difference occurred in the performance of teams with and without the EWS, although the evidence suggests a change in this direction. STUDY METHODS Students in four independent business policy sections played the Executive Simulation (ES) for 9 simulated quarters; one trial decision set and two simulated years of operation. One professor taught two sections and two different professors taught the other two sections. The sections were designated A, B, D and C. Sections C and D were taught by the same professor. Sections B and D were randomly selected to receive the EWS with one random choice being made from A and B and one from C and D. For simplification we will hereafter designate the sections receiving the EWS as BW and DW. The sections utilizing handscored worksheets will be designated A and C. Sample Characteristics Table 1 describes the demographic characteristics of each business policy class. An analysis of variance across all sections indicates that there were no differences in demographic characteristics of sections except on Sex and years of employment worked part time. Therefore a Duncan’s Multiple Range test was run on these two characteristics. The results of this test are shown in Table 2. From Table 2 it can be seen that there are significantly more females than males in section C than in section A. Section BW also had significantly more female students than did section A or section rA4. Under part time employment it can be seen from Table 2 that section A had significantly more years of part time employment than section C or DW. While the class composition by sex and part time employment could affect game and attitudinal results, the differences cited are not significantly different on any characteristics across the two classes taught by the same instructor, C and DW. None of the instructors of the policy sections evaluated had had previous business game experience. The game was introduced by the authors and scored by the same graduate assistant for all four classes. All game decisions and analysis was done outside of class. All questions concerning the ES were referred to the ES author or to his graduate assistant. No attempts were made to integrate the game with other course materials. The instructors in the classes referred all game questions to the game author outside of class. Students were taught to use the EWS by a co-author. Step-by-step instructions for using handscored work sheets were provided in the student game manual. Each professor utilized different materials for the case and strategic management portion of their class. The following summary will describe these materials. Section A used the text by Thompson and Strickland, entitled, ic Management, Third Edition, by BPI. Cases were assigned daily and one case was written from outside research. The game was used as a class discussion, but all game decisions were made outside of class as in the other sections. Daily case discussion composed 80 percent of total grade and a comprehensive case write-up and a final exam counted 20 percent. The ES was discussed during one class period and it was implied that it would count as part of the case discussion grade. Section BW used the text by Byars, entitled Strategic Management, by Harper and Row. Readings and cases were assigned daily and 7 cases were required during the semester. Four exams were given over the text chapters. Two case analyses were required of each student in addition to daily preparation. Although the game was assigned a grade of 10 percent of the course no specific game grade was given and students received a game grade of pass/fail. Section C and DW used the text by Wheelen and Hunger entitled Strategic Management and Business Policy, published by Addison/Wesley, 1983. Daily case assignments were made and a group case project was assigned. The same groups assigned to cases worked together in making game decisions. The group case and peer ratings counted 20 percent of the grade assigned in the course. While no specific grade was assigned the game, it was implied that the game would count as part of the group grade. Two hourly exams counted for 50 percent of the grade. Class work counted 15- 20 percent, and group presentations counted 15- 20 percent and a final project counted 10-15 percent. Two major tests of EWS affects were administered at the end of the game to all participants: an eleven item questionnaire with subquestions on the planning sheets and functional areas (see Table 3 and 4) and a ten item attitude questionnaire utilizing a five point Likert scale (see tables 5 and 6). Performance with and without the EWS was evaluated by an analysis of variance performed, utilizing ROI for each team for each quarter (see Tables 7 and 8). The formula used for ROI is earnings before interest and taxes divided by total assets. Developments in Business Simulation & Experiential Exercises, Volume 13, 1986 222 HYPOTHESES TESTED The following hypotheses were formulated to test the affects of attitude development and performance effects of the use of decision support systems in a game-the EWS. H1 The use of the EWS produces a more positive overall attitude toward game play than does the use of the HWS. H2 The use of the EWS produces the willingness to perform more planning for game play than does the use of the HWS. H3 The use of the EWS produces a better understanding of game play and planning than does the use of the HWS. H4 The use of the EWS produces a higher ROI than does the use of the HWS. Hi was tested by the ten attitude questionnaire. H2 and H3 were examined by the eleven item questionnaire. H4 was tested by the analysis of ROI across the groups with and without the EWS. RESULTS Results from Questionnaire and Attitude Survey Table 3 presents the results of an analysis of variance between the two sections playing the game with the EWS, BW, and DW, and the two sections playing the game without the EWS, A and C, on the eleven item questionnaire. The means for every question answered by the EWS groups BW and DW were more positive than the means for questions answered by the HWS groups, A and C. The difference in the means was significant at the .05 level and more positive for all questions except questions 1, 3 and 6. EMS user sections did not think their teams were more successful, did not make significantly more alternative decisions than non-users, and did not meet significantly more often than non-users. On the other hand EWS user groups were found to be significantly different from nonuser groups in that they: completed worksheets more frequently, reached a consensus decision more often, spent more time on each decision set, found the simulation reinforced concepts taught in the policy course to a greater extent, thought they had a greater improvement in understanding, used financial ratios in alternative evaluation more often, and believed more strongly that their performance would improve if they played the game again. Table 4, presents only the data on the questionnaire pertaining to groups taught by one instructor, C and DW, and hence more likely to represent only difference stemming from the use of the EWS. In these two sections the direction of the larger means is the same as that discovered across all four sections. However, there are more questions that did not produce significant differences at the .05 level. As in Table 3, questions 1, 2 and 6 were not significant. However, question 11, belief that performance would improve if they played the game again, did not prove to be significantly different for the two groups. The results of the attitude questionnaire items are presented in Table S for all students, with analysis of variance performed for sections without the EWS, A and C, across the sections with the EWS, sections BW and DW. The students responded to each question on a 5 point Likert type scale ranging from Strongly Agree to Strongly Disagree. All questions elicited means that were more positive for the students using the EWS than for ones without it. Additionally, all F scores were significant at beyond the .05 level of significance. Students seem to enjoy the game more using the EMS. With the EWS they considered the game to be less work and less difficult. The use of the EWS apparently caused them to view the planning and the learning process more positively. The teams utilizing the EWS say they did more planning and less guessing and that they completed the planning sheets more often than reported by the teams without the EWS. EWS users are more interested in playing the game again, believe that they understand the planning process better, and are more positive toward the ES as a learning experience. Table 6 presents the findings for the same attitude questionnaire, but with analysis of variance reported only across the two groups taught by one instructor, C and DW. In every case the direction of attitude shift was in the direction hypothesized. On questions that were significantly different, the EWS section answered the questions in a more positive manner. Results of Performance Analysis Tables 7 and 8 present the results of analysis of variance across teams with and without the EWS on ROI for each quarter of the ES game play. In every case BW and DW sections produced ROIs that exceeded those of A and C. However, the differences were significant only in quarters 1 and 8 and for the over-all game ROI. The same pattern was discovered for ROI Mean differences between sections C and DW, the two sections taught by the same instructor. All mean ROI’s for the section using the EMS were greater, but only in quarter 6 and for the overall mean were significant differences found. DISCUSSION AND CONCLUSIONS Although all mean questionnaire scores across the sections with and without the EWS supports Hi, Hi cannot be accepted without qualification on this basis alone, since the populations were found to be significantly different in sex and part time employment composition and since they were taught by different instructors using different materials. A further test of Hi across the means of the two policy sections taught by one instructor, C and DW (Table 6) produced means in the expected direction and all ten items showed significant differences at greater than the .05 level. Therefore Hi is accepted. In a test of H2 and H3 all means were found to be significantly different and in the expected direction, except means for 3 questions which were not significant. Again, H2 and H3 are not accepted unqualifiedly because of sex and part time employment differences. A further test utilizing only sections C and DW produced means in the direction hypothesized, except for four questions that were not found to be significantly different. Since the significant findings dealt with the planning question of H2, H2 is accepted. The significant findings also dealt with an understanding of game play and planning, therefore, H3 is accepted. H4 was tested by analysis of variance of mean ROI’s for each quarter. All of the ROI differences across the four sections A and BW and C and DW (Table 7) Developments in Business Simulation & Experiential Exercises, Volume 13, 1986 223 and across the two sections, C and DW (Table 8) were found to lie in the expected direction and the mean differences seem to be growing more significantly different as the game progressed. The overall mean was also significantly greater for the EWS group. However, H4 cannot be accepted from the test of all 4 sections because of sex and part time employment differences. In Table 8 dealing only with C and DW the mean ROI differences support 114, but only one quarter and the overall mean proved significantly different. H4 must be considered nonconclusive. The present study indicates that DSS packages such as the Electronic Work Sheets used with Executive Simulation are useful in that they produce a positive attitude toward game play by students and tend to cause them to do more planning. Students indicate that they have a better understanding of game play with the EWS than with hand scored planning sheets. There was some indication that the use of the EWS produces superior performance on the part of students than they can obtain without them. However, the results were inconclusive in this area. A future research design should utilize only one instructor, enroll teams across classes and industries, and place a heavier grade on game performance. Developments in Business Simulation & Experiential Exercises, Volume 13, 1986 224 Developments in Business Simulation & Experiential Exercises, Volume 13, 1986 225 Developments in Business Simulation & Experiential Exercises, Volume 13, 1986 226 REFERENCES Note: Review of Literature omitted from paper. Selected references shown below. Entire paper available on request. 1. Athappily, Kuriakose “Successful Decision Making Starts with DSS Evaluation,” Data Management, February, 1985, 24-28. 2. Keys, Bernard and Leftwich, Howard, The Executive Simulation, Third Edition, Dubuque, Iowa, Kendall Hunt Publishing Company, 1985. 3. Markulis, Peter K., and Strang, Daniel R. “The Use of Decision Support Systems (DSS) and Operations Research/Management Science (OR/MS) Techniques to Enhance the Learning Experience of Students Participating in Computerized Simulations,” Developments in Business Simulation Experiential Exercises, Volume 12, Proceeding of the Twelfth Annual Conference of the Association for Business Simulation and Experiential Learning, 30-34. Table of Contents Volume 13, 1986 How to Use Microcomputers in Human Resource Management Courses What's My Line? An Exercise in Job Analysis, Description and Classification Relationships Between Team Cohesion Dimensions and Business Game Performance Four Factors Affecting Group Performance in Business Policy Simulations Enhancing Strategic Management Goal-Setting Skills in the Business Policy Course Using a Business Simulation in the Principles of Management Course - Learning Outcomes and Perceptions Personal Computers: Drexel's Experience Using Psychological Type to Enhance Negotiation Skills The Effects of Conflict Handling Styles on Negotiation Behavior: An Analysis of an Experiential Exercise An Experiential Test of Bargaining Decisions and Representation Self-Grading of Examination by Students, A Viable Alternative to Traditional Procedures Perceived Instructor Enthusiasm and Student Achievement An Experiential Approach to Teaching Subordinate-Oriented Communication Assessing the Effects of a Computerized Study Guide in Macroeconomic Principles: A Statistical Analysis A Sales Management Simulation for the PC an Integrative Tool for the Sales Management Course Teaching Re-evaluation of Salespersons Through the Use of a Simulation Game A Test of Student Performance and Attitudes and Performance Under Varying Game Conditions Developing and Testing Airways: A Marketing Simulation Experiential Learning Revisited: Some Thoughts on Designing More Adaptive Management Education Programs Developing the Competencies of Creativity and Accurate Self-Assessment Self-View and Norms: Some Longitudinal Laboratory Results Strategic Planning with an Experiential Case The ABSEL Research Consortium: Preliminary Steps and Position Papers An Analysis of ABSEL Conference papers (1974-1985) The Creation and Operation of a Data-base System for ABSEL Research Funding for a Research Consortium on Simulation and Experiential Learning Simulation - Indoctrination or Learning The Dilemma In Evaluating Classroom Innovations Personality Variables on Group Cohesion, Team Participation and Total Learning Justifications for and Problems in Developing and Using Computerized Experiential Activities Management Competency Models and the Life-Long Learning Project: What Role for ABSEL External Validity of Business Games Restructuring the University - An Experiential Exercise The Surrogate Class Exercise Images of Effectiveness: A Classroom Exercise The Algebran Industry: A Tutorial Framework Another Chance: A Multi-Leveled, Macro Business Game for Micro-Computers A Water Quality Management Simulation Game Acquisitions? Divestitures? Progress Report on a Conglomerate Game Enterprise: A Multi-Purpose Management Development Simulation Marketer: A Microcomputer Simulation in a High Tech Industry A Pedagogical Analysis of Alternative Tax Elections for Exploration Costs in the Mining Industry Using of Multiple Microcomputers Application Programs to Teach Fundamental Business Concepts and Practices Demonstrating the Role of Simulation in Strategic Operations Planning: A Case Study in Bank Check Processing Location Analysis The Subjective Side of the Decision Support System: A Pitfall for the Panacea A Group-Based Procedure for Revealing Judgmental Heuristics and Biases An Experiential Exercise in Bayesian Decision-Making Values for Selected Parameters in Physical Distribution Simulation and Games Computerized Business Management Simulations for Tyros The Management Decision Laboratory at New York University An Example: The Use of Management Games on Microcomputers by Computer Novices Enhancing Mainframe Simulation Via Microcomputers: Designing Decision Support Systems Integrating Personal Computers Into a Course as a Decision Support Tool A Decision Support System for Capital Funds Forecasting Performance and Attitudinal Affects of a Decision Support Package in a Business Game Using the Real World: Interviews with Woman Mangers The Cooperatively-Authored Student Handbook International Buying: An Experiential Exercise The Research/Teaching Interface: Turning a Pretest into an Experiential Exercise Building Microcomputer Business Simulations Simulation with Discrete and Continuous Mathematical Modeling An Interpolation Approach to Developing mathematical Functions for Business Simulations The Design of Simulation Models Using Thought Organizers The Assessment Center as a Teaching/Learning Device An Approach to Meeting the AACSB Guidelines for Introducing Skills and Personal Characteristics (SACs) Using an Interclass Simulating and Role-Playing Pedagogy Experiential Learning about the World of Work: A Program for Primary and Secondary School Educators Assessment of Relevant Personal Characteristics of Potential Entrepreneurs