THE EFFECT OF DECISION FORMAT AND EVALUATION ON SIMULATION PERFORMANCE, DECISION TIME, AND TEAM COHESION Developments In Business Simulation & Experiential Exercises, Volume 17, 1990 170 THE EFFECT OF DECISION FORMAT AND EVALUATION ON SIMULATION PERFORMANCE, DECISION TIME, AND TEAM COHESION William Wellington and A. T. Faria, University of Windsor ABSTRACT The simulation performance of 188 students formed into two decision format groups, regional managers and committee managers, was measured along with CPA, introductory marketing grades, previous simulation performance, total time spent on the simulation decisions, group cohesiveness and other attitude measures. Committee decision format groups outperformed regional decision format groups, spent more time making decisions, and reported more learning benefits. There were no cohesiveness differences between the two decision format groups. INTRODUCTION The interest of simulation game users in improving the effectiveness of this learning technique can be shown by the large and growing body of research covering all aspects of game usage and student characteristics as related to game performance. Variables examined include the personality characteristics of game participants, team size, degree of instructor involvement, previous business experience, grade weighting, degree of formal planning required group selection method, decision time pressure, game complexity, and many others. Comprehensive reviews of this research can he found in Greenlaw and Wyman (1973), Keys (1976), Wolfe (1985), and Miles, Riggs and Schubert (1986). Despite the considerable research, several variables remain largely unexamined One such variable ts the decision format scheme that was tested in the present study. Team decision format was selected for study because it presented an opportunity to examine the effectiveness of a regional management decision approach, an organizational approach often used by business firms, versus the more traditional total team approach to simulation decision making. in addition, ft is one of the few variables over which an Instructor can exert some control in assessing the performance of students and assigning grades. BACKGROUND AND PAST RESEARCH The main focus of the present study was to examine the effect of two distinct decision approaches on overall team performance. One decision approach had all members of the student teams sharing in the making of each periods decisions in group or committee Fashion and submitting one overall set of decisions. This ts generally the most commonly used approach to team simulation decision making. The second approach assigned each member of the student team to a distinct geographic territory of the company’s operation with complete decision responsibility for the assigned territory. The total team decision under this second approach then became the three territory decisions of the three territory managers. Overall team performance was monitored under these two decision approaches. in addition, at the end of the simulation competition, a questionnaire was administered to all students to measure actual company organization, team cohesion, attitudes toward simulation games, and time spent making each periods decisions. As well, student records were examined to gather information on overall grade point averages, grades in an introductory marketing course and performance in a previous simulation competition, to examine the relationship between these variables and current simulation game performance. Decision Format No previous research has examined team/group decision making versus territorial decision making in a team simulation environment. The closest research has been studies of team size and organizational structure. Studies focusing on team size have shown that teams of three or four students tended to outperform smaller and larger size teams, larger teams tended to exhibit more dissension, and smaller teams tended to take less time to make their decisions (Gentry 19R0; Remus and Jenner 1981; Newgren, Stair, and Kuehn 1980; Wolfe and Chacko 1983; and Wilson 1974L Edge and Remus (1984) and Lucas (1979) examined organizational structure. in the Edge and Remus (1984) study ft was found that egalitarian teams exhibited higher ROT performance than hierarchically organized teams while Lucas (1979) reported no performance differences between functionally and hierarchically organized teams. Grade Point Average Simulation game performance has been examined in relation to grade point average (CPA) and other measures of a student Th ability to learn. While mixed, the results from this research have generally been positive. Dill (1961) found no correlation between scores on the Aptitude Test for Graduate Schools of Business (ATGSB) and simulation performance while Potter (1965) obtained slight correlations between ATGSB's anti team ROI. McKenney and Dill (1966) created teams of above average, average, and below average ability based on GPAs and Found, as hypothesized, that the above average teams performed the best while the below average teams exhibited the poorest economic performance. Vance and Gray (1967) and Gray (1972) found correlation’s ranging from .2S5 to .365 between CPAs and team economic performance. Estes (1979) reported a similar correlation. Finally, Wolfe (1978) found strong correlations between both CPA and SAT scores and simulation game performance. Team Cohesion A number of studies have examined team cohesion, or some proxy, as a factor in team performance. Two early studies (McKenney and Dill 1966 and Deep, Bass and Vaughn 1967) reported negative relationships, the less cohesive teams outperformed the more cohesive more recent studies, however, have all reported a positive relationship between cohesion, or some proxy, Developments In Business Simulation & Experiential Exercises, Volume 17, 1990 171 and some measure of team performance (Norris and Niebuhr 1980; Miesing and Preble 1985; Hsu 1984; Gosenpud, Milton and Larson 1985; and Wolfe and Box 1988). METHODOLOGY AND HYPOTHESES The present study examines the effect of decision format (committee/team management versus regional management) on student performance in a marketing simulation game. The study was undertaken in required second year marketing course. All students in the class had previously completed an introductory marketing course. The simulation game used wag Compete: A Dynamic Marketing Simulation (Faria, Nulsen and Roussos 1984). A total of 96 teams of three students each competed in 24 industries of four companies, two committee decision-making companies and two regional decision making companies per industry. Teams were formed by self-selection (the most commonly used format) and competed over twelve decision periods. Once the teams and industries were formed, half of the companies were told that they would be evaluated on overall team earnings relative to the other companies in their industry while half were told that they would be evaluated on the basis of earnings in each region relative to earnings of other companies in their industry in the same region. The simulation used provided printouts of company financial performance by each of three distinct geographic regions. Thus, the regional management teams had to select one student as the manager for each region and the names of the regional managers were given to the course instructor. While the grading approach being used encouraged committee management on the part of some teams and regional management on the part of others, an end of course questionnaire was administered to determine actual decision making organization for each of the companies. Based on the questionnaire results, it was found that some students had organized contrary to their grading scheme and were eliminated from this study for analysis purposes. After eliminating these students, 88 companies remained including 44 operated and graded as committee management teams and 44 operated and graded as regional management teams. The end of course questionnaire administered also gathered information on student attitudes toward the simulation competition, team operation and time spent making each periods decisions. As well, student records were examined to gather information on grade point average, grades earned in the introductory rnarketing course, and performance in the simulation game played in the introductory marketing course (Laptop: A Marketing Simulation, Faria and Dickinson 1987) Earnings performance was the basis selected for evaluation in the simulation competition as it represents a goal oriented Outcome from the simulation experience. A goal orientation is critical to simulation gaming as it is motivating force for competing in the simulation and provides a yardstick for measuring decision making outcomes. A profit maximizing motive was employed to represent the typical real world assumption of economists that businessmen in competitive industries have to try to maximize their profits" (Caves 1982, p. 2). In order to insure that each team had the same motivation to maximize profits, relative earnings per share was selected as the measure of ream performance and was the basis by which team performance grades were assigned. A decision had to be made as to whether data analysis for this study would be on the basis of team or individual performance as some students were graded on the basis of overall team earnings while other students were graded on the basis of their regional performance. As the vast majority of questionnaire and student record data gathered was individual, the decision was made to analyze all data on the basis of the individual student. All data was analyzed using ANOVA and regression analysis. Attitude and Performance Variable Beyond the decision format variable that was experimentally manipulated as described above, thirteen additional independent variables were measured. These consisted of six simulation attitude variables, four performance correlates, and three performance measures. The attitude variables were measured using a seven-point agreement scale anchored by the statements Strongly Agree (1) and Strongly Disagree (7). A team cohesiveness scale (alpha = .9299) was developed based on responses to four Statements involving working relationships among team members; an enjoyment of simulation scale (alpha 7993) was based on three enjoyment statements; while a comparison to other learning methods scale (alpha = .7890) was based on three statements comparing simulation games to lectures, readings and case analyses. In addition, using the same agreement scale, students rated the method of performance evaluation being used, whether their simulation performance was a good reflection of their skills, and their attitude toward group versus individual work. The six attitude variables formed were the result of a Varimax rotated principle components factor analysis of twenty-four attitude items which were measured. This was done to identify meaningful constructs to form post hoc measurement scales and to reduce the number of variables for subsequent analysts (Hair et. al. 1979). The criterion for the number of factors extracted was the latent root criterion with eigenvalue equal to one. A Varimax rotation was used because uncorrelated variables were desired in the subsequent analysis. The Varimax rotation resulted in a six-factor solution, which generally met the criteria for simple structure. Four other principle components factor solutions were undertaken to identify a set of factors that made the most theoretical sense (Hair et. al., p. 231). Surrogate, variables for subsequent analysis were selected to represent each of the six factors. Where three or more items loaded highly on a particular factor, scales were formed and examined for reliability using coefficient alpha. If the value of alpha was not .80 or higher (Nunnally 1978, p. 245), the single item with the highest loading was selected to represent the scale and factor from which it was derived. The seven final data items gathered included each students current grade point average, grade in an introductory marketing course, grade received in the simulation game played in the introductory marketing courses total current simulation decision time, cumulative earnings per share in the current simulation competition, relative regional contribution margins, and relative performance grade as a percentage of the industry leader. Developments In Business Simulation & Experiential Exercises, Volume 17, 1990 172 Hypotheses The nature of the data gathered, results from past research, and intuitive judgement, led to the formulation of three hypotheses. H1: Mean performance as measured by cumulative EPS, relative regional contribution margin, and relative total FPS will be significantly higher for the committee decision format then for the regional decision format. It was judged that the committee form of decision making would result in more collective effort in making each period’s decisions and thus would produce better results. The reasoning, essentially, was that three heads are better than one. Further, past research (Gentry 1980) has shown that teams of three students have outperformed smaller teams. In terms of the current research, the committee groups might he viewed as teams of three competing against single regional managers. H2: Cohesiveness among regional decision format team members will be significantly greater than among committee decision format team members. The regional manager approach would result in less need to work with other team members to formulate each period's decisions and, thus, less friction should result. In the committee format, all group members would be required to share in the making of all decisions during each period, often having to reach compromise positions, with the resulting conflict’s. Further, past research (Gentry 1980) has shown that larger groups result in more ream dissention. H3: Total decision making time will he significantly greater for the committee decision format than for the regional decision format. Total decision making time should he less under a regional decision format as each regional’ manager is responsible for a certain portion of the total team decision and does not have to discuss these decisions or reach a compromise with other team members. further, past research has shown that smaller teams tend to take less time when making simulation decisions than larger teams (Gentry 1980). FINDINGS The overall study findings are presented in Tables I through 3. The ANOVA results of the decision format versus the thirteen dependent variables are presented in Table 1. Regression analysis results for four traditional a priori correlates of simulation performance from the literature are reported in Table 2 and a regression analysis of the attitude correlates of simulation performance are reported in Table 3. As shown in Table I, the committee groups exhibited significantly higher (p < .05) earnings performance than the regional manager decision groups. As further shown, there was no significant differences in the decision format groups in terms of GPAs, grades in introductory marketing, previous simulation game performance, or team cohesiveness. As such, the difference in earnings would seem to be solely attributable to the decision format rnanipulation. Thus, hypothesis one ts accepted. As further shown in Table 1, there is also a significant difference (p < .05’) in total decision making time between the decision format groups with the committee groups taking an average of approximately 94 minutes to make each decision and the regional manager groups approximately 18 minutes. Thus, hypotheses three ts also accepted. Finally, while the results with regard to team cohesiveness were in the direction hypothesized, they were not statistically significant (see Table 1). Therefore, hypothesis two ts not accepted. DISCUSSION The results from this study would suggest that a team or committee management approach produces better earnings performance than a regional manager decision format. This conclusion is further supported by the face that both decision format groups agreed that team work is more beneficial than working alone,. From a teaching perspective, it is Important to note that the committee format groups rated their learning experience from the simulation game relative to cages, lectures and readings higher than the regional format groups. While the committee groups also reported that they enjoyed the simulation more, rate the method of simulation evaluation more positively, and felt that the simulation was a better reflection of their abilities than the regional groups, this may well be a reflection of their superior simulation performance rather than the decision format. This ts supported by the regression results reported in Table 3 showing these three variables to he significant correlates of performance. It would appear that there is some synergy to he gained from team decision making. The committee groups exhibited significantly higher earnings performance even though there was no difference in CPA, grades in introductory marketing or pervious simulation performance between the decision format Developments In Business Simulation & Experiential Exercises, Volume 17, 1990 173 groups. Further supporting this conclusion is the fact that none of these variables emerged as significant predictors of performance (gee Table 2). CONCLUSIONS Student teams Operating in the more traditional decision making format of committee managers outperformed teams operating as regional managers. In addition, committee groups reported greater learning benefits from the simulation experience relative to assess, readings and lectures, and more enjoyment form the simulation experience. At the same time, there was no loss of cohesiveness brought about by the need to reach compromises n some decision areas. Decision making time was lower for the regional management groups. The results form this study would suggest that teachers wishing to achieve the best performance results from their simulation teams while also providing a more positive simulation environment should use the traditional committee or group decision making approach. REFERENCE Caves, R. 91982), American Industry: Structure, Conduct, Performance, Englewood Cliffs, NJ: Prentice-Hall, Inc. Cohen, J. and P. Cohen (1983), Applied Multiple Regression/ correlation Analysis for the Behavioral Sciences, Hillsdale, NJ: Lawrence Erlbaum Associates. Deep, S.D., B.M. Bass and J.A. Vaughn 91967), “Some Effects on Business Gaming of Previous Quasi-T Group Affiliations”, Journal of Applied Psychology, 51 (No.), 250-372 Developments In Business Simulation & Experiential Exercises, Volume 17, 1990 174 Dill, W.R. (1961), “The Educational Effects of Management Games’, in W.R. Dill, J.R. Jackson and J.W. Sweeney (eds.), Proceedings of the Conference on Business Games, New Orleans: Tulane University, 61-68. Edge, A.E. and W. Remus (1984), The impact of Hierarchical and Egalitarian Organization Structure on Group Decision Making and Attitudes, in D.M. Curie and J.R. Gentry (eds.), Proceedings of the Eleventh Annual Conference of the Association for Business Simulation and Experiential Learning, 35-39. Estes, J.E. (1972), "Research on the Effectiveness of Using Computerized Simulation in the Basic Management Course’, in S.C. Certo and D.C. Brenenstuhl (eds.), Proceedings of the Sixth Annual Conference of the Association for Business Simulation and Experiential Learning, 25-28. Farla, A.J. and J.R. Dickinson (1987), Laptop: A Marketing Simulation, Piano, TX: Business Publications, Inc. Faria, A.J. R.O Nulsen and 1).S. Roussos (1984), Compete: A Dynamic Marketing Simulation, Piano, TX: Business Publications, Inc. Gentry, J.W. (1980), The Effects of Group Size on Attitudes Toward the Simulation, in D.C. Brenenstuhl an W.D. Biggs (eds.), Proceedings of the Seventh Annual Conference of the Association for Business Simulation and Experiential Learning, 35-39. Gosenpud, J., C. Milton and A. Larson (1985), “Predicting Performance over the Course of the Simulation”, in J.W. Gentry and A.C. Burns (Eds.), Proceedings of the Twelfth Annual Conference of the Association for Business Simulation and Experiential learning, 5-10. Gray, C.F. (1972), Performance as Criterion Variable in Measuring Business Gaming Success: An Experiment with a Multiple Objective Performance Model, Southeastern American Institute for the Decision Sciences Conference, 15-39. Greenlaw, P.S. and F.P. Wyman (1973), “The teaching Effectiveness of Games in Collegiate Business Courses”, Simulation & Games, 4 (No. 3’), 259-2Q4. Hair, J.F., R.E. Anderson, R.L. Thatham and B.J. Grablowsky (1979), Multivariate Data Analysis, Tulsa, OK: petroleum Publishing Company. Hsu, T. (1984), A Further Test of Group Formation and its Impact in a Simulated Business Environment, In D.M. Currie and J.W. Gentry (eds.), Proceedings of the Eleventh Annual Conference of the Association for Business Simulation and Experiential Learning, 6-9. Keys, 8. (1976), A Review of Learning Research in Business Gaming", in R.H. Sord (ed.), Proceedings of the Third Annual Conference of the Association for Business Simulation and Experiential Learning, 91-93. Lucas, H.C. (1979), “Performance in a Complex Management Game”, Simulation & Games, 10 (No. 1), 61-74. McKenney, J.L. and W.R. Dill (1966), ‘Influences on learning in Simulation Games”, American Behavioral Scientism 10 (No. 1), 28-32. Miesing, P. and J.F. Preble (1985), “Group Processes and performance in a Complex Business Simulation", Small Group Behavior, 16 (No. 3), 325-338. Miles, W.G., W.D. Biggs and J.N. Schubert (1986), Student Perceptions of Skill Acquisition through Cases and a General Management Simulation’, Simulation & Games, 17 (No. 1), 7-24. Newgren, K.E., R.M. Stair and R.R. Keuhn (1980), “The Relationship between Group Size and the Learning Curve Effect in Gaming Environment, in D.C. Brenenstuhl and W.D. Biggs (eds.), Proceedings of the Seventh Annual Conference of the Association for Business Simulation and Experiential Learning, 203- 205. Norris, T). R. and R.F. Niebuhr (1980), Group Variables and Gaming Success, Simulation & Games, 11 (No. 6), 301-312. Nunnally, J.C. (1978),, Psychometric Theory, New York: McGraw-Hill Publishing Company. Remus, W. and S. Jenner (1979), “Playing Business Games: Attitudinal Differences Between Students Playing Singly and as Teams, Simulation & Games, 10 (No. 1), 75-86. Vance, S.C. and C.F. Gray (1967), “Use of a Performance Evaluation ton Mode’ for Research in Business Gaming”, Academy of Management Journal, 10 (No. 1’), 27-37. Wolfe, J. (1978), Correlation’s Between Academic Achievement, Aptitude, and Business Game Performance, in D.C. Brenenstuhl and S.C. Certo (eds.), Proceedings of the Fifth Annual Conference of the association for Business Simulation and Experiential Learning, 316-324. Wolfe, J. (1985), The Teaching Effectiveness of Games in Collegiate Business Courses; a 1973-1983 Update, Simulate ton & Games, 16 (No. 3), 251-288. Wolfe, J. and 1.1. Chacko (1983), “Team-Size Effects on Business Game Performance and Decision-Making Behaviors”, Decision Sciences, 14 (No. 2), 121 133. Wolfe, J. and T.M. Box “Team Cohesion Effects on Business Game Performance”, Simulation & Games, 19 (No. 1), 82-98. Table of Contents Volume 17, 1990 The Impact of Decision Support Systems on the Effectiveness of Small Group Decisions - Revisited The Relationship Between Financial Performance and Other Measures of Learning on a Simulation Exercise Use and Effectiveness of an Analogy-Based Expert System Suggestions for Computerized Business Authors Dealing with Power: An Experiential Exercise Using Movie and Personal Diary Analysis Techniques A Model for Developing Student Skills and Assessing Outcomes Through Outdoor Training Computer-Aided Exercises Versus Workbook Exercises as Learning Facilitator in the Principles of Marketing Course An Exposition of Guilford's Si Model as a Means of Diagnosing and Generating Pedagogical Strategies in Collegiate Business Education Formal Planning and Simulation Team Performance: A Cross Sectional Approach Cases: Real Organizations in Real Time in the Classroom An Empirical Investigation of the Internal Validity of Marketing Simulation Game An Empirical Evaluation of the Pedagogical Value of Playing a Simulation Game in a Principles of Marketing Course Factors Affecting Effective Teaching of Strategic Planning: Some Preliminary Evidence An Experiential Exercise for Learning About the Relationship Between Organizational Form & the Project Management Process Accounting Communication Skills can be Taught in the Auditing Course Modeling Cost Functions in Computerized Business Simulation: An Application of Duality Theory and Sheppard's Lemma A Life Cycle Analysis of Decision Making for a Strategic Management Team What's the Problem? A Dynamic Model for Teaching Problem Solving Skills Experientially International Currency Fluctuations: Money$im, A Simulation Superstores: A Specialized Retailing Simulation Within a Specialized Marketing Curriculum Factors Affecting Student Perceptions of Learning in a Business Policy Game VC + EL = VL The Name Game: An Experiential Exercise in Intergroup Relations The Effects of Experiential Accounting Work Experience on Student Performance in Intermediate Accounting Courses The Results of Using the Experiential Activity Group Performance Evaluation in a Business Policy Setting Using a Legal Database to Describe the Legal Environment of Marketing (and Business) Matching Environmental Uncertainty and Organizational Configuration An Instructional Computer Simulation of Tampering in QC Executive Evaluation of Student Learning in the Looking Glass Simulation Group Personality Composition and Total Enterprise Simulation Performance An Expert System for Selecting Analytical Techniques for Analyzing Marketing Research Data Effects of Cognitive Styles on Responses in an In-Basket Simulation A Psychometric Analysis of Kolb's Revised Learning Style-Inventory Selecting and Developing Experiential Exercises Using Movies Application of a Real-World Strategic Management Model in the Classroom Demand Equations which Include Product Attributes Consumption as the Objective in Computer-Scored Total Enterprise Simulations The Effects of Decision Format and Evaluation on Simulation Performance, Decision Time, and Team Cohesion The Effects of Computer Related Assignments on Student Performance in Business Administration The Money Game: A Dynamic Simulation Including Random Shocks for Money and Banking Courses Methods for Evaluating Performance on Business Simulations: A Survey The Effects of Synergogy on the Policy Course: Significant Improvements in Student Learning and Teacher Evaluation Conditions and Outcomes of Trust in a Two-Person Bargaining Exercise Bankgame Enhancing Computer Business Simulation with the Use of VGA Graphics An Experiential Approach to Entrepreneurship An Advanced Simulation Method (ASM) for Multiple Objective Problems The Influence of Experiential learning Techniques on Student Recognition of Non-Primary Learning Styles Negotiating Mergers and Acquisitions: A Cocktail Napkin Approach Identification of Unintended Effects in Experiential Laboratory Exercises An Experimental Comparison of Paper and Pencil and Computer Aided Decision Support Tools Porting a Simulation from the IBM World to the Macintosh World A Transaction Cost Analysis of Experiential Learning The Development of Experiential Exercises for Courses in Entrepreneurship and Small Business Management The Assessment Center as and Experiential Classroom Exercise Pricing Strategy Algorithms for Playing Business Simulations Organizational Structures for International Operations: An Experiential Activity Simulation Emphasis in the Business School Capstone Course An Integrated Approach to Computerizing the Business Curriculum Cross-Cultural Business Negotiations Exercise Organizational Socialization and Gender Differences in Students at Work Understanding Student Work Experience: A Content-Analytic Approach Introducing Executive MBA Programs with Management Games An Analysis of Improvement in Business Decision Outcome with Sequential Use of Two Simulation Games Teaching Business Policy Utilizing Mass Lecture and Individual Case Labs Potholes Along the Road to Evaluating Learning Outcomes: The Case of Outdoor Management Training Experiential Learning for Interior Design Students: Using CADD, Lotus 1-2-3, and Wordperfect Cognitive Learning Using a Computer-Based, Qualitative Interactive Business Simulation Sex Discrimination: Does the Woman get the Job or Does the Best Man Win? A Search for Visual Aids to Support Experiential Learning Through the 1990's An Experimental Analysis fo the Effectiveness of Student Role-Playing in Sales Training How to Have Students Learn from their Term Projects Self-Evaluation Exercise (SEE): An Assessment of Class Contribution A Study of the Influence of Team Formation on Attitudes and Performance in Management Games A Hardware Based CIM Simulation Laboratory Model Test Substance Abuse in Organizations Micro Computer Training Models Teaching Forecasting, Cash Budgeting and Inventory Model Building Using SBTools A Comparison of the Effects of Experiential Learning Activities and Traditional Lecture Classes An Adjunct Writing Instruction Assistant, The Computer; With an Illustration Classroom Software for ABC Analysis Utilizing Information Processing Technology to Enhance the Business Policy Simulation Experience Progressive Cases Realistic Job Previews Vs. Traditional Job Previews: Experiencing the Differences and Understanding the Consequences Investment Analysis Using the Pragmatic Multiplier Approach A Computer Simulation Interface for Competitive and Firm Analysis Self-Assessment of Ethical Decision Making Predispositions Preparing Managers for Overseas Assignments An Inquiry into Japanese Marketing: Workshop on Teaching Japanese Marketing The Performance Appraisal Feedback Interview: A Role Play for Human Resources Management Teaching Counselor Selling techniques Using Experiential Techniques to Teach International Topics International Management Simulation Gaming: Current Status and Future Developments A Realism Comparison of Simulation Technologies/Methodologies A Time-Efficient Game to Illustrate Concepts Taught in Management Courses and Management Development Programs Teaching the Management of Technology The Concept of Face and the Applicability of Experiential Exercises in an Oriental Culture's