PREDICTING SIMULATION PERFORMANCE: DIFFERENCES BETWEEN GROUPS AND INDIVIDUALS Development In Business Simulation & Experiential Exercises, Volume 18, 1991 44 PREDICTING SIMULATION PERFORMANCE: DIFFERENCES BETWEEN GROUPS AND INDIVIDUALS Jerry J. Gosenpud, University of Wisconsin-Whitewater John B. Washbush, University of Wisconsin-Whitewater ABSTRACT This study explored different antecedents and their relationship to simulation performance when the game was played in teams versus played by individuals. The method was for two sections of undergraduates to play a simulation in teams and two others in the same course in the same university play much of the game as individuals. The results showed that university GPA and academic major predicted performance for individual players but not for teams while carefully choosing teammates varied with performance for teams but not for individuals. BACKGROUND In a review of the research on predicting performance in whole enterprise simulations, Gosenpud (1987) concluded that no independent variable consistently predicts simulation performance. One reason may be that situational conditions affect how independent variables influence performance. One such condition is whether or not the game is played in teams. It is contended here that participation in teams could counteract the influence of background factors such as GPA or academic major. Perhaps individual background factors are more likely to predict performance with individual play than with group play. There is evidence supporting this notion. Norris and Niebuhr (1980), in a study of student teams, concluded that team cohesion correlated significantly with performance while GPA did not. Gosenpud and Miesing (1984), also studying teams, found that six of nine variables loading on performance in the final regression equation were team related. Of the three variables that were significant, two were team related and accounted for 17% of performance variance. Only two of the nine variables in the final equation, accounting for less than two percent of the variance, were background characteristics of individuals. The results of predictive studies given individual play are different. Vance and Grey (1967) and Wolfe (1975) found strong correlations between previous academic record and performance. Gosenpud (1989), in a study in which students began simulation play in teams and finished as individuals, found that three of the five variables loading on performance were individual background factors, measurable before the game began. These accounted for 15% of performance variance. Only one of the five was team related, accounting for less than 3% of the variance. The purpose of this study was to test the hypothesis that the variables which significantly predict performance for team play are different from those which significantly predict performance for individual play. Specifically, it was hypothesized that: (1) Variables influencing performance when the game is played in teams are different from those influencing performance when the game is played by individuals. (2) The individual background factors of academic ability, academic interests, and initial motivation to play the simulation show a stronger statistical relationship with simulation performance when the game is played by individuals than when it is played in teams. METHOD Research Design The ideal research design would entail randomly assigning some subjects to simulation play as (and against) teams and others from the same population to play as (and against) individuals. However, this is pedagogically impractical. If students began play as individuals, most would find the simulation too novel an experience and consequently fail to learn anything substantial during initial encounters. Early performance would depend on ability to adjust or on past experience with similar simulations. Students need to work directly with someone, either colleagues or authorities, during the first several decisions, in order to understand play and relationships between decisions and consequences. Therefore, all subjects in this study experienced group play. Student teams played the game for five weeks. At the five- week point, teams from two randomly selected sections were abandoned and these students continued as single-member firms using their team’s past performance as a historical basis. Although market trends for each of the new industries were based on the old industry, new industry curves differed from each other. Students from the other two sections continued in their teams. Subjects The subjects for this study were 124 business major seniors from four sections, taught by the same instructor, of the required, capstone Administrative Policy class at a medium sized state university with a large business program. Of 124 students enrolled, 95 were graduating seniors and 29 were first-semester seniors. The Simulation Stratplan (Hinton & Smith, 1985) was used in this study because of its flexibility for selecting various levels of complexity and environmental dynamics. The scenario chosen created a domestic environment with two products and three markets. Although market growth varied by product and area, product demand growth for each market was available to students at the beginning of the simulation. During each round of play, students made 68 decisions including how much to produce and where, sales branch expenditures, changes in capital structure, an(I amounts invested in technology and product improvement. Simulation performance was 25% of the course grade. Performance was measured by net income (50%), return on equity (30%), stock price (10%), and market share (10%). Peer evaluations were used when students played the entire simulation in teams. Variables Independent (Individual). Attributes of individuals Development In Business Simulation & Experiential Exercises, Volume 18, 1991 45 expected to predict performance consisted of academic ability, academic interests, and initial motivational factors. Academic Ability. This study included academic ability because it has predicted success in many settings, including graduate school (Webb, 1957), on a land survey project (Terborg, Castore and DeNinno, 1976), in overall careers (Korman, 1970), and in computerized simulations (Vance and Gray, 1967; Wolfe, 1976). This study used the student’s overall college grade point average as a measure of academic ability. Academic Interests. Academic Interests was included because performance in academic settings has been found to vary with interest in subject matter (Sjoberg, 1984; Skaalvik, 1983). This study used academic major as the measure of academic interests because simulation performance has been found to vary with academic discipline (Vance and Gray, 1967; Wolfe, 1976). Motivation. Motivation was chosen because it is believed to positively affect performance in both educational and work settings (Porter and Lawler, 1960). In addition, it has been found to affect performance in negotiating purchases (Huber and Neale, 1986), decision making quality and quantity in a simulated applicant selection task (Erez and Arad, 1986), and performance in a marketing simulation (Lant and Montgomery, 1987). This study measured motivation in two ways. First, a forced choice question asked students why they selected their teammates. The question contained two choices reflecting the need to achieve (Atkinson, 1958), two choices reflecting the need to affiliate (Atkinson, 1958), and two choices reflecting a selection of teammates by chance. Second, students were asked to respond to questions about how much and why they desired to play as the game progressed. Student responses thus reflected their overall interest in playing. Responses were categorized, using content analysis, according to interest in the simulation and whether desire to play increased, stayed the same, or decreased as the game progressed. Independent (Team). Team attributes expected to predict performance included cohesion and organizational formality. Cohesion. Cohesion was included because many authors (including Bass, 1982; Cartwright, 1968; and Gladstein, 1984) hypothesize a positive relationship between cohesion and performance. There is also evidence of a positive relationship between cohesion and performance in the laboratory (Mott, 1972; Gladstein, 1984), in the workplace (Bass, 1982; O’Reilly and Roberts, 1977), and in computer simulations (Miesing and Preble, 1985; Norris and Niebuhr, 1980; Wolfe, 1975; Yantis and Nixon, 1982). This study measured cohesion by determining (1) how well teammates knew each other before the game began and (2) their connectedness during the game. Eleven individual Likert-type questions asked about: - The degree to which teams thoroughly discussed decisions, worked on maintaining good relations, and seemed cohesive - Teammate familiarity before the game began - The equitability of workload - Whether working together was enjoyable - Whether members were supportive and worked together to enjoy each other - The degree to which individuals were frustrated because of a lack of influence. Organizational Formality. Organizational formality was chosen because of an expected association between formality and performance. Organization-related variables such as formal coordination (Mott 1972), centralized structure (Mulder, 963), arid group orderliness (Bass, 1982) have been found to be associated with performance. This study defined organizational formality as clarity of task expectations. Four Likert-type questions measured clarity of workload distribution, the degree of frustration due to disorganization, the formality of decision making, and the degree to which the group Was organized. Dependent: Performance. Performance was defined in terms of accumulated net income made during the simulation. However, since there were sixteen industries following three market curves in this study, a simple tally was inappropriate. For example, a specific dollar figure might be the highest in one industry while trailing in another. The performance measure used for this study was the ratio of (1) the difference between a given company’s net income and that of the industry leader to (2) the average difference between leaders and followers for that particular market curve. Where: I = net income I = company (team or individual) 1 = leader in that industry m = market curve Thus, if (a) team 4 of industry A was ten million dollars behind the winner, (b) team 3 won industry A, (c) team 6 of industry B was 30 million dollars behind the winner, (d) industries A and B used market curve Q, and (e) followers averaged 20 million dollars behind leaders in industries using market curve Q; then A-4’s performance score would be .5, team A-3’s would be 0 and team B-6’s would be 1.5. Procedure During the second class, students were introduced to the business game and the research project, asked to form teams of three, and told of the researcher’s general purpose. They also filled out an initial questionnaire requesting their major, grade point average, extra curricular background and basis for selecting teammates. At the five week point, teams from two randomly selected sections broke up for individual play, and all students filled out a second set of questionnaires requesting information about interest in the game, team cohesion, and organizational formality. Play continued in all sect ions for an additional five weeks, and then all students filled out a final questionnaire asking about interest in the game and, for sections still in teams, further questions on cohesion. A debriefing session was used to explain questionnaire purposes and the study’s hypotheses. RESULTS Hypothesis 1 Hypothesis 1 predicted that the variables influencing performance when tie game is played in teams would be different than those influencing performance when the game is played by individuals. To test this hypothesis, a stepwise multiple regression was used with this study’s “behind the leader” performance variable as the dependent variable and all continuous ability, interest, motivation, cohesion, and organization variables as m itoni i n NINIsum NINIScoreePerformanc ))((_ 11 1 − − = = Development In Business Simulation & Experiential Exercises, Volume 18, 1991 46 independent. Table 1 shows the results from the regression equations for the individual and team conditions. It displays the independent variables retained in the final regression equation because of the magnitude of association with the dependent variable and the lack of multicollinearity. The two final regression equations in Table 1 contain different independent variables suggesting that the variables influencing performance when the game is played in teams are different from those influencing performance when the game is played by individuals. This result is not surprising because the data is taken from different class conditions, but it does support hypothesis 1. TABLE 1 STEPWISE REGRESSION OF INDEPENDENT VARIABLES ON PERFORMANCE TEAM INDIVIDUAL Variable Beta Sig. Variable Beta Sig. Variables in Overall Collaborate Final Equation Attitude .60 .00 After .43 .00 Disbanding Division Desir. to of Labor - .25 .02 Play .46 .00 (Tim. 3) (Time 2) Fun —.22 .04 University -.29 .03 (Time 2) CPA Frustration With .29 .03 Influence (Time 2) Adj. R2 .45 .50 F 13.92 9.09 p .00 .0 Hypothesis 2 Hypothesis 2 suggested that characteristics of individuals will predict performance when the game is played by individuals but not when the game is played by teams. Tables 2 and 3 show the results of testing that hypothesis. Table 2 shows correlations between performance and individual background continuous variables for both team and individual conditions. Of the variables measurable before the game began, two, accounting and finance course GPAs, correlated significantly with performance in the individual condition only. This finding suggests that past performance in accounting and finance courses predicts performance for individual play but not for team play. TABLE 2 CORRELATIONS BETWEEN PERFORMANCE AND CONTINUOUS INDIVIDUAL BACKGROUND VARIABLES TEAM INDIVIDUAL r C Ability University CPA .04 .15 Finance CPA -.01 .20* Accounting GPA .10 .27** Interest Number of: Finance Courses -.02 .02 Accounting Courses -.15 .12 Extracurricular Offices .02 .18 Motivation Desire to Play at Time 1 .01 .15 Reasons for Choosing Teammates: Achievement .31** .06 Affiliation .22** .12 Accidental -.14 -.09 p < .10 P <.05 P <.001 Two of the variables measured before the game began correlated significantly with performance in the team condition but not in the individual condition. For team play, performance was higher when teammates chose each other because of achievement or affiliation expectations. However, this was not the case when the last half of the game was played by individuals. This is important because players did not know whether they would finish the game in teams when they chose their teammates. Whatever performance enhancing attitude influenced players to choose teammates on the basis of expected achievement or amicability, that attitude maintained itself and was associated with better performance when teams stayed intact. However this same performance enhancing disposition did not maintain itself when teams disbanded. Other variables measured before the game’s start, namely overall grade average, number of accounting and finance courses taken, number of extracurricular offices held, and initial desire to play, did not correlate significantly with performance in either the individual or the team condition. Table 3 shows results of analysis of variance of performance by academic major in both individual and team conditions. Performance did not vary significantly across major when the game was played in teams, but did vary significantly when the game was played individually. Scheffe tests showed no significant differences in performance across any pairs of majors. However, majors in management computer systems, finance, and production performed relatively well while marketing majors performed relatively poorly. TABLE 3 ANALYSIS OF VARIANCE OF PERFORMANCE BY ACADEMIC MAJOR TEAM INDIVIDUAL N MEAN STD DEV N MEAN STD DEV Major Accounting 16 79.4 71.6 12 76.7 60.9 Finance 20 83.9 71.8 8 61.3 41.7 Mgmt. Computer Sys. 1 110.0 0 5 0.0 0 General Business 4 56.5 80.2 6 87.0 66.6 Production/Operations 2 77.0 108.9 6 67.3 6.4 General Management 3 53.0 37.8 1 102.0 0 Personnel/HR 3 70.7 34.6 5 71.4 49.8 Office Administration 2 88.0 45.2 2 121.5 112.4 Marketing 13 68.5 46.4 12 128.1 57.3 Financial Planning 2 26.5 37.5 1 70.0 0 Finance/Marketing 1 56.0 0 1 212.0 0 Management/Marketing 1 120.0 0 0 -- -- F Ratio 0.30 (p = .98) 2.92 (p = .006) Results from the previously described regression analyses also support Hypothesis 2. As indicated in Table 1. university GPA was significantly associated with performance for individuals but not for teams. The results from this study show some support for Hypothesis 2. In the individual condition, performance varied significantly with four individual background variables, namely academic major, CPA in accounting courses, GPA in finance courses, and university GPA. When the game was played in teams, the relationship between these background factors and performance was not significant. DISCUSSION This study explored different antecedents and their relation to whole-enterprise simulation performance when the game was played in teams versus when played by individuals. Specifically, we predicted that the impact of individual background factors would be relatively unimportant when the game was played in teams. While the data supported the hypotheses, the more important Development In Business Simulation & Experiential Exercises, Volume 18, 1991 47 findings involved the impact of group-related variables. Two results reveal the important impact of the team experience on performance. First, as shown in Table 1, two of the four variables loading significantly on performance in the individual condition were group-cohesion related. Thus, performance was affected by the team experience even though teams had disbanded 5 weeks before performance was measured. One of the variables loading on performance in the individual condition was post-disbanding collaboration. Individuals who collaborated performed better, and most of this collaboration was with former teammates. It appears that some of those with favorable team experiences continued to productively associate with former team members. Second, as noted above for the team condition, performance was superior when teammates chose each other from achievement or affiliation expectations. This may suggest that the choosing of teammates leads to performance enhancing cohesion which remains as long as teams are together. While this study’s data do not provide a detailed explanation of how cohesion affects performance, the results suggest that, whether or not a team remains intact, the team experience impacts effectiveness. Recent research has explored the influence of team composition (Patz, 1990) and team building (Wolfe, Bowen & Roberts, 1989) on simulation outcomes. The results of this study suggest strongly that such explorations should prove fruitful. For those who administer simulations, the results suggest strongly that enhancing those factors that promote cohesion will likely have positive impact on performance, whether or not teams remain intact. In this study, cohesion-related variables correlated with and predicted performance in both group and individual conditions, suggesting the power of group experiences. Enhancing those experiences appears to be a good idea. REFERENCES Atkinson, J.W. (1958). Motives in Fantasy, Action and Society. Princeton, NJ: Von Nostrand. Bass, B. (1982). “Team performance and productivity.” In Dunette, D.M. & Fleishman, E.A. (Eds.), Human Performance and Productivity: Human Capacity Assessment. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc., 1, 179-232. Cartwright, D. (1968). “The nature of group cohesiveness.” In Cartwright, D. & Zander, A. (Eds.), Group Dynamics. New York: Harper & Row. Erez, M. & Arad, R. (1986). “Participative goal-setting Social, motivational, and cognitive factors.” Journal of Applied Psychology, 71, 591- 597. Gladstein, D.L. (1984). “Groups in context: A model of task group effectiveness.” Administrative Science Quarterly, 29, 499-517. Gosenpud, J.J. (1987). “Research on Predicting Performance in the Simulation.” Developments in Business Simulation & Experiential Exercises, 14, 75-79. Gosenpud, J.J. (1989). “Prediction of simulation performance as it is affected by time.” Simulation and Games, 20, 319-350. Gosenpud, J.J. & Miesing, P. (1984) “Determinants of Performance in the Simulation.” Developments in Business Simulation & Experiential Exercises, 10, 53- 56. Hinton, R.W. & Smith, D.C. (1985). Stratplan. Englewood Cliffs, NJ: Prentice-Hall. Huber, V.L. & Neale, M.A. (1986). “Effect of cognitive heuristics and goals on negotiator performance and subsequent goal setting.” Organizational Behavior and Human Decision Process, 38, 342-365. Korman, A.K. (1970). “Towards a hypothesis of work behavior.” Journal of Applied Psychology, 54, 31-41. Lant, T.K. & Montgomery, D.B. (1987). “Learning from strategic success and failure.” Journal of Business Research, 15, 503-517. Miesing, P. & Preble, J.F. (1985). “Group processes and performance in a complex business simulation.” Small Group Behavior, 16, 325-338. Mott, P. (1972). The Characteristics of Effective Organizations. New York: Harper & Row. Mulder, M. (1963). Group Structure, Motivation and Group Performance. The Hague - Paris: Mouton. Norris, D.R. & Niebuhr, R.E. (1980). “Group variables and gaming success.” Simulation and Games, 11, 301-312. O’Reilly III, C.A. & Roberts, K.H. (1977). “Task group structure, communication, and effectiveness in three organizations.” Journal of Applied Psychology, 62, 674- 681. Patz, A.L. (1990). “Group personality composition and total enterprise simulation performance,” Developments in Business Simulation & Experiential Exercises, 17, 132- 137. Porter, L.W. & Lawler III, E.E. (1960). Managerial Attitudes and Performance. Homewood, IL: Irwin. Sjoberg, L. (1984). “Interests, effort, achievement, and vocational preference.” British Journal of Educational Psychology, 54, 189-205. Skaalvik, E.M. (1983). “Academic achievement, self-esteem and the valuing of the school -- some sex differences.” British Journal of Educational Psychology, 54, 189–205 Terborg, J.R., Castore, C.H. & DeNinno, J.A. (1976). “A longitudinal field investigation of the impact of group composition on group performance and cohesion.” Journal of Personality and Social Psychology, 34, 782- 790. Vance, S.C. & Grey, C.F. (1967). “Use of a performance evaluation model for research in business gaming.” Academy of Management Journal, 10, 27-37. Webb, S.C. (1957). “Differential prediction of success in graduate school.” Journal of Educational Research, 50, 45-54. Wolfe, J. (1975). “Effective performance behaviors in a simulated policy and decision making environment.” Management Science, 21, 872-882. Wolfe, J. (1976). “Correlates and measures of external validity of computer-based business policy decision making environments.” Simulation & Games, 7, 411- 437. Development In Business Simulation & Experiential Exercises, Volume 18, 1991 48 Wolfe, J., Bowen, D.D., a Roberta, C.R. (1989). “Team building effects on company performance.” Simulation & Games, 20, 388-408. Yantis, B. & Nixon, J.13. (1982). “Interpersonal compatibility effects on simulation game outcomes.” Simulation and Games, 13, 337-349. Table of Contents Volume 18, 1991 Personality Types and Total Enterprise Simulation Performance Using DIS 'n DAT as a Decision Support System for a Marketing Simulation Game Theoretical Derivation of a Market Demand Function for Business Simulators The Ethnographic Case Study: An Experiential Approach to Teaching Retail Management Electronic Bulletin Board Systems (BBS): Support Software for Computer Simulations The New Budget Game Negame: A Cross-Cultural Role-Play to Introduce Students to the Familiarization Stage of Negotiations Modeling Short-Run Cost and Production Functions Using Sheppard's Lemma in Computerized Business Simulations Increasing Simulation Realism through the Modeling of Step Costs Predicting Simulation Performance: Differences Between Groups and Individuals A Facility Location Case to Stimulate Classroom Interaction Educational Effectiveness of Business Simulation Gaming: A Comparative Study of Students and Practitioner Perspective Ethical Dilemmas in Experiential Learning: Issues and Strategies A Critical Review and Assessment of ABSEL's Award-Winning Procedures and Protocols Political Risk: A Simulation for Business Practitioners Upside Down: A Cross-Cultural Game in Experiential Learning Gorby's Dilemma: From Communism to Free Enterprise in Two Hours Strategic Market: Planning with the COMPLETE Product Portfolio Analysis Package: A Marketing Decision Support System Career Concepts and Total Enterprise Simulation Performance Experiential Learning in Human Resources: A Performance Appraisal Application Managerial Motivation and Realism Among MBA Student as Viewed through The Looking Glass, Inc. Simulation An Experiential Approach to Teaching Data Analysis Using MYSTAT: Rationale, Procedures and Results Practicing What Was Preached: A Sequential Learning Model put to the Test Student Attitudes about Policy Course Simulations An Investigation of the Relationship Between Simulation Play, Performance Level and Recency of Play on Exam Scores The Effect of Leadership and Cognitive Processing Styles upon Peer Performance Evaluation: Implication for the Utilization of Simulations in Business Pedagogy On the Transfer of Market Oriented Business Games to Socialist Cultures An Application of Financial Analysis of the Business Firm in a Simulated Competitive Environment Collective Bargaining Simulation: An Exercise based on a Familiar Theme Making Business Policy a Strategic Management Experience A Student Exercise for Intergrating the Concepts of Power and Motivation The Boundaries Extended: An Experiment Comparing Dialectical Inquiry, Devil's Advocacy and Consensus Using the Executive Game Using a Simulation Package to Develop a Simulation Exercise in Cost Accounting Success Factors in Experiential Training for Creative Problem-Solving Teams An Experiential Approach for teaching Quality management Critical Success Ratios: A Comparison of Two Business Simulations in a Multi-Year Environment Stocklogs: A Classroom Exercise for Teaching the Logistical Relationship of Location and Inventory The Organizational Leadership Program Simulating Business Decision-Making: Using Statistical Cases for Classroom Exercises Scripting for the Classroom Upgrading the Business Strategy and Policy Game Developing Student Team-Building and Leadership Skills Using Computer-Aided Experiential learning Strategies Organizing and Outward Bound Field Trip An Architecture for Extensible Simulation Games Performance in the Capstone Business Course: What is the effect of Pedagogy, Learning Styles, and Student Motivation? Operational Strategy with Participant-Modifiable Parameters An Example of a Personal Selling Case Transformed into a Role Play Scenario Instructional Software: It's Evolution and Current State of the Art in the Business Curriculum Ascertaining Performance Variables for use in Determining Student's Grades in Courses Employing a Business Simulation Designing Management Seminars Using Business Simulations The Accounting Information Systems Course: Bridging the Gap between the Classroom and the Real World The Good Cooks Guide to Training Excellence: Working with Passion A Demonstration on Multiple Data Collection Methods: Seeing Strategic Issues Through the Looking Glass Simulation Systems Analysis and Design: Why Undergraduate Education gets a Failing Grade Accommodating Organizational Culture: An Evaluation of Management Development Delivery Modes in Varying Organizational Cultures Modeling Total Quality into Business Simulations The Political Futures Game Meeting Meeting Objectives