ANTECEDENTS OF GAME PERFORMANCE Developments in Business Simulation and Experiential Learning, Volume 31, 2004 ANTECEDENTS OF GAME PERFORMANCE Svein J. Glomnes Buskerud University College, Norway svein-jahr.glomnes@hibu.no ABSTRACT This paper reports empirical findings of a study of undergraduate business students involved in total enterprise simulation. The games were played in teams. The study shows a positive and significant relation between performance and the students’ expected grade in the course (business management) and between performance and the students’ efforts at carrying out special tasks and analyses. It also shows that males outperform females. No significant relation was found between performance and variables like age, average grades from secondary school or college (GPA), or time spent on decision-making. INTRODUCTION This study attempts (once again) to shed some light on the question: What factors influence performance in business simulations? The main purpose of this study is to detect possible effects on game performance from variables connected with the students’ efforts throughout the game. In addition several personal background variables like grades, age and gender have been included in the study. The reason for using a business simulator in a business management course at Buskerud University College is a belief that it will lead to better and increased learning for the students who participate. It represents an alternative and a supplement to the traditional classroom teaching and may help to motivate the students. Previous research indicates that students have more positive attitudes toward learning from business games than from other teaching approaches (Faria 2000). During the game period the students have to do analyses and solve problems related to the different subjects of the course, for instance investment theory, pricing (including internal pricing), cost analysis, budgeting and accounting. The use of a simulator gives a good chance to exemplify the different theories and to apply them on problems connected to the simulation. The Norwegian business simulator model NHH-7 developed by Holmesland & Langholm (1983) has been utilized. This is a total enterprise or top management game (Keys 1987). The model describes the interaction between a number (from four to eight) of competing industrial companies, which produce and sell the same three products. The model is an open simulator in the sense that a team of students makes periodic decisions on behalf of each company. Decisions are made in the areas of investment, product development, production, advertising and pricing. An overall economic aim of long-term profit making and value is assumed, but short term goals, plans and strategies are worked out by the participants themselves on the basis of accumulating data as well as experience, judgment and inclination. The model is programmed in the sense that the results of each period’s decisions are computed following fixed mathematical formulas. Thus the companies compete with one another, and in a sense with the mathematical model, but not with the simulation supervisors, who do not interfere. The students make decisions by filling in a decision form. Results are communicated to participants after each period in the form of computer printouts. This model does not aim at maximum complexity and realism in the description of each economic function covered by these decisions. Its focus is rather on the complicated interaction between functions and between companies at any given moments and as it develops over time. It is believed that this type of simulation can promote insight into economic integration and dynamics, as well as training in the analysis of complex data and in cooperation within goal oriented groups. Model NHH-7 is the last in a consecutive series of business decision simulators developed at the Norwegian School of Economics, and it has been used in internal and external courses as well as for research purposes (for more information, please see Holmesland & Langholm 1983). The dependent variable in this study is game performance measured as the value of the company at the end of the simulation. The antecedents are the number of special tasks and analyses each team carried out during the simulations, the extent to which the team actively used the budget model (developed by themselves) throughout the simulation, the time spent on decision making in each period of the simulation, gender, age, the students’ grades from secondary school and from college (GPA) and finally the expected grade in the business management course. A great many studies have been carried out examining the relationship between game performance and a variety of independent variables including respondent characteristics, team characteristics, simulation characteristics and environmental characteristics. Major review articles (Faria 2000; Keys 1977; Keys & Wolfe 1990; Miles, Biggs & Shubert 1986; Wolfe 1985) have summarized the research. Below attention is drawn to findings concerning one or more of the variables of special interest in this study. Faria (2000) reviewed simulation research through twenty-five years of ABSEL history. He concluded among other things that ABSEL members have reasonably clearly determined a number of factors that correlate with simulation game success. Among these factors are past academic achievement (usually measured as GPA) and academic ability and team characteristics like degree of planning and formal decision-making. He reports contradictory findings concerning the relation between game performance and time spent on decisions. Keys & Wolfe (1990) reviewed the total enterprise management gaming literature. Among other things they reported contradictory results concerning the relation between performance and GPA. Also the relation between performance 229 mailto:svein-jahr.glomnes@hibu.no Developments in Business Simulation and Experiential Learning, Volume 31, 2004 and the use of decision support systems (DSSs) showed conflicting results. Badgett, Brenenstuhl & Marshall (1978) found in a study that many variables suggested to be related to performance such as age, gender and GPA and expected grade in the course, were not. Gosenpud & Washbush (1991) reported findings of a study indicating that university GPA and academic major predicted performance for individual players but not for teams. Hornaday (2001) found in a study that all male teams produced higher simulation scores than female teams, but lower grades on plans and reports related to the simulation. All male teams also had lower GPAs and lower course grades. Hornaday & Wheatley (1986) found in their study that GPA bore no relationship to group performance, and that all female teams outperformed male teams. Johnson, Johnson & Golden (1997) argued that the role played by gender is an important yet neglected area of interest in the simulation literature. In their study they found among other things that game performance measures show no significant gender differences. Lynch & Michael (1989) reported findings indicating that GPA, previous course grades and grades in the course where the simulation was used were related to performance. They also found that females outperformed male students. Wellington & Faria (1990) found that total time spent on decisions had a positive and significant effect on performance. Wood (1987) reported superior performance of men, compared with women, when working both individually as well as when working in same gender teams. Her explanation was that task content or setting favored men’s interest and abilities over women’s. When contradictory findings show up, it is perhaps worthwhile paying attention to Gosenpud (1987) who concluded that no independent variable consistently predicts simulation performance. Situational conditions are stated as one reason for this. Whether or not the simulation is played in teams is an example of a situational condition. Based on the results from the previous mentioned studies and on what would seem to be intuitively logical, it is here assumed that both students efforts (represented by the number of special tasks and analyses the team carries out, the amount of time each participant spends on decision making and the extent to which the team makes active use of the budget model during the simulation) and factors like the participants’ experience (including business experience) and general knowledge and wisdom (all reflected in age), professional skills (reflected in their grades from secondary school and college and expected grade in the business management course), all will influence the quality of the decisions made and thus the value of the company at the end of the simulation. Therefore the following hypotheses are formulated: H 1: The number of special tasks and analyses a participant’s team carries out during the simulation has a positive effect on game performance. H 2: Active use of the budget model has a positive effect on game performance. H 3: The amount of time a participant spends on decision making during each period of the simulations, has a positive effect on game performance. H 4: Male participants outperform female participants. H 5: The age of the participant has a positive effect on game performance. H 6: The participant’s average grade from secondary school has a positive effect on game performance. H 7: The participant’s average grade from college (GPA) has a positive effect on game performance. H 8: The participant’s expected grade in the business management course has a positive effect on game performance. TABLE 1 Sample characteristics Year 1997 1998 2000 2001 Total Number of groups 12 6 8 8 34 Of this: Female groups 4 1 0 0 5 Male groups 4 2 1 0 7 Mixed groups 4 3 7 8 22 Number of participants 48 24 40 38 150 Of this: Women 22 14 27 26 89 Men 26 10 13 12 61 Number of returned questionnaires 38 13 34 20 105 Of this: Women 18 6 23 15 62 Men 20 7 11 5 43 Response rate 79,2 54,2 85 52,6 70 Of this: Women 82 43 85 58 70 Men 77 70 85 42 70 230 Developments in Business Simulation and Experiential Learning, Volume 31, 2004 METHOD The study is based on data from four different simulations in the years 1997-2001. (There was no simulation in 1999). The Norwegian total enterprise simulator NHH-7 was used for all the games. Participants are second year university college students attending a business management course. The sample characteristics are shown in Table 1. The sample equals the total population of 150 participants. An average response rate of seventy for the four years is considered satisfactory. Both genders are equally well represented. The games were played in teams. Each team determined its own composition concerning the number of participants, gender, age etc. Each team consisted of two to five undergraduate students. The aim of the participants was through all the simulations to maximize long-term profits after tax and thereby the value of the companies. It is worth noting that the performance of the teams constituted no part of the grade obtained in this course. The students played with relatively little interference from the instructor’s side through the whole game. The number of periods or decision cycles in each game varied between ten and twelve. The simulations went over three to five weeks. The number of periods pr. week varied between two and four. The dependent variable (game performance) is the adjusted value of the company (measured as equity with smaller adjustments for inventory value, goodwill etc.) at the end of the game. The computer reports equity for each company after every period of the game. Because the adjusted value can fluctuate a lot from game to game, it has been categorized and related to the average adjusted value for each of the four games. The following scale has been used: Adjusted value Code + 25 % above average 7 = very good 15 - 25 % above average 6 = good 5,1 - 14,9 % above average 5 = above average Average +/- 5% 4 = average 5,1 - 14,9 % below average 3 = below average 15 - 25 % below average 2 = bad + 25 % below average 1 = very bad The antecedents and measures were the following: 1. The number of special tasks and analyses each group carried out during the simulation (4 categories). Types of special tasks and analyses were: Developed one or more PC models, made an income budget for one or more periods, made a balance budget for one or more periods, calculated demand functions for one or two products without the use of regression analysis, calculated demand functions for one or two products using regression analysis, calculated the effect of advertising and product development on price or quality, made investment analyses, made calculations for pricing, made profitability analysis or other tasks/analyses specified by the students. 2. To what extent the team made active use of the budget model throughout the game (5 point Likert scale). 3. The time spent on making decisions pr. period pr. participant (4 categories). 4. Gender. 5. Age (4 categories). 6. Average grade from secondary school (9 categories). 7. Average grade from college (GPA – 6 categories) and 8. Expected grade in the business management course (11 categories). All the information on grades was given by the students themselves and was not checked in any way. In addition to answering the questions the students used the opportunity to give comments. The database consists of data from the four years 1997, 1998, 2000 and 2001 (there was no simulation in 1999). Data collection was carried out just after the simulations were finished. The participants answered the questionnaire under the condition of anonymity. The questionnaire was originally constructed to get feedback from the participants mainly for the purpose of improving the simulations from the participants’ point of view. From 1997 personal characteristics were included in the questionnaire enabling more extensive (and interesting) analyses to be carried out. Since then the questionnaire has been unchanged. In addition to the data from the questionnaires, data was collected from the computer- generated reports after each period of the simulations. The analysis was carried out on an individual basis. This means that both team achievements and some team characteristics have been assigned to each individual in the team. In addition personal variables have been included in the analysis. All the data from the four years has been treated as one big set of data. Correlation and regression analyses are used for the analysis of data. RESULTS The presentation of the results starts by the report of the correlation matrix shown in Table 2. 231 Developments in Business Simulation and Experiential Learning, Volume 31, 2004 TABLE 2 Correlation Matrix Mean S.D. ADV NTA UBM TSD GEN AGE GSS GPA Adjusted value (7 – point scale) 3.93 1.61 Number of tasks and analyses 1.90 1.00 .32** Use of budget model 2.35 1.47 .08 .41** Time spent making decisions pr. period 2.20 .71 .16 .49** .39** Gender 1.41 .49 .30** .14 .06 -.07 Age 24.04 3.29 .12 .25* .20* .25* .09 Grades from sec. school 4.18 .53 -.01 -.04 .14 -.01 .01 .04 Grades from college (GPA) 2.38 .51 .23* .09 .27** -.06 .04 .24* .25* Expected grade from course 2.58 .41 .29** .16 .19 .08 -.03 .25* .20* .67** ** Correlation is significant at the 0.01 level (2-tailed) * Correlation is significant at the 0.05 level (2-tailed) The correlation matrix shows no coefficients higher than .49. This suggests that multicolinearity is not a great threat, and all the variables are included in the further analysis. Multiple regression analysis is used to test the hypotheses. The results are shown in Table 3. Model summary shows R square = .277 and R square adjusted = .212 with F = 4.301 (sig.< .000) which is better than expected (but still far from R square = 1!). From the table it is clear that hypotheses 2, 3, 5, 6 and 7 are rejected which means that the variables “active use of the budget model”, “the amount of time spent on decision-making”, “the age of the participant”, “the participant’s average grade from secondary school” and “the participant’s average grade from college (GPA)”, have no significant effect on game performance. On the other hand, hypotheses 1, 4 and 8 are supported. This means that the variables “number of tasks and analyses carried out by the teams” and “the participants’ expected grade in the business management course” has a positive and significant effect on game performance (.05 >sig.> .01). Also the hypothesis “male participants outperform female participants” is supported (sig.< .01). In fact, the beta values indicate that gender is the most important single factor influencing performance with “the participant’s expected grade from course” on second place. DISCUSSION Also the results of this study should be viewed in the light of Gosenpud (1987) who concluded that no independent variable consistently predicts simulation performance. Situational conditions are always important. When interpreting the results of this study it is therefore important to remember that it is based on simulations using the Norwegian total enterprise simulator NNH-7. The participants were undergraduate university college students forming self- determined teams, and they played with relatively little interference from the instructor’s side through the whole game. Another situational condition that is worth mentioning is the fact that the simulation performance did not influence the grade in the course directly, only indirectly through the learning process that hopefully took place. This is important with regard to gender. From the students’ comments it seems clear that the female participants gave less priority to the simulation than did the males. The reason they state for this is precisely the fact that the simulation is of less importance to or irrelevant for the grade in the course. At the same time males typically want to play (and win) regardless of grades. Also important for the evaluation of the result of this study are the methods used in measuring the variables. Without going into details, it must be said that an analysis carried out on an individual basis, which is the case here, could show different results from analysis based on group level. For that reason further analysis on group level with the data from this study would be of interest. 232 Developments in Business Simulation and Experiential Learning, Volume 31, 2004 TABLE 3 Model summary and coefficients Model summary: R square .277 Adjusted R square .212 F 4.301*** N 105 Coefficients: Beta T Number of tasks and analyses .226 2.061* Use of budget model -.064 -.594 Time spent making decisions pr. period .090 .806 Gender .324 3.517** Age -.118 -1.208 Grades from sec. school -.029 -.316 Grades from college (GPA) .066 .504 Expected grade from course .288 2.306* Beta = standardized regression coefficients *** Significant at the .000 level ** Significant at the 0.01 level * Significant at the 0.05 level REFERENCES Badgett, Brenenstuhl & Marshall (1978) “An analysis of performance in simulation games compared to performance on structured course criteria: A case study.” Exploring Experiential Learning: Simulations and Experiential Exercises, 5 32-38. Faria (2000) “The changing nature of simulation research: A brief Absel history.” Developments in Business Simulation &Experiential Learning, 27, 84-90. Gosenpud (1987) “Research on predicting performance in the simulation.” Developments in Business Simulation & Experiential Exercises, 14, 75-79. Gosenpud & Washbush (1991) “Predicting simulation performance: Differences between groups and individuals.” Developments in Business Simulation & Experiential Exercises, 18, 44-48. Holmesland & Langholm (1983) Business decision simulator. Model NHH-6. Brief manual. Bergen: Norwegian School of Economics. Hornaday (2001) “Sex composition, cohesion, consensus, potency and performance of simulation teams.” Developments in Business Simulation and Experiential Learning, 28, 102-105. Hornaday & Wheatley (1986) “Four factors affecting group performance in business policy simulations.” Developments in Business Simulation & Experiential Exercises, 13, 17-21. Johnson, Johnson & Golden (1997) “Multinational business gaming: Is gender important?” In Wolfe & Keys (ed.), Business simulations, games and experiential learning in international business education, (pp. 65-82). New York: International Business Press. Keys (1977) “A review of learning research in business gaming.” In Sord (ed.), Computer simulation and learning theory, (pp. 173-184). Austin: Bureau of Business Research, The University of Texas. Keys (1987) “Total enterprise business games.” Simulation & Games, 18 (2), 225-241. Keys & Wolfe (1990) “The role of management games and simulations in education and research.” Yearly review, Journal of Management, 16, 2, 307-336. Lynch & Michael (1989) “Predicting individual decision making performance in a business simulation.” Developments in Business Simulation & Experiential Exercisis, 16, 182-187. Miles, Biggs & Shubert (1986) “Student perceptions of skill acquisition through cases and a general management simulation.” Simulation & Games, 10, 75-86. Wellington & Faria (1990) “The effect of decision format and evaluation on simulation performance, decision time and the team cohesion.” Developments in Business Simulation & Experiential Exercises, 17, 170-174. Wolfe (1985) “The teaching effectiveness of games in collegiate business courses: A 1973-1983 update.” Simulation & Games, 16, 251-288. Wood (1987) “Meta-analytic review of sex differences in group performance.” Psychological Bulletin, 102, 53-71. 233 Table of Contents Volume 31, 2004 Controlling the Complexity and Orenting Target Groups by a Modular, Server-Based Business Game System Learning Network Demonstration: Delivering Business Education in a Distance Learning Environment Economic Evolution, Human Capital Investment, and Adult Distributed Electronic Learning: A Literature Review Designing a Globalization Simulation to Teach Corporate Social Responsibility Developing and Teaching an Online / In-Class Hybrid: A Demonstration A Model for Evaluating Online Instruction An Evaluation of a Distributed Learning Course: A Students'-Eye Perspective Blended Learning Strategy Improved Business Writing Skills How to Receive and Process Attachemnts while Greatly Reducing the Risk of Viruses and Trojans Introducing Online Components to a Class: How to Increase teh Likelihood of Success Teaching Strategic Communications Online: Using Learning Outcomes to Develop a Case-Based Course Implementing Distance Approaches to Education: A Panel Discussion for ABSEL: Las Vegas, 2004 MANDI: Learning Management Through Field Sales Experience An International Capital budgeting Experiential Exercise A Primer To Combating Terrorism: Playing It Safe While On Overseas Assignment (An Experiential Exercise) Integrating The Business Curriculum With A Comprehensive Case Study: A Prototype The Case Brief: A Model For Case Analysis, Writing And Discussion Technology Infused Pedagogy And Delivery – A Sure Bet? A Proposal For Panel Discussion Absel Conference 2004 Research Strategy And The Bkl: Getting The Most From The Absel Archives Simple But Effective: Rediscovering The Class Discussion Needle And Thread: An Activity For Examining Various Management Behaviors A Customized Excel Data Analysis System For Use In Undergraduate Marketing Research Team Leader Selection - Does It Matter? The Power Of Perspective: Reframing Your Framing Skills For Innovative Instruction In Leadership And Influence Exercise: How Should Merit Raises Be Allocated? An Online Situation For Problem-Based Learning In A Junior-Level Management Course The Eden Alternative As A Roadway For Change: A Service Learning Quality Improvement Project Avoiding Catastrophe: The Role Of Individual Accountability In Team Effectiveness Omega Systems: A Change Management Exercise The Risks And Rewards Of Providing Students A Structured Cheating Opportunity Experimentation With Assessment Techniques: A Proposal For Panel Discussion Using A 2 - Page Case To Introduce Concepts Of Business Strategy Interactive Session The Integration Of Appreciative Inquiry And Experiential Learning For Peak Performance Appreciative Inquiry Case Story: New York City Leadership Challenge Individual Achievement Versus Team Performance: An Empirical Study With Business Games Some Strategists Don't Learn Or Can't Learn Computer Simulation: A Design Architectonic On The Value Of Bugs In Simulation Environments Online Sales Forecasting With The Multiple Regression Analysis Data Matrices Package Simulation Exercises And Problem Based Learning: Is There A Fit? A Study Of Business Game Stock Price Algorithms Assessing Individual Performance In A Total Enterprise Simulation Information Use In A Business Game Determining The Value Of A Firm Unsorting Algorithms For An Ordered List And Its Application To Business Simulations Teaching Public Finance Management Through Simulation Antecedents Of Game Performance Student Expectations Of Classroom Teaching Practices In Developing And Presenting Course Information In Hong Kong Implementation And Impacts Of The Balanced Scorecard: An Experiment With Business Games Impact: Shocking The Legacy Mindset Implementation Of The Eepad Framework Of Business Processes In An Accounting Information Systems Course Are Business Games Really Delivering What Students Are Led To Believe?? Reporting Lessons Learned: What Gets Reported; Who Gains Value Teacher Expectations Of Classroom Teaching Practices In Developing And Presenting Course Information In Hong Kong Student Reactions To The Use Of A Computer-Based Simulation As An Integrating Mechanism For A Mba Curriculum A Cognitive Investigation Of The Internal Validity Of A Management Strategy Simulation Game The Casino Challenge: Making Simulation Delivery A Safe Bet! Accounting For Company Reputation: Variations On The Gold Standard Foreign Currency Hedging: A Simulation The Influence Of Variables Easily Controlled By The Instructor/Administrator On Simulation Outcomes: In Particular, The Variable, Reflection. Absel Awareness Among Business School Faculty Validating Business Simulations: Does High Market Share Lead To High Profitability? Simulation Debriefing Procedures Coaching And Business Simulations: A Formula For Success? A Seminal Inventory Of Basic Research Using Business Simulation Games The Influence Of Scorecard Evaluation On Decisions And Outcomes