Modelling An Introductory Project Management Simulation In Undergraduate Courses Page 157 - Developments in Business Simulation and Experiential Learning, volume 41, 2014 BUSINESS SIMULATION TEAM PERFORMANCE AFTER COMPLETING AN INDIVIDUAL PRACTICE MODULE Michael Nugent Stony Brook University College of Business Michael.Nugent@stonybrook.edu ABSTRACT Many factors can contribute to a student’s successful learning experience when participating in team based simulations. What is the best way to deploy simulation assignments to students? Is there a benefit to having students complete a significant amount of simulation practice rounds before competing against each other in a team environment? The main objective of this research: using comparative data from two classes to define the effectiveness of student team performance within a business simulation after an individual simulation practice assignment has been completed. INTRODUCTION The utilization of business simulation programs in undergraduate education has grown increasingly popular over the last 40 years. “The number of AACSB member schools using business simulation games in their programs remains high. Usage has now reached 97.5%.” (Faria, 1998). The majority of these simulations help students gain an understanding through a virtual experience in operating a corporate firm within a competitive marketplace. Today most major business simulations have evolved and now provide an individual practice module. These practice modules offer the same simulation game for individual players who compete against computer opponents. This research paper examines changes in game and class performance when students complete the individual practice simulation modules. THIS RESEARCH PAPER WILL INVESTIGATE THE FOLLOWING:  Hypothesis (1): Students who complete six simulation rounds of individual practice will perform significantly better when competing in six simulation rounds of team based competition, than students who only engage in the team based simulation rounds.  Hypothesis (2): The mean of each teams individual practice scores is significant in relationship to team competition scores.  Hypothesis (3): The mean of each team’s GPAs is a predictor of team competition scores.  Hypothesis (4): The mean of each team’s course grade is a predictor of team competition scores.  Hypothesis (5): The individual practice simulation score will be a predictor of course grade.  Hypothesis (6): Students who complete six simulation rounds of individual practice will perform significantly better on homework and exams, than students who only engage in the team based simulation rounds. LITERATURE REVIEW More than 200 papers have been published that examine business simulation performance, business simulation team performance, effectiveness of business simulation in learning classroom material, and learning objectives of business simulations. Some important findings have already been well established:  Increasing instructor involvement, in feedback and instruction of simulation improves simulation performance. (Biggs 1975); (Nulsen & Faria 1977); (Snow, Gehlen & Green 2002)  Highly cohesive teams perform better than less cohesive teams. (Etnyre & Wolf, 1975)  Team performance is higher than individual performance. (Nielsen, 1975)  Simulation teams perform better with instructors who review results and performance after each round. (Hodgetts & Kreitner, 1975)  Teams perform better when instructors assign additional homework related to the learning goals of the simulation. (Parish, 1975)  Individual simulation performance is positively correlated with GPA. (Lynch and Michael 1989)  Teams perform at a higher level when the simulation is a higher percentage of the final grade. (A. J. Faria 1986)  The most common form of evaluating performance on a simulation exercise is to compare the ranking of a student’s team to other teams on a number of predetermined measures generated by the simulation. (Anderson & Lawton 1988) mailto:Michael.Nugent@stonybrook.edu Page 158 - Developments in Business Simulation and Experiential Learning, volume 41, 2014 SIMULATION For this study students worked with the “Zoom Business Simulation” from Jupiter Interactive. This simulation is comparable to: Capsim’s “Capstone simulation”, Marketplace’s “Business simulator”, and Smartsim’s “Mikes Bikes simulation”. All four of these commonly used simulations have a similar design; students make a series of top management decisions in various business areas, such as: sales, marketing, operations, finance and human resources. Student teams compete directly against each other for market share. After each round these simulations provide feedback and reports on team performance in various measures. In the Zoom Business Simulation students research and design, forecast, market, produce and manage four vehicle class products: Economy, Sedan, Truck, and Luxury automobiles. Each of the four major decision areas: Sales, Marketing & Advertising, Production and Finance, has the following number of team based decisions: Sales: 28 decisions, Marketing & Advertising: 30 decisions, Production: 24 decisions, and Finance: 6 decisions. Each round student’s earn a “overview score” based on year over year performance utilizing nineteen measurements: Revenue Per Share, Quick Ratio, Debt Ratio, Gross Profit Margin, Operating Profit Margin, Net Profit Margin, Total Asset Turnover, Equity Multiplier, Return On Equity, Earnings Per Share, Return On Assets, Market Capitalization, Book Value per share, Sales Verses Forecast for each car class: Economy, Sedan, Truck, and Luxury, Operational Investments, Cash Surplus/Deficit. Teams earn or lose one point for each percentage increase or decrease, respectfully, in year over year performance. Description from the simulation web site “Enable your students this semester to take control over a new automobile company, making all the relevant sales, marketing, operational and financial decisions necessary to propel their company towards success, while competing directly against other student teams within your class. The Zoom Business Simulation Game is the perfect project to reinforce core business principles for any Intro to Business, Business Strategy, Marketing, Management, Operations or Finance course.” Retrieved from: www.jupiterinteractive.net METHOD This study was based on data collected from two sections of undergraduate finance taught at Stony Brook EXHIBIT 1 ACADEMIC STANDINGS EXHIBIT 2 GENDER EXHIBIT 3 FULL/PART TIME Academic Standing Fall 2010 Academic Standing Fall 2011 Year Count Percent Spring 2011 Count Percent Freshman 2 1.6% Freshman 1 0.9% Junior 48 38.4% Junior 39 33.9% Senior 58 46.4% Senior 36 31.3% Sophomore 17 13.6% Sophomore 39 33.9% Grand Total 125 Grand Total 115 Gender Fall 2010 Gender Spring 2011 Fall 2010 Count Percent Spring 2011 Count Percent Female 56 44.8% Female 48 41.7% Male 69 55.2% Male 67 58.3% Grand Total 125 Grand Total 115 Full/Part Time Fall 2010 Full/Part Time Spring 2011 Fall 2010 Count Percent Spring 2011 Count Percent Full Time 120 96.0% Full Time 111 96.5% Part Time 5 4.0% Part Time 4 3.5% Grand Total 125 Grand Total 115 http://www.jupiterinteractive.net Page 159 - Developments in Business Simulation and Experiential Learning, volume 41, 2014 University’s College of Business. The first section of 125 students occurred during the fall semester of 2010 and the second section of 115 students occurred during the spring semester of 2011. The fall 2010 section had 41 teams of 3 students, and 1 team of 2 students. Each of these teams worked on the simulation during the last six weeks of class. The fall 2010 section completed six rounds of team simulation competition without any individual practice. The spring 2011 section had 37 teams of 3 students and 1 team of 2 students. Each student completed six individual practice rounds during the first four weeks of class, and then completed six rounds of team competition during the last six weeks of class. The six rounds of practice have the same decision and game dynamics as the six rounds of team competition. The only significant differences between the practice and team competition modes are the computer players the individual practice simulation uses to compete against individual students. It should be noted that I was the instructor for both sections. This research paper was conceived five months after the last day of the spring 2011 section, May 19th 2011. I collected the data with the help of Jupiter Interactive and Stony Brook University in November of 2011. Students in each section were randomly assigned to three person teams. Both sections were the same subject: Finance, all course materials were identical, chapters covered, textbook, homework’s, in class assignments, and exams. STUDENT COMPOSITION Both sets of students are comparable, no major differences were evident. RESULTS  Hypothesis (1): Students who complete six simulation rounds of individual practice will perform significantly better when competing in six simulation rounds of team based competition, than students who only engage in the team based simulation rounds. A “Two Sample T Test” was conducted for hypothesis testing. this compared whether the average difference between two groups (PRACTICE GROUP) spring 2011 students, and (NO PRACTICE GROUP) fall 2010 students, is significant or due instead to random chance. A 95% confidence interval and a significance level of 5% were used.  Null Hypothesis 1 (H0): the difference between the (PRACTICE GROUP) and (NO PRACTICE GROUP) team simulation scores is 0.  Alternative Hypothesis 1 (Ha): the difference between the observed mean of (PRACTICE GROUP) and (NO PRACTICE GROUP) team simulation scores is not zero. Based on the results: T=3.95, F=1.80 and P=0.0001, the Null Hypothesis was rejected. The difference between the two groups is significant. HYPOTHESIS 1 TEAM STATISTICS Additionally we can see that the average team scores for PRACTICE GROUP increased by 378.1 points or 14.37% and the standard deviation decreased by 127.97 points.  Hypothesis (2): the mean of each teams individual practice scores has significance to team competition scores.  A “Paired T Test” conducted for hypothesis 2 testing. Comparing whether the difference between individual practice scores and the team competition scores for the spring 2011 class is significant or due instead to random chance. Each team members practice score were averaged together to create a comparable averaged practice score. A 95% confidence interval and a significance level of 5% were used.  Null Hypothesis 2 (H0): the difference between the practice scores (mean of practice scores by team) and team scores simulation scores is 0.  Alternative Hypothesis 2 (Ha): the difference between the observed mean (mean of practice scores by team) and team simulation scores is not zero.  Based on the results: T=16.46 and P=0.00001, the Null Hypothesis was rejected. The difference between the two groups is significant. EXHIBIT 4 MEAN, STANDARD DEVIATION AND STANDARD ERROR Variable N Mean SD SE Practice Group Team Scores 40 3007.8 372.92 58.964 No Practice Group Team Scores 44 2629.7 500.89 75.512 Difference 378.1 444.64 97.14 Page 160 - Developments in Business Simulation and Experiential Learning, volume 41, 2014 PRACTICE SCORES OF PRACTICE GROUP A Linear regression model with the (No Practice Group) as the dependent variable and (Practice Group) as the independent variable shows some significance between the two sets of scores (R2=.193, F=9.09, P=.0046 ) STATISTICS OF TEAM & (MEAN OF INDIVIDUAL PRACTICE) SCORES  Hypothesis (3) the mean of each team members GPA is a predictor of team competition scores. GPA & TEAM SCORES SPRING 2011 A Linear regression model using the Spring 2011 Practice Group’s team competition scores as the dependent variable and Spring 2011 Practice Group’s GPA as the independent variable shows no significance between the two sets of data (R2=.0459, P=.1812)  Hypothesis (4) the mean of each team’s course grade is a predictor of team simulation competition scores. COURSE GRADE & TEAM SCORE SPRING 2011 A Linear regression model using the Spring 2011 Practice Group’s class grade as the dependent variable and Spring 2011 Practice Group’s team competition scores as the independent variable shows no significance between the two sets of data (R2=.0015, P=.8151)  Hypothesis (5) the individual practice simulation score will be a predictor of course grade. INDIVIDUAL COURSE GRADE AND PRACTICE SCORE SPRING 2011 A Linear regression model using the Spring 2011 Practice Group’s course grade as the dependent variable and Spring 2011 Practice Group’s individual practice scores as the independent variable shows significance between the two sets of data (R2=.2309, P=.000001), the two are positively correlated. Additionally a positive correlation is expressed when comparing individual simulation scores with GPA’s and Class Grades in a multiple regression model. The Results (R2=.2437, F=18.04, P=.000001) were significant.  Hypothesis (6) students who complete six simulation rounds of individual practice will perform significantly better on homework and exams, than students who only engage in the team based simulation rounds EXHIBIT 5 PRACTICE SCORES y = 0.3652x + 961.41 R² = 0.193 0 500 1000 1500 2000 2500 3000 0 1000 2000 3000 4000 Sim Practice Scores IndSimScore Linear (IndSimScore) Sim Team Scores EXHIBIT 6 STATISTICS OF TEAM & (MEAN OF INDIVIDUAL PRACTICE) SCORES Variable N Mean SD SE Group Team Scores 40 3000.6 369.74 58.451 Mean Of Practice Scores By Team 40 2057.3 307.42 48.607 Difference 943.3 62.32 9.854 Page 161 - Developments in Business Simulation and Experiential Learning, volume 41, 2014 After backing out the influence of the grade for the simulation assignment, the average grade for (No Practice Group) Fall 2010 was 81.98 compared to an average grade for (Practice Group) Spring 2011 of 83.28. The difference of 1.3 is not significant, DISCUSSION OF RESULTS The analysis of Hypothesis (1) did show that students who complete six simulation rounds of individual practice performed significantly better on the team based competition, than students who only engage in the team based simulation rounds. The Two Sample T test produced a P=.0001. The mean increase in scores for students who completed the practice round was 378 points higher than students who did not complete the practice rounds. In addition the students who completed the practice rounds had a tighter dispersion of team competition results based on a smaller standard deviation of 373 versus a standard deviation of 500 for students who did not complete any practice rounds. These results suggest that a student who completes the practice module has a significant advantage and scores higher on the team competition rounds. Using a Paired T Test I was able to show significance in the data related to Hypothesis (2) T=16.46, P=.00001. The scatter plot graph and the linear regression model produced a (R2=.193), the linear regression showed additional significance F=9.09, P=.0046.) This detailed a positive correlation between the mean of each teams individual practice scores and team competition scores. The results suggested performance measured by calculating a mean of the individual practice scores by team is a significant factor in predicting team competition scores. This significance is reduced because individuals compete against computer opponents, while teams compete against each other. The significance in this model may also be explained by the fact that the practice rounds were completed before team competition rounds. Student teams had a mean increase of 944 points on team scores compared to individual mean scores. The (Practice Group) mean team score of 3001 is higher than the (No Practice Group) simulation mean score of 2057. (See Table: Statistics of Team & (Mean of Individual Practice) Scores) This may support early findings which suggested that teams perform better than individuals. (Nielsen, 1975) Both the scatter plot and linear regression model failed to support Hypothesis (3) The linear regression resulted in a (R2=.0459, P=.1812) no significance was found to support the idea that, the mean of each team’s GPA is a predictor of team competition scores. Similarly Hypothesis (4) failed to show significance, the linear regression model produced a (R2=.0015, P=.8151) clearly the mean of each team’s course grade is not a good a predictor of team competition scores. Strong support was documented for Hypothesis (5) the individual practice simulation score will be a predictor of course grade. A positive correlation is expressed when EXHIBIT 7 GPA AND TEAM SCORES SPRING 2011 Page 162 - Developments in Business Simulation and Experiential Learning, volume 41, 2014 comparing individual simulation scores with GPA and course grade in a multiple regression model. The Results (R2=.2437, F=18.04, P=.000001) were significant. This finding supports data collected and published by Lynch and Michael, they discovered that “Individual simulation performance is positively correlated with GPA.” (Lynch and Michael, 1989) I was not able to find any significant evidence to support Hypothesis (6) students who complete six simulation rounds of individual practice will perform significantly better on homework and exams, than students who only engage in the team based simulation rounds. After backing out the influence of grade, for the simulation assignment, for both sections, the average grade for fall 2010 class was 81.98 compared to an average grade for spring 2011 class of 83.28. The 1.3 difference is not significant; completing the practice rounds does not help students to significantly perform better on homework or exams. Additionally a linear regression model was developed to compare GPA to course grade, on the (Practice Group) spring 2011 and (No Practice Group) fall 2010 data sets. The fall 2010 class GPA‘s were compared to course grades, regression results (R2=.39, F=69, P=.00001.) Additionally the spring 2011 class GPA‘s and course grades, regression results (R2=.36, F=64, P=.000001.) Both set of results were highly significant showing a positive correlation between GPA and class grades. These results are supportive of a “Study on the Impact of GPA on Perceived Improvement of Higher-Order Cognitive Skills” “The results suggest that, in a favorable learning environment, as students’ level of cumulative GPA increases, the level of reported improvement in higher order cognitive skills also increases.” (Bradley, R. V., Sankar, C. S., Clayton, H. R., Mbarika, V. W., & Raju, P. K. (2007) LIMITATIONS  This study only looked at one simulation game; different simulations may have different results between student practice and team competition modes.  In the practice rounds students compete against computerized players; this may effect their performance in relation to competing against human players. Specifically when looking at Hypothesis 2, the competition dynamics are not the same.  The study looked at only two sections of finance, taught at different points in time, by the same instructor. However the students in each class were comparable; they did not have any major differences.  Results were not compared to other groups of students, at different schools. RECOMMENDATIONS FOR FUTURE STUDY  I suggest repeating this study using other popular simulation games to see of the results can be confirmed across platforms.  Limiting the practice rounds to five, four, three or less to see if a significant improvement in team performance continues to occur.  Replicating this study on larger scale by including other schools and instructors, to further confirm the results of this research.  Future studies comparing the effectiveness of practice EXHIBIT 8 COURSE GRADE AND TEAM SCORE SPRING 2011 Page 163 - Developments in Business Simulation and Experiential Learning, volume 41, 2014 rounds to students with different academic standings. CONCLUSION As a pedagogy tool simulations are an innovative and active learning experience. Students apply business concepts within a simulated real world environment. Simulations have a tremendous potential to enhance a student’s comprehension of the fundamentals of business. Research on improving the effectiveness of student performance and the best practices of deploying simulations in the classroom can have a significant impact on student learning. The results of this research will be valuable to both faculty and simulation designers. Gaining new knowledge in this area and examining how students can improve simulation performance and enhance knowledge retention with business simulation is critical to the success of future students. The findings of this research, which demonstrated that individual simulation practice can enhance the performance of student teams competing in a simulated business environment, will be beneficial to institutions and faculty who incorporate practice modules in future classroom implementations of business simulations. REFERENCES Anderson, P. H., & Lawton, L. (2003). The optimal timing for introducing business simulations. Developments in Business Simulation and Experiential Learning: Vol. 20, 1-4. Bradley, R. V., Sankar, C. S., Clayton, H. R., Mbarika, V. W., & Raju, P. K. (2007)., A Study on the Impact of GPA on Perceived Improvement of Higher-Order Cognitive Skills. Decision Sciences Journal Of Innovative Education 5(1), 151-168. doi:10.1111/ j.1540-4609.2007.00131 Faria, A. J. (1998) Business Simulation Games: Current Usage Levels--An Update Simulation Gaming, Sep 1998; vol. 29: pp. 295-308 Faria, A. J. (2000). The changing nature of simulation Developments in Business Simulation and Experiential Learning, Volume 34, 2007 160 research: A brief ABSEL history. Developments in Business Simulation and Experiential Learning, Vol. 27, 84-90. Faria, A. J. (2001) The Changing Nature of Business Simulation/ Gaming Research: A Brief History. Simulation Gaming, Mar 2001; vol. 32: pp. 97-110 Hodgetts, R.,&Kreitner, R. (1975). Motivating simulation game performance and satisfaction with performance- contingent consequences. Simulation Games and Experiential Learning in Action, 2, 151-156. Kamrani, Farzad, Ayani, Rassul, Moradi, Farshad (2011) A framework for simulation-based optimization of business process models. SIMULATION, Sep 2011; vol. 0: 0037549711417880 Lynch, R., & Michael, T. (1989). Predicting individual decision making performance in a business simulation. Developments in Business Simulation & Experiential Exercises, 16, 182-187. Miles, G. M., Biggs,W., & Schubert, J. (1986). Student perceptions of skill acquisition through cases and a general management simulation. Simulation & Games: An International Journal, 17(1), 7-24. EXHIBIT 9 INDIVIDUAL COURSE GRADE AND PRACTICE SCORE SPRING 2011 Page 164 - Developments in Business Simulation and Experiential Learning, volume 41, 2014 Nielsen, C. (1975). Player performance under differing player configurations in The Investment Game: Some preliminary findings. Simulation Games and Experiential Learning in Action, 2, 111-115. Parish, L. (1975). Business simulations: Competition or learning. Simulation Games and Experiential Learning in Action, 2, 284-288. Snow, Sheldon C., Gehlen, Frieda L., Green, Jack C. (2002) Different Ways to Introduce a Business Simulation: The Effect on Student Performance. Simulation Gaming, Dec 2002; vol. 33: pp. 526-532 Wenzler, Ivo, Chartier, Don (1999) Why Do We Bother with Games and Simulations: An Organizational Learning Perspective. Simulation Gaming, Sep 1999; vol. 30: pp. 375-384 Wolfe, Joseph (1997) The Effectiveness of Business Games in Strategic Management Course Work Simulation Gaming, Dec 1997; vol. 28: pp. 360-376