Simulations for Strategy Courses: Page 141 - Developments in Business Simulation and Experiential Learning, Volume 43, 2016 ABSTRACT Simulations are an important part of capstone strategy courses - they facilitate transfer of training by providing “learning-by- doing” opportunities to the students. They also allow instruc- tors to provide authentic activities situated amidst relevant con- text, enable learners to grasp not just ‘how’ an activity must be done, but the ‘why,’ the ‘what, and the ‘with whom.’ Simula- tions have become an accepted part of strategy classes both at the undergraduate and graduate levels. One area that has not been fully explored is the impact of teamwork on simulation results. Professors using simulations believe anecdotally what the literature on teamwork has suggested for decades: the qual- ity of team processes is directly correlated to the quality of team results. However, detailed measurements are lacking. This pa- per uses a graduate strategy course as an exploratory study to see if there was a relationship between teamwork and team re- sults. It then includes a just completed graduate course to pro- vide additional data for this study which illustrates some of the complexity in evaluating team performance. INTRODUCTION Simulations are an important part of capstone strategy course. Edgar Dale (1969) illustrated this with his research when he developed the "Cone of Learning". This concept states that after two weeks we remember only 10% of what we read, but we remember 90% of what we do! Simulation-based train- ing puts learning objectives into the context of a scenario which allow the learner to experience training as it relates to a life-like situation. The best simulations are team based with each member having a specified role in the simulation – good simulations are too complex for just one person to play by himself. However, what is the relationship between teamwork and simulation re- sults. Professors using simulations believe anecdotally what the literature on teamwork has suggested for decades - the quality of team processes is directly correlated to the quality of team results. However, detailed measurements were lacking. It was diffi- cult to turn anecdotes into evidence without administering com- plex research instruments – until now. Capsim, a leader in busi- ness simulations, has developed a new tool that will simplify the process of quantifying team dynamics and individual ac- countability within the team. Also, the results can be cross- referenced with overall team performance. CAPSIM SIMULATION One of the leading business simulations is offered by Cap- sim, Inc. The Capsim business simulation engages participants in a dynamic competition to turn struggling companies into suc- cessful, profitable businesses. Classes are divided into teams that compete against each other by making strategy, finance, production, and marketing decisions that interact to grow their business. Instructors focus on using the simulation experience to reach defined learning goals. With each round of decisions (each representing a full year for the company), participants build their business acumen and decision-making ability as they interpret data, shape strategies, and discuss the results. A wide variety of Capsim results, data, and services support the creation of a dynamic, highly interac- tive learning experience. Delivered online, in the classroom, or a combination of both, and delivered in condensed or expanded time frames, Capsim simulations have the flexibility to adapt to many aca- demic or corporate curriculum. Capsim simulations have been used extensively at more than 500 business schools and leading corporations in the US and around the world (Chasteen and Damonte, 2007). BUSINESS EDUCATION A capstone strategy class is usually one of the final courses taken for both undergraduate and graduate business programs. This course integrates all the material from previous classes such as marketing, accounting, and finance. Students draw on their awareness of various environmental influences (social and political) to solve business problems. Management alternatives are examined with an ethical perspective relating policy trends to the strategic planning mode (Chasteen, 2014). The purpose of a capstone course is to integrate the learn- ing achieved in individual business courses taken to earn a busi- ness degree. The knowledge acquired in finance, accounting, operations, MIS, marketing, and organizational behavior classes is utilized to study the strategic management of the firm as well as the responsibilities of the general manager. This is often ac- complished with the extensive use of case studies. This model has proven effective and has been copied by business schools worldwide. Another approach to integrate the learning achieved in indi- vidual business courses taken to earn a business degree is the use of simulations. Students participate in a simulation that re- quires taking into account multiple decision-making factors while balancing all sectors of the firm’s environment. Accord- ing to Dale (1969), an active learning method can provide even better results. Therefore, simulations have become a standard part of many US undergraduate and graduate capstone strategy courses. They are also becoming common in strategy courses in many other countries. THE NEED FOR TEAMWORK SKILLS The use and need for teams is well documented in contem- Simulations For Strategy Courses: Measuring Teamwork Larry Chasteen University of Texas at Dallas chasteen@utdallas.edu mailto:chasteen@utdallas.edu Page 142 - Developments in Business Simulation and Experiential Learning, Volume 43, 2016 porary firms for a variety of purposes and across a variety of industries. Today’s technology is just too complex for employ- ees to work entirely on their own. Good business simulations have followed this general approach. Simulations require stu- dents to apply functional business knowledge while performing in a team-based context, i.e., making decisions about business strategy and operations as members of a top management team (Anderson and Coffey, 2004). Like all performance in team settings, success in simula- tions requires accomplishing operational and technical tasks while engaging in coordination, cooperation, and adaptive ac- tions in order to make effective decisions. Both “task work” and “team work” are necessary. Task work can be defined as opera- tional decisions required to run a company. Team work can be defined as interpersonal interactions required to reach decisions. One important issue is how to accurately measure task work and team work. Simulations simplify measuring “task work” – one year of operation can be simulated by processing one set of decisions. The results show how successful the com- pany was with respect to sales, profits, stock price, etc. (Capsim Users Guide, 2012). Accurately measuring “team work” has traditionally been harder – how does the faculty really know the contribution of each team member. One tool that has been used is peer evalua- tions, but traditional peer evaluations have had problems as dis- cussed later in this paper. The main issue is that students usually don’t like to give bad ratings to their team members even when those members are not doing their fair share (Lam and Shau- broeck, 2000). PEER EVALUATIONS ` Peer evaluations are a vital tool for measuring performance in team based activities. Capsim has used peer evaluations in its business simulations for more than a decade. Capsim’s new peer evaluation, launched in 2012, is a more sophisticated and refined instrument. Capsim created the new peer evaluation after a thorough analysis of thirty years of literature on team dynamics (Capsim, 2012). Peer evaluations are a crucial measurement tool in team based activities where the goal is:  To promote individual level accountability,  To assess individual engagement with and contribution to the team  To measure individual performance in the team. Capsim’s new peer evaluation allows:  Improved ability to evaluate individual contribution to a team  Closer oversight and measurement of team processes  Questions and a roadmap for team development PURPOSE AND RESEARCH QUESTIONS The purpose of this study was to examine if student success in the simulation was related to the quality of the team work. This study used a graduate strategy course at a US university as an exploratory study. The topics covered in the course were traditional class lectures, exams, case discussions, and a busi- ness simulation. The simulation exercise was implemented with a web based simulation game. The simulation lasted eight rounds (eight years) and was scored by using the Balanced Scorecard (BSC). The following research question was addressed: “Is there a relationship between the scoring by a team with the quality of team work for that team?” PROCEDURES The simulation exercise used the Capsim Foundation Simu- lation. Capsim is the best-selling business simulation in the world. This simulation is used at over 500 universities and col- leges in the US. The class was divided into teams to compete in EXHIBIT 1 SELF-REPORTED PEER EVALUATIONS Categories Items Self‐management/Accountability 1. Meeting attendance 2. Meeting preparation 3. Timely communication Quality of Work & Contextual Performance 1. Contributions reflected a thorough understand- ing of the team’s task 2. Presence on the team improved our team’s per- formance 3. Offered consistently high quality contributions 4. Was professional in all team interactions 5. Was open to hearing others’ opinions 6. Paid close attention to important details 7. Remained engaged even when the team’s re- sults were not very good 8. Put forth good effort. Quantity of Work 1. Thinking about all the work your team accom- plished in completing the Capstone simulation, please rate each team member regarding his share of the work. Page 143 - Developments in Business Simulation and Experiential Learning, Volume 43, 2016 a computer simulation by managing an imaginary firm that manufactured electronic sensors. The teams had to make re- search and development, production, marketing, and financial decisions concerning the product. The teams entered their deci- sions into the simulation and then analyzed the results once all the other team decisions were entered and processed. The simu- lation lasted for eight rounds representing eight years. Twenty- three students were divided into six teams for the preliminary exploratory study. Task Work Data The Balanced Scorecard scoring from the simulation was used to compare student success in the classes. The Balanced Scorecard allows companies to gauge their performance by as- sessing measures in four categories:  Financial - includes profitability, leverage and stock price;  Internal Business Process - ranks, among other measures, contribution margin, plant utilization and days of working capital; EXHIBIT 2 PRELIMINARY TEAM BSC RESULTS Round 1 2 3 4 5 6 7 8 Re- cap To- tal Possible Pts 8 2 8 9 8 9 1 00 1 00 1 00 1 00 1 00 240 100 0 Andrews 3 7 4 9 4 7 5 7 5 8 5 9 5 8 7 4 162 600 Baldwin 4 5 4 8 5 8 5 6 5 0 5 1 7 1 7 0 153 602 Chester 3 9 5 1 6 5 6 9 7 7 6 9 7 5 7 7 157 680 Digby 4 8 5 7 6 3 6 7 6 8 6 7 6 6 6 7 150 653 Erie 4 4 5 1 5 1 5 5 6 1 5 9 6 1 6 2 103 546 Ferris 4 9 5 6 5 8 6 8 6 9 7 2 7 0 7 3 151 667 EXHIBIT 3 PEER EVALUATION GRAPHICS Page 144 - Developments in Business Simulation and Experiential Learning, Volume 43, 2016  Customer - examines the company's product line, including how well it satisfies buying criteria and awareness / acces- sibility levels;  Learning and Growth - evaluates employee productivity. The Balanced Scorecard allocates points in each of these four areas for each of the rounds and a final recap score. The team with the highest BSC is considered to be the winner of the simulation rather than just the team with the highest stock price or highest profits. Since the Balanced Scorecard allocates points in four major sections, it is considered a more representative measure of success. Team Work Data The new peer evaluation appears as an online questionnaire with a simple slider scale of 1 to 5. Each student completes the peer evaluation after the simulation is completed. The process is quick and easy to complete. Each student provides a self rank- ing and a peer ranking for each member of their team. The measurement criterion for the new Capsim peer evaluation has three conceptual categories:  Self management/Accountability – Includes meeting at- tendance, preparation for meetings, and timely communica- tion  Quality of Work & Contextual Performance – Includes quality of contributions, consideration of other members, EXHIBIT 4 PRELIMINARY PEER EVALUATION SUMMARY Self-management/ Accountability Questions Self- Ratings Peer Rating of You Team Average Meeting Attendance 4.5 5 4.94 Meeting Preparation. 4.3 3.8 4.63 Timely Communication. 4.5 4.7 4.88 Overall Quality of Work & Performance Score 4.43 4.5 4.82 Quality of Work & Contextual Performance Questions Self- Ratings Peer Rating of You Team Average Presence on team improved our team's performance. 3.9 4.5 4.77 Contributions reflected an understanding of the task. 3.9 5 4.88 Offered consistently high quality contributions. 4 4.4 4.76 Was courteous and professional in team interactions. 4 5 4.89 Was open to hearing others' opinions. 4 5 4.77 Paid close attention to important details. 4 5 4.89 Remained engaged when team's results not very good. 4 4.65 4.81 Put forth good effort. 4 4.4 4.72 Overall Quantity of Work Score 3.98 4.74 4.81 Quantity of Work Self- Ratings Peer Rating of You Team Average Overall Self-management/ Accountability Score 30 34 33.33 Comments Student 1 - It seems that he is very busy with work, he had limited participation Student 2 - No comments submitted. Student 3 - Participates in the discussions, gives good analysis of each round of the simulation. Page 145 - Developments in Business Simulation and Experiential Learning, Volume 43, 2016 and de- EXHIBIT 5 ADDITIONAL TEAM BSC RESULTS Round 1 2 3 4 5 6 7 8 Re cap To tal Possible Pts 82 89 89 10 0 10 0 10 0 10 0 10 0 24 0 10 00 An- drews 58 55 54 60 60 61 66 70 11 8 60 2 Baldwin 48 62 72 88 89 89 90 92 24 0 87 0 Chester 50 58 53 25 35 31 34 32 56 37 4 Digby 53 67 66 93 94 95 85 96 22 5 87 4 Erie 52 51 52 57 64 58 62 61 98 55 5 Ferris 57 65 57 67 69 70 69 73 11 2 63 9 EXHIBIT 6 ADDITIONAL PEER EVALUATION SUMMARY Self-management/ Accountability Questions Self- Ratings Peer Rating of You Team Aver- age Meeting Attendance 5 4 4.78 Meeting Preparation. 5 3.4 4.57 Timely Communication. 5 3 4.29 Overall Quality of Work & Performance Score 5 3.47 4.55 Quality of Work & Contextual Performance Questions Self- Ratings Peer Rating of You Team Aver- age Presence on team improved our team's performance. 5 4 4.47 Contributions reflected an understanding of the task. 5 4 4.51 Offered consistently high quality contributions. 5 3.25 4.43 Was courteous and professional in team interac- tions. 5 4.95 4.97 Was open to hearing others' opinions. 5 4.95 4.87 Paid close attention to important details. 5 3.65 4.63 Remained engaged when team's results not very good. 5 4.5 4.67 Put forth good effort. 5 3.4 4.6 Overall Quantity of Work Score 5 4.09 4.64 Quantity of Work Self- Ratings Peer Rating of You Team Aver- age Overall Self-management/ Accountability Score 33.3 25 33.33 Comments Student 1 – Great team. Student 2 - Needs to translate ideas into sim inputs. Student 3 - Never responded to emails about the simulation. Page 146 - Developments in Business Simulation and Experiential Learning, Volume 43, 2016 pendability  Quantity of Work – covers willingness to do a “fair share” of necessary work. Details are shown in exhibit 1. FINDINGS Preliminary Findings The research question was originally evaluated based on the data gathered from one class with six teams. Additional data will be collected from more classes during future semesters to extend this study. An example from a Fall 2015 class is included under additional findings below. 1. The BSC ranking of the six teams. Exhibit 2 shows the BSC ranking of the six teams. As can be seen, the team Chester had the highest BSC score and therefore was considered the best team with respect to task work. 2. The peer evaluations graphics. Exhibit 3 shows an example of the graphics that is available from the peer evaluations for one member of one of the six teams. 3. The peer evaluations of the six teams. Exhibit 4 shows the details of the peer evaluations of one member of the six teams. Based on the data from all the members of the six teams, it was determined that team Chester had the highest team work score. In the original exploratory study, this is the same team that had the highest task work score which vali- dates our assumption for this one case. Additional data will be collected on additional graduate and undergraduate clas- ses – see additional findings below. Additional Findings Additional data was collected on a graduate strategy course which completed in Fall 2015. This data illustrates some of the complexity in evaluating team performance as discussed below. The class was a small class of 8 students which was divided into 3 teams (Baldwin, Chester, and Digby) of 3, 2, and 3 students with the remaining 3 teams being computer teams with average ability. Two of the teams (Baldwin and Digby) had outstanding BSC scores as shown in exhibit 5 but Digby had team peer evalu- ation problems – one of their team members didn’t contribute to the team - as shown in the team peer evaluations in exhibit 6. However the other 2 team members were able to overcome that lack of contribution and still placed first in the BSC scoring. Even though the peer evaluations did not predict the BSC winner as in the original study, this additional data set still pro- vided some new insights into evaluating team performance. Lack of a team member’s support can be overcome by the other team members providing additional work. The capsim peer evaluation was a useful tool for team members evaluating each other. They did rate one of their members very low instead of the typical evaluation of just saying everyone did their share. Since the cap- sim peer evaluation has multiple areas for evaluation, it seems that the students were more thoughtful in assigning team member scores. Discussion University and government reports show that team work is now required in today’s workplace. Universities are increasing the use of team work in almost every class. However, grading of individual performance is more complex with the extensive use of teams. This exploratory study showed how improved peer evalua- tions can be a useful tool to determine individual performance in a group setting. However, due to the complexity of team work, dedicated teams seem to be able to overcome lack of contribution of some team members and still be very successful in the task work. These improved peer evaluation are helpful in evaluating classes that use team based projects but are not a perfect predictor of team task work due to the complexity of team work and the possible additional effort by a few dedicated team members. Ad- ditional data will be collected for further insight into the complex relationship of task work and team work. Anderson, S. and Coffey, B. (2004). The student’s view of a business simulation: Perceived value of the learning expe- rience, Proceedings of the Academy of Management Con- ference, New Orleans, Louisiana, August, 2004. Capsim: New Peer Evaluation Instructor Guide (2015), Capsim Management Simulations, Inc. Chasteen, L. and Damonte, S. (2007). International Business Education: Insight from the Changing Environment of the EU. Proceedings of the Southwest Academy of Manage- ment Conference, San Diego, California, April, 2008. Chasteen, L. (2014). Simulations for Strategy Courses: Compar- ing Online and On-campus courses. Proceedings of the ABSEL Conference, Orlando, Florida, March, 2014. Dale, Edgar. (1969). Building a Learning Environment. Bloom- ington, Ind.: Phi Delta Kappa Educational Foundation Lam, S. K., & Shaubroeck, J. (2000). Improving group deci- sions by better pooling information: A comparative ad- vantage of group decision support systems. Journal of Ap- plied Psychology, 85(4): 565–573. www.capsim.com (2015). Website of Capsim Management Simulations, Inc. REFERENCES