Simulations for Strategy Courses: Difficulty vs. Realism, Preliminary Findings Page 64 - Developments in Business Simulation and Experiential Learning, Volume 45, 2018 ABSTRACT Simulations are an important part of capstone strategy courses - they facilitate the transfer of knowledge, skills and ability by providing “learning-by-doing” opportunities to the students. They also allow instructors to provide authentic activities situated amidst relevant context, enable learners to grasp not just ‘how’ to do an activity, but the ‘why’ the ‘what, and the ‘with whom.’ Simulations have become an accepted part of strategy classes both at the undergraduate and graduate levels. To be explored is the impact of difficulty versus realism on simulation. Professors using simulations believe anecdotally what the literature on simulation has suggested for decades: that the simulation should be as realistic as possible, but not too complex that students lose interest and give up. However, detailed measurements are lacking. Previous investigations have shown that this is a complex issue - sometimes simple simulations lead to good team success, but sometimes they don’t. There are many influencing factors. This paper uses a recent dataset collected at a US university in an exploratory study to see the relationship between difficulty, realism, and team results. INTRODUCTION Simulations are an important part of capstone strategy course. Edgar Dale (1969) illustrated this with his research when he developed the "Cone of Learning." This concept states that after two weeks we remember only 10% of what we read, but we remember 90% of what we do! Simulation-based education puts learning objectives into the context of a scenario which allows the learner to experience the concepts as they relate to a life-like situation. The use and need for teams are well documented in contemporary firms for a variety of purposes and across a variety of industries. Today’s technology is too complex for employees to work entirely on their own. Good business simulations have followed this general approach. Simulations require students to apply functional business knowledge while performing in a team- based context, i.e., making decisions about business strategy and operations as members of a top management team (Anderson and Coffey, 2004). The best simulations are team based with each member having a specified role in the simulation – good simulations are also too complex for just one person to play alone. However, what is the relationship between difficulty and simulation results? Professors using simulations believe anecdotally what the literature on simulations has suggested for decades - that the simulation should be as realistic as possible, but not too complex that students lose interest and give up. However, detailed measurements are lacking. It is difficult to turn anecdotes into evidence without collecting a large data set. However, a recent large dataset collected at a US university allows an exploratory study to see the relationship between difficulty, realism, and team results. CAPSIM SIMULATION Capsim, Inc offers one of leading business simulations. This business simulation engages participants in a dynamic competition to turn struggling companies into successful, profitable businesses. Dividing students into teams that compete against each other by making strategy, finance, production, and marketing decisions that interact to grow their business. Instructors focus on using the simulation experience to reach defined learning goals. Simulations for Strategy Courses: Difficulty vs. Realism, Preliminary Findings Larry Chasteen University of Texas at Dallas chasteen@utdallas.edu James Szot University of Texas at Dallas jimszot@utdallas.edu Richard Teach Georgia Institute of Technology Richard.teach@scheller.gatech.edu Page 65 - Developments in Business Simulation and Experiential Learning, Volume 45, 2018 With each round of decisions (each representing a full year for the company), participants build their business acumen and decision-making ability as they interpret data, shape strategies, and discuss the results. A wide variety of Capsim results, data, and services support the creation of a dynamic, highly interactive learning experience. Delivered online, in the classroom, or a combination of both, and delivered in condensed or expanded time frames, Capsim simulations have the flexibility to adapt to many academic or corporate curricula. Capsim simulations have been used extensively at more than 500 business schools and leading corporations in the US and around the world (Chasteen and Damonte, 2007). BUSINESS EDUCATION A capstone strategy class is usually one of the final courses taken for both undergraduate and graduate business programs. This course integrates all the material from previous classes such as marketing, accounting, and finance. Students draw on their awareness of various environmental influences (social and political) to solve business problems. They examine management alternatives with an ethical perspective relating policy trends to the strategic planning mode (Chasteen, 2014). The purpose of a capstone course is to integrate the learning achieved in individual business courses taken to earn a business degree. The knowledge acquired in finance, accounting, operations, management information systems, marketing, and organizational behavior classes is utilized to study the strategic management of the firm as well as the responsibilities of the general manager. The use of extensive case studies embeds sets of knowledge in the minds of the business students. This model has proven effective and has been copied by business schools worldwide. Another approach to integrating the learning achieved in individual business courses taken to earn a business degree is the use of simulations. Students participate in a simulation that requires taking into account multiple decision-making factors while balancing all sectors of the firm’s environment. According to Dale (1969), an active learning method can provide even better results. Therefore, simulations have become a standard part of many US undergraduate and graduate capstone strategy courses. They are also becoming common in strategy courses in many other countries. DIFFICULTY VS. REALISM In simulations, realism aids in learning, particularly in the process of transferring learning from the conceptual base to its ultimate application. The strong relationship between the degree of perceived realism and the perceived contribution of the business game to learning was earlier observed in a study assessing the perceived realism of a single game. (Dittrich, 1975). However, Dittrich’s 1975 article did not precisely define realism. Hall (2015) gives a more detailed definition that represents two viewpoints. “External Validity” is replicating the real world as exactly as possible. “Psychological Validity” focuses on the relevance of the simulation to the workplace and if this improves job performance. In another study, Dittrich (1976) states that management games can be compared for differences in perceived realism in the three major functional areas of business administration (e.g., marketing production, and finance), and in the realism of their respective interactions. Thus, the degree of realism of a simulation can provide important data on business games, to potential adapters, and especially to those considering games for use in the business policy course. If one wants realism in a simulation, it results in a more complex and more difficult to play simulation. This conflict between the degree of realism and the level of difficulty remains elusive. An early paper by Joseph Wolfe (1978) used two simulations, one less complex than the other. Professor Wolfe hypothesized the differences between the two competitions, were caused by the differences in the complexity of the two simulations. As previously noted, professors using simulations believe anecdotally what the literature on simulation has suggested for decades - that the simulation should be as realistic as possible, but not too complex that students lose interest and give up. However, detailed measurements are lacking. Previous investigations have shown that this is a complex issue - sometimes simple simulations lead to good team success, but sometimes they don’t. There are many influencing factors. This paper uses a recent dataset collected at a US university in an exploratory study to see the relationship between difficulty, realism, and team results. THE PILOT SURVEY Pilot surveys were conducted at a large southern university during the summer term 2017. Three instructors using three different simulations forwarded an anonymous Qualtrics survey to their students. This preliminary study will focus on the results from one of the instructors that used teams for his simulation. It was felt that team-based simulations with each member having a specified role in the simulation was the most realistic approach. Good simulations are too complex for just one person to play by himself. However, what is the relationship between difficulty and simulation results? Results are from two graduate strategy courses – an executive MBA class and a traditional MBA class. Page 66 - Developments in Business Simulation and Experiential Learning, Volume 45, 2018 PURPOSE AND RESEARCH QUESTIONS The purpose of this study is to examine if student success in the simulation was related to the difficulty and realism of the simulation. The topics covered in the course were traditional class lectures, exams, case discussions, and a business simulation. We implemented the simulation exercise with a web-based simulation game. The simulation lasted eight rounds (eight years) and was scored by using the Balanced Scorecard (BSC). We addressed the following two issues: 1. Is there a relationship between the scoring by a team with their view on the realism of the simulation? 2. Is there a relationship between the scoring by a team with their view on the difficulty of the simulation? PROCEDURES The simulation exercise used the Capsim Foundation Simulation. The students in the class were divided into teams to compete in a computer simulation by managing an imaginary firm that manufactured electronic sensors. The teams had to make research and development, production, marketing, and financial decisions concerning the product. The teams entered their decisions into the simulation and then analyzed the results once all the other team decisions were entered and processed. The simulation lasted for eight rounds representing eight years. The executive MBA class had nine students divided into three teams (there were also three computer teams for a total of six in this simulation). The traditional MBA had thirty-six students divided into two sets of six teams. RESULTS The Balanced Scorecard scoring from the simulation was used to compare student success in the classes. The Balanced Scorecard allows companies to gauge their performance by assessing measures in four categories: • Financial - includes profitability, leverage and stock price; • Internal Business Process - ranks, among other measures, contribution margin, plant utilization and days of working capital; • Customer - examines the company's product line, including how well it satisfies buying criteria and awareness/ accessibility levels; • Learning and Growth - evaluates employee productivity. The Balanced Scorecard allocates points in each of these four areas for each of the rounds and a final recap score. The team with the highest BSC is considered to be the winner of the simulation rather than just the team with the highest stock price or highest profits. Since the Balanced Scorecard allocates points in four major sections, it is considered a more representative measure of success. SURVEY DATA This study uses responses from a subset of the questions in an extensive survey designed for multiple purposes (Teach and Szot, 2018). Students in both classes were asked to complete this survey anonymously. Students were asked to evaluate both the difficulty and the realism of the simulation by indicated their level of agreement with the statements: • Reflecting on the effort and rewards of participating in the simulation: The simulation took too much time. • Reflecting on the effort and rewards of participating in the simulation: The simulation was very unrealistic. Table 1 summarizes a preliminary comparison of the student responses with their Balanced Scorecard results. TABLE 1 – RESULTS VS. DIFFICULTY VS. REALISM Results Difficulty Realism Best Exec teams Not too much time Realistic Worst Exec teams Too much time Not Realistic Best Trad teams Too much time Both - average Worst Trad teams Not too much time Both - average Page 67 - Developments in Business Simulation and Experiential Learning, Volume 45, 2018 FINDINGS PRELIMINARY FINDINGS The research question was evaluated based on the data gathered from one Executive MBA class with three student teams and three computers and one traditional MBA class with six teams. Additional data will be collected from more classes including undergraduate classes during future semesters to extend this study. 1. The executive MBA class divided into two groups – good performers and poor performers. The good performers felt the simulation was realistic and didn’t take too much time. In general, they liked the simulation. The poor performers felt the simulation was not realistic and took too much time. In general, they did not like the simulation. Results seem to track their view of the simulation. 2. The traditional MBA class also had good performers and poor performers. However, both the good performers and poor performers felt the simulation was realistic. However, the good performers felt that the simulation took too much time, but they did what was required to get a good score. The poor performers felt that the simulation did not take too much time and probably did not put in the effort to get a good score. In general, both groups liked the simulation. 3. Based on this limited sample set, it seemed that if the executive MBA class had a good impression of simulation, they would do what was required to get a good score. If the executive MBA class had a bad impression of simulation, they did not do the work required to get a good score. 4. Based on this limited sample set, it seemed that all students in the traditional MBA class had a good impression of the realism of the simulation. However, the good performers would do what was required to get a good score while the poor performers did not do the work required to get a good score. Additional data will be collected on additional graduate and undergraduate classes to see the difference with undergraduate students. The assumption is that undergraduates will follow the traditional MBA – think that the simulation is realistic and do all that is required to get a good score. DISCUSSION We have created the traditional 2x2 matrix, commonly used in many management studies showing the two variables (difficulty and realism). A general feeling of the realism of the simulation is needed to generate the initial energy and commitment the simulation. The degree of difficulty also has an impact on how well the team does on the BSC scoring. The four quadrants illustrating the resulting situations and where they fit on this 2X2 matrix is shown in Table 2. Additional studies are needed to collect more data on all these four quadrants. CONCLUSIONS As this is only the results of a small-scale pilot study, we make no conclusions. However, we can claim that many more interactions exist among the different groups of students when they experience a business simulation than we previously thought. The authors expect to continue this study on a much larger and more representative sample of students and business programs within the coming year. TABLE 2. THE FOUR QUADRANTS OF DIFFICULTY VS. REALISM Page 68 - Developments in Business Simulation and Experiential Learning, Volume 45, 2018 Anderson, S. and Coffey, B. (2004). The student’s view of a business simulation: Perceived value of the learning experience, Proceedings of the Academy of Management Conference, New Orleans, Louisiana, August 2004. Chasteen, L. and Damonte, S. (2007). International Business Education: Insight from the Changing Environment of the EU. Proceedings of the Southwest Academy of Management Conference, San Diego, California, April 2008. Chasteen, L. (2014). Simulations for Strategy Courses: Comparing Online and On-campus courses. Developments in Business Simulation and Experiential Learning, Volume 41 Dale, Edgar. (1969). Building a Learning Environment. Bloomington, Ind.: Phi Delta Kappa Educational Foundation Dittrich, John. (1975). Perceptions of Realism: An Approach to Measurement in Simulation Applications. Proceedings of NASAGA, (Fall, 1975). Dittrich, John. (1976). Realism in Business Games: A Three Game Comparison. Computer Simulation and Learning Theory, Volume 3, pp 273 to 280. Hall, Jemery. (2015). Business Simulations: Reality AND BEYOND! Developments in Business Simulation and Experiential Learning, Volume 42 Teach, R. and Szot, J. (2018). How students “play” business simulations and what they learn. Proceedings of the ABSEL Conference (forthcoming), Seattle, Washington, March 2018. Wolfe, Joseph, (1978). The effects of game complexity on the acquisition of business policy knowledge. Decision Sciences, Vol. 9, No, 1, pp 143to155. www.capsim.com (2015), is the Website of Capsim Management Simulations, Inc. REFERENCES