Using Business Simulations to Prepare Students to Think Critically, Make Better Decisions, and Solve Business Problems Page 283 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 ABSTRACT Business simulations are a type of experiential learning that prepare students for skills that they will need in the workforce. This learn by doing approach brings the real business world right into the classroom and develops and enhances students’ ability to think critically, make better decisions, and solve business problems. This paper will describe what business simulations are, their types, benefits and limitations, and how they are used. Then, this paper will discuss how to select and use a simulation in a course, including integrating interdisciplinary content and using a Simulation Implementation Framework. Finally, examples will be provided describing how the framework was applied to two courses. Keywords: business simulation, experiential learning, critical thinking, decision making, business acumen, leadership INTRODUCTION AND BACKGROUND A simulation is a connector of theory and practice. It is a springboard to various types of content and real-life examples. The more realistic the simulation and its supporting instructional tools such as case studies, current real-world examples, and industry speakers, the more relevant and practical insights can be obtained. How does one prepare for something that they have yet to experience? How do you prepare others to think critically and make better decisions in a risky or stressful environment? Where can one make mistakes in a ‘risk-free’ environment? How do you create real-life conditions that others can learn and retain from in a resource-effective way? One can read about it, have someone explain it to them, or observe someone else doing it. However, just like learning how to ride a bicycle or to drive a car, learning by actually practicing it is a very effective way. Simulations are one effective way to learn by doing. Simulations are already used in many aspects of our lives to prepare us for planned and unplanned events. For example, emergency responder preparedness drills, fire drills, airplane and rocket flight simulators, military exercises, surgery and sports-game scenarios practice, and management assessment centers all save time, money, and other resources. Business simulations provide a safe learning environment for students to practice managing tradeoffs and using judgement to make the preferred business decision given the context. Computer-based simulations are “a computer model of a real-life system or process represented in an abstracted or scaled-down form” (Lunce, 2004, p/29). “They facilitate “interactive practice” (Heinich, Molenda, Russell, Smaldino, 1999) of real-world skills by focusing on essential elements of a real problem or system” (Lunce, 2004, p/30). Simulations are one form of experiential learning (EL). EL is also known as learning by doing or learning through action, experience, discovery, or exploration. EL principles include (1) structure so students take initiative and are accountable for results, (2) critical thinking and analysis, synthesis, and reflection, and (3) personal learning results that create future learning opportunities (AEE, 2011). The EL process includes (1) a hands-on experience with little instructor help, (2) reflecting on what happened, (3) analyzing what is important, (4) identifying real-life principles, and (5) applying lessons learned to other situations. Some other higher education EL opportunities include apprenticeships, clinical experiences, coops, fellowships, field work, internships, practicum, service learning, student teaching, study abroad, and volunteering (NIU, 2014). Since computer-based simulations are dynamic and flexible, they can facilitate the achievement of specific learning outcomes (Gibbons, Fairweather, Anderson, & Merrill, 1997). They are interactive and engaging, provide formative feedback, help students to understand results, and facilitate critical thinking (Granland, Bergland, & Erikksson, 2000). Business simulations can come in various forms such as computer-based, web-based, board games, discovery maps, and role plays. They can be used in person or remotely, synchronously or asynchronously, and by an individual or a group. Instructional tools when using a simulation include prework, simulation briefing, software demonstration, decision-making round(s), results analysis and debrief, linkages to content taught and real-life examples, and reflection. The two major categories of business simulations are total enterprise simulations (TES) and scenario-based simulations (SBS). Both provide opportunities for critical thinking, decision making, feedback, reflection, and performance metrics. TES enable students to comprehend how a business operates and the interconnectedness of its various functions. They typically feature the integration of various business acumen disciplines such as strategy, marketing, finance, and operations. Students typically assume the leadership of an existing business, make annual decisions, and manage it through several simulated years of operations. This enables students to sense, know, experience, and find out the challenges that business leaders confront, analyze, and overcome on a regular basis. Characteristics of TES include a student-run company competing against other student-run companies or auto-competitors in an industry. When there are several teams competing against each other over several rounds, students develop the ability to make adjustments based on the external environment’s interactive effects, i.e., the competitors within an industry (Anderson, 2005). The annual process for each simulated year includes strategy formulation, planning and analysis, tactical decision making, a Using Business Simulations to Prepare Students to Think Critically, Make Better Decisions, and Solve Business Problems James V. Caruso jcaruso91@gsb.columbia.edu Page 284 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 review of the results, and individual and group reflection. Adding more decisions and variables makes the simulation more complex and realistic (Willis, Hovey, & Hovey, 1987) and it involves more financial investment and time. SBS typically highlight one academic discipline within business acumen or leadership.. Topics include strategy, marketing, business analytics, mergers and acquisitions, change management, influence, teamwork, negotiation, and conflict. The length of a decision-making round is typically briefer than a TES round, and there can be more than one round. To facilitate a SBS, reading the case scenario, making decisions, analyzing and reflecting on the feedback and results, and debriefing the experience are usual steps in the process. There are many benefits to business simulations. Students can learn by doing, from others, and by making mistakes in a ‘risk-free’ environment that is an abstraction of reality that compresses time and provides immediate feedback. Learning from mistakes in this experiential learning laboratory is also known as “just-in-time” or “just-a-little-late” learning (Teach & Patel, 2007). Simulations immerse students into a micro-world that is engaging and provides them the real-world business context to reinforce the academic discipline being taught (Schiflett, Elliott, Salas, & Coovert, 2004). Since computer-based simulations perform calculation and record keeping tasks, students can keep their focus on the instructional objectives (Willis et. al., 1987). In a multi-round simulation, students can correct their mistakes in the next round. They can learn about their mistakes from the instructor debrief, individual/group analysis and reflection, or by conducting an after action review (AAR). Students use the AAR process to discuss what they intended to do, what actually happened, and what do they need to do to improve their performance in the next round. Students can use reflection as a scaffolding mechanism to obtain deeper insights with each round. This enables them “to redo their actions by focusing on the point of error instead of restarting from the beginning” (Scoresby & Shelton, 2014, p/1). There are several limitations to using computer-based simulations compared to other instructional methodologies. These include cost, resources, time, complexity level, relevance, instructor learning curve, facilitation, and shelf-life. Simulation development requires significant financial and human capital resources for design, development, and testing, as well as developing supporting student and instructor materials. Instructors need to spend significant preparation time before using the simulation in a class. They need to understand the simulation context, navigation mechanics, decisions, the results from specific decisions, results analysis, and technical troubleshooting. The instructor also needs to know when and how to perform the role of instructor, facilitator, consultant, and coach. Students may not obtain much from discovery learning if the instructor does not effectively perform these roles (Min, 2001; Heinich et. al., 1999). Computer-based simulations also generally take more time since students need to spend significant preparation time to be able to successfully immerse themselves in the business problem (Heinich et. al., 1999). Students may also misinterpret the real business world problem since simulations oversimplify the real world (Heinich et. al., 1999). Finally, regardless of a simulation’s complexity, the business context related to the simulation may change tremendously that it is no longer relevant and useful (Heinich et. al., 1999). HOW SIMULATIONS ARE USED There are numerous ways that business simulations can be used to teach various content disciplines. Here are some examples of how academic content has been integrated with business simulations based on a literature review and the author’s extensive experience facilitating over 265 programs and courses using TES and SBS simulations with corporations and graduate students. There is a wide range in the level of complexity of TES, ranging from simple (Level 1) to complex (Level 5). Level 1 includes single-player simulations with individual or team play. Level 2 includes multi-player simulations with a smaller number of products and markets. Level 3 simulations are multi-player with a larger number of products and markets. Level 4 are Level 3 simulations, but the team and decisions are subdivided. They can include negotiated transactions with other teams and a much larger business ecosystem where the industry players determine if they want to make, buy, and/or sell a diverse range of products and services. Finally, Level 5 includes Level 4 simulations that are customized, possess more complex negotiations, and provide team and individual participant coaching. TES Levels 1-3 have been used to teach finance and accounting concepts such as financial management, financial statements, financial statement analysis, margin analysis, business performance measures, capital investment analysis, and shareholder value. Strategy, marketing, and operations have been individually taught using SBS covering such topics as strategic planning and alignment, the marketing mix, supply chain, forecasting, and data analytics. The integration of all these business acumen topics have been taught using TES Levels 2-3. Additional topics covered included strategy implementation, customer segmentation, product portfolio analysis, and production and inventory planning. Management researchers have found that the MBA curriculum has become less-than-relevant to the practice of management (Podolny, 2009). This is because business schools have moved away from relevant and practical programs and toward a more rigorous teaching approach to mirror the hard sciences’ academic excellence standards (Bennis & O’Toole, 2005). This has reduced students’ ability to develop their ability to think from creative, critical, systems, and multi-discipline perspectives (Wallace, 2010). However, successful business capstone courses integrate various disciplines of the entire business school curriculum (Inamdar & Roldan, 2013) and are enhanced by using a TES. Many MBA capstone courses use a TES placed at the end of the core curriculum or the program. For example, the University of Pittsburgh’s MBA Capstone course (Magnuson & Good, 2017) uses a TES as an internship experience where students apply concepts that have already learned. Methods to make this feel like a real-world assignment include (1) using terminology (executives vs. students), (2) executives applying for company positions, (3) the company president determining compensation increases and conducting performance evaluations, (4) networking, and (5) having a board of directors approve capital expenditure decisions (Magnuson et. al., 2017). Some MBA programs use a TES at the beginning of a program. Pepperdine University uses a TES to integrate coursework Page 285 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 throughout their MBA program. They have two integration modules, one at the end of the first four core courses, and then one at the end of the last four core courses. The simulation is used in the two integration modules and the final strategic management course (Green & McQuaid, 2007). To instruct leadership topics, SBS have been typically used. Examples of topics taught include teamwork, influence, change management, conflict, performance management, coaching, project leadership, and sales account management. SBS have also been used to integrate business acumen and project leadership. Finally, business acumen and leadership content have been integrated using TES Levels 3-5. Additional topics included business portfolio management and make/buy/sell partnerships with other companies. Finally, other instructional tools beyond the computer-based simulation included union negotiations and executive coaching. As described in Caruso (2018), SBS are typically implemented in one or two class sessions. One instructor (Beaudry, 2016), integrated the Leadership and Team Simulation: Everest v2 (Roberto & Edmondson, 2011) and related themes (Roberto et. al., 2011) over an entire organizational behavior course (Beaudry, 2016). The course was designed into three modules (individual, team, and organization). Individual theories, such as values and motivation, were taught in the individual module. Then, the students applied these theories using self-assessments and to the mountaineers, Edmond Hillary and Tenzing Norgay (Beaudry, 2016). Team concepts such as influence, power, team effectiveness, communication, and conflict were taught in the team module. Then, the students created a team charter, became acclimated to the five simulation team roles, implemented simulation decisions, captured their reflections in a journal, and produced a video of their entire team experience. Finally, during the organization module, all teams combined to create a mountain expedition company and determined their name, purpose, roles, structure, and culture. Here, the students practiced instructing and learning in mixed groups and delivering proposal presentations (Beaudry, 2016). One university developed an engaging computer-based consulting simulation for students that helped them develop their critical thinking, higher order analysis, and synthesis skills (Bos & Gordon, 2005). To stress the importance of partnering with customers, students in a marketing course used relational marketing concepts to reduce transaction costs in the cost function of a marketing simulation (Cannon, Cannon, Koksal, & Johnson, 2014). Finally, simulations can be designed as self-paced individual study without instructor involvement by including an automated consultant to provide feedback to students. The “Beat the Market Online: A Microeconomic Game” (Gold, 2017) is used as a foundation to economics or business for large class sizes. The simulation includes two levels (introductory and advanced) and various market environment scenarios from highly concentrated to highly competitive (Gold 2018). HOW TO USE A SIMULATION IN A COURSE The paper will now describe how simulations can be incorporated into business courses, by first understanding how simulations are designed, and then how they can be integrated into courses. Then, a Simulation Implementation Framework will be introduced along with the author’s two examples of using the framework, highlighting the thought process in selecting and using a simulation. When designing a simulation, there are various types of complexity that impact time and cost (Hall, 2007). Decision complexity refers to the amount and type of decisions. The type and structure of reports for participants and instructors is reporting complexity. The structure and size of the simulation model is model complexity while the structure and size of the database is data complexity. The number of help screens refers to online help complexity. Finally, a simulation’s complex feedback loops and dynamic behavior are known as calibration complexity (Hall, 2007). Cannon looks at complexity from two dimensions – information load and uncertainty. Simple simulations are low on both scales, while complex simulations are high on both scales. Complex simulations help participants practice real-life business decision making (Cannon, Friesen, Lawrence, & Feinstein, 2009). More complex simulations can use a systems-dynamic design approach. Instead of individual functional perspectives that are linear, systems-dynamic simulations look at the interconnectedness of the firm in a non-linear way. Various algorithms calculate market demand, firm demand, cost and production, profit, and stock price (Gold, 2003). There are many types of functional decisions that can be included in a total enterprise simulation. Marketing decisions can include price, advertising, sales representatives (number of, salary, training, and commission), research and development, marketing research, and multiple products and markets (Goosen, Jensen, & Wells, 1999). Production decisions can include plant capacity changes, scheduling production, automation, quality control, overtime, and purchasing materials. Finally, finance decisions can include short-term debt, long-term debt, emergency loans, stock issuance, dividends, and credit and supplier terms. Decisions are the input and financial statements are the output. Finally, designers need to choose whether to use external or internal financial statement reporting formats (Goosen et. al., 1999). Successful simulation designs incorporate game design principles and collaboration between business management educators and instructional technologists (Byers & Cannon, 2007). The items that are part of a simulation design document provide context to help users of simulations, such as teachers and instructional designers, select which simulation to use. These items include objectives, content topics, detailed description, and audience target, size, and knowledge (Byers et. al., 2007). Courses should be designed based on the course’s learning outcomes, instructional design theory, and adult learning principles. Computer-based simulations are one of many instructional methodologies available to use when designing courses. To increase engagement and interactivity, a mix of instructional design methodologies should be purposefully structured. These include lecture, discussion, examples, exercises, case studies, simulation, experiential learning, reflection, and guest speakers. Goosen (2002) states that simulations are a dynamic case study and “the simulation should not be regarded as means of replacing other methods of instruction including lectures. Simulations should be regarded as an application tool that makes other instructional methods more effective” (Goosen, 2002, p/311). The instructor must determine the course learning objectives prior to selecting which simulation to use. This task is facilitated when simulation designers clearly state the simulation’s learning objectives (Teach & Patel, 2007). Simulations are a teaching tool and potentially a learning tool (Goosen, 2002). The knowledge, skill, and direct involvement Page 286 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 of the instructor provide the learning value. Using simulations and the instructor’s integrated knowledge of business acumen enhance the instructor’s credibility and achievement of course objectives (Goosen, 2002). He states that enrichment techniques (“teacher + experience = student learning”) (Goosen, 2002, p/314), and not the simulation itself, facilitate learning from simulations as the simulation is the pedagogical tool chosen by the instructor to expedite knowledge transfer. Examples of enrichment techniques that provide measurable and significant learning include students writing and presenting oral reports such as strategic plans and analyzing performance, all using knowledge and instructions provided by the teacher (Goosen, 2002). In addition, it is important for the instructor to advise the students of the simulation’s strengths and limitations (Goosen, 2002). This helps to manage students’ expectations of the course and simulation. Finally, debriefing the simulation after each round enables students to learn from their mistakes and make better decisions in the next round (Goosen, 2002). According to Gold (2015), the instructor’s ability to integrate a comprehensive simulation and scaffold simulation exercises based on specific topics throughout the course, helps develop students’ critical thinking skills. Instructors need to choose when to introduce a business simulation in a course. Will the simulation be used at the beginning, middle, end, or throughout the entire course? When used at the start, the simulation immerses the students in the content at a high level and motivates them to learn content over the course that they do not know (Anderson & Lawton, 2003). Thus, they can learn from their mistakes. When used at the end, it becomes a ‘capstone’ and reinforces and applies the content presented throughout the course. A benefit of using it throughout the course is that the content and simulation lessons learned can be scaffolded, enabling students to grasp, apply, and retain content. Things to consider when introducing a simulation include the simulation’s scope, the students’ level of preparedness, and the instructor’s goal for using the simulation (Anderson et. al., 2003). New concept introduction, practical concept application, and the analysis of complex business context information are all types of objectives (Anderson et. al., EXHIBIT 1 SIMULATION IMPLEMENTATION FRAMEWORK Course/Program Design • When will students take the course during their program? • How does this course fit within the entire curriculum? What is the program goal? • Is the course a required or an elective course? • What are the learning objectives of the course? • Do other courses in the curriculum use simulations and for what purpose? • What content is being taught and applied in this course? • How is the course delivered – face-to-face, blended, or online/virtual? • What is the length of the course and each class session? • When will the simulation be used during the course – beginning, middle, end, or throughout the entire course? • What will be the order of the content topics and simulation rounds? Will the content be scaffolded? Will a flipped classroom be used? • What other instructional design methodologies and assessments are used in the course? • What current real-life examples will be used? Will examples from the students’ companies and industries be used? • How will students be engaged? What methods will be used to make students accountable and take ownership for their learn- ing? • What are the budget and resources (financial, time, and quality) for this course? Students • What is the student level – high school, undergraduate, graduate? • What is class profile – business students, non-business students, or both? • What is the minimum and maximum enrollment for this course? • What is the actual class size? • Where are the students located? • What are the recommended minimum and maximum number of student teams? What is the ideal number? • What are the recommended minimum and maximum number of students per team? What is the ideal number? Instructor • Will internal or external instructors be used? • What is the instructor’s comfort level with facilitating simulations? • Does the instructor possess an integrated knowledge of the business acumen and leadership topics that can be covered using the simulation? • How many instructors are needed for the course? • Is a teaching or research assistant needed? • What role(s) will the teacher perform – instructor, facilitator, consultant, and/or coach? • Will a train-the-trainer session be used to acclimate multiple instructors and to insure consistent delivery? Page 287 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 2003). Sometimes, simulations are used to gain various behavioral and intellectual objectives so that students can see the inter- relationships between performance, analysis, and concepts (Anderson et. al., 2003). When introducing a simulation to students and setting expectations, an instructor should explain the gap that exists between the simulation and the real world by explaining what the simulation mirrors and what it does not (Heinich et. al., 1999; Sternberg, 1999). The gap can be bridged by connecting to current real-world examples and students’ personal experience that reinforce the content and course objectives. This can be accomplished during the simulation debriefs, content lectures, or by incorporating industry guest speakers. To develop critical thinking skills, the Socratic teaching method can be facilitated during lectures, debriefing sessions, and while teams are making decisions. This helps students figure it out for themselves. To hasten the simulation learning curve, instructors can take advantage of the numerous resources made available from the simulation providers. These include student and instructor manuals, slides for content and simulation briefings and debriefings, software demonstrations, webinars, and instructor-led workshops. How do you assess students using a simulation in a learning environment? Measuring what knowledge and learning students obtain is better than just using firm or product performance. Various financial metrics have been used such as cumulative profit. However, if a team has a very challenging year in the beginning, it is very difficult to catch up to the dominant industry leader. For example, 85% of dominant firms never lost their lead after three years (Teach & Patel, 2007). Broad-based metrics can be better, especially if they measure both financial and non-financial metrics such as the balanced scorecard (Teach & Patel, 2007). The Balanced Scorecard (Kaplan & Norton, 1996) is used by many companies in business practice and is also a great way to measure performance in business simulations given its financial, customer, internal, and learning and growth perspectives (Dickinson, 2003). One resource to help instructors and course designers determine which simulation to use is The Typology of Serious Games and Exercises (Baldissin, Greco, Nonino, & Wolfe, 2018). It is a good starting point prior to searching for more information about specific simulations online or directly with the simulation provider. It is a classification system that provides descriptions and reviews of over 385 simulation and experiential learning entries. The typology sections are “activity administration, participants/ relationships, engine/model, interface, outcomes, and ancillaries” (Baldissin et. al., 2018). Key classifications include “name, content topic, adopter goal (e.g., assessment, education), and sophistication level (e.g., high school, college, graduate)” (Baldissin et. al., 2018). The typology (Baldissin et. al., 2018) is now applied to two simulations that were used in examples described in more detail later in the paper. For the TES example, the typology provides key information to understand Capsim’s Foundation simulation (Capsim Foundation, 2018). It states that high school, undergraduate, and graduate students who are introductory simulation players can benefit from the simulation. It can be run for up to eight simulation periods and can be played with one to six teams. The simulation provides various coaching and feedback reports to the students, plus there is a peer-based team evaluation tool. Since it is a TES, it has broad strategic management content applicability with depth in finance, marketing, production, and research and development. Finally, it lists support materials such as participant and instructor guides, onboarding tools, and functional videos (Baldissin et. al., 2018). According to the typology (Baldissin et. al., 2018) for the SBS example, Harvard Business Publishing’s Data Analytics Simulation: Strategic Decision Making (Davenport, 2016) is effective for both graduate and college students. Simulation • What is the budget for the simulation? • Is there discounted pricing for universities? • Will the university or students pay to use the simulation? • What simulation works best for this course and its objectives: • Total enterprise or scenario-based? • Off-the-shelf/generic or customized? • Breadth and depth of content? Industry-focused? Functional-focused? • Can the instructor tailor complexity? • Are optional modules available? Will they be used? • Is the simulation single-player (individual) or multi-player (team)? • Will the students compete against other student teams and/or computer teams? • How will the teams be determined? • Will the simulation be used synchronously or asynchronously? • Can the simulation be used on a mobile device? • What are the recommended minimum and maximum number of simulation rounds? What is the ideal number? • What is the total time needed to run the simulation taking into consideration the number of rounds, the length of decision- making rounds, demonstrating the simulation software, and briefing and debriefing the simulation round(s)? • Is any hardware required or recommended (i.e., printers)? Resources • Is there technical support available for the instructor and students? When is it available and via which communication meth- ods? • What support materials are available to the participants? • What support materials are available to the instructor? Page 288 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 It is a single-player simulation that needs approximately 75 minutes to finish the four annual decision rounds. This web- based simulation gives students immediate feedback. The instructor can compare results among students when it is conducted in a group setting. In addition, students predict market demand, and set channel price, product formulation, and promotional budgets, enabling them to develop their technical and conceptual skills. Finally, simulation navigation, participant, and instructor guides, debriefing slides, a Harvard case study, a teaching note, and instructor videos are the types of support materials made available to instructors (Baldissin et. al., 2018). The purpose of the Simulation Implementation Framework (Exhibit 1) is to make practitioners, such as instructors and instructional designers, aware of all the considerations in deciding whether to use a simulation in a course, which one to use, and how to use it. Instructors can use this framework to effectively integrate interdisciplinary content and instructional design methodologies, including business simulations. It is designed for use in an academic setting, but it can also be applied and adapted to a corporate setting. The consultative process to make the best decision given the context is enhanced by the instructor’s simulation experience and the application of critical thinking and judgment. The five primary design considerations are course/program design, students, instructor, simulation, and resources. The instructor needs to understand the context in aggregate, considering the program and course goals; student profile; instructor capability; simulation level, time needed, and limitations; and resources. In addition to the individual content disciplines that can be taught using a SBS, Exhibit 2 also demonstrates the multi-discipline content that can be integrated with a TES. Using a mix of instructional methodologies and assessments in a course is recommended after considering which ones are already used in the program, for what purpose, and how to potentially integrate them. Examples include lecture, discussion, simulation, case study, project-based learning, and guest speakers. How will critical thinking be developed? Will sensitivity analyses, an AAR, and reflection be incorporated? The instructor needs to consider how the simulation will be used in the course. How will the instructor facilitate the course on the telling-discovery continuum? Will students learn specific topical content first or will they learn the content after they discover it in a simulation round? How much guidance will the instructor provide? If an overwhelming and challenging scenario is created, EXHIBIT 2 SAMPLE BUSINESS ACUMEN AND LEADERSHIP CONTENT THAT CAN BE INTEGRATED USING A MULTI-PLAYER TOTAL ENTERPRISE SIMULATION Finance and Accounting • Financial statements (income statement, balance sheet, and statement of cash flows) – components, construction, drivers, link- ages, financial condition, and operating performance analysis • Accounting equation; revenue and expense recognition; margin maximization; cost of goods manufactured, cost of goods sold • Financial statement analysis/methods (common size, percent/dollar change, and ratio); performance evaluation, linking finan- cial performance and strategy; cash flow measurement • Financial forecasting; sensitivity/scenario analysis - pro-forma financial statements • Corporate finance; corporate investment and financing decisions - capital budgeting; working capital management; debt and equity financing; capital structure • Cost/volume/profit analysis; variable and fixed costs Strategy and Marketing • Strategic planning, implementation, and alignment • Competitive strategies, 4P’s, marketing strategy, market segmentation, targeting, positioning • Demand; market structure; market research, sales forecasting • Product portfolio analysis • Customer satisfaction Operations and Analytics • Operations optimization; operational forecasting; project management • Gap analysis; AAR; process improvement • Production and total quality management • Quantitative and qualitative analysis tools • Descriptive, diagnostic, and predictive analytics Leadership • Organizational/team structure • Teamwork/team processes; giving and receiving feedback; managing conflict; coaching; change and time manage- ment; conducting effective meetings • Goal setting; reflection • Ethics; stakeholder analysis and management Communication • Oral presentations • Written communications • Communication styles • Data visualization skills • Influencing and negotiation skills Page 289 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 will there be more guidance and ‘hand-holding’ or a ‘sink or swim’ facilitation style? TES multi-player simulations are constrained by the number of teams in an industry and therefore the number of participants per team. For determining the team composition for online courses, the instructor needs to take the student’s time zone into consideration. Since students of both online and face-to-face programs take online courses, the instructor has the option of keeping those two types of students in separate teams. Simulations that can be used on a mobile device provide more flexibility and would especially benefit online delivery. Finally, for face-to-face courses, will the students make team decisions in the main classroom or will there be separate breakout rooms to facilitate more private team discussions? This paper now looks at two examples using the Simulation Implementation Framework to successfully identify and use EXHIBIT 3 APPLYING THE SIMULATION IMPLEMENTATION FRAMEWORK TO A COURSE USING A TOTAL ENTERPRISE SIMULATION – PART 1 MGMT 601 - Managing the Total Enterprise Total enterprise simulation – Capsim Foundation Simulation (Capsim Foundation, 2018) Course/Program Design • Fit within curriculum – integrates core MBA curriculum • Required graduate course for MBA program / elective course for all others • Content – strategic management, financial statements and analysis, forecasting, marketing, team processes, presentation skills • Other courses using simulations – marketing, analytics, new product development • Simulation used throughout the entire course • Scaffolded content and simulation debriefs • Student engagement, accountability, and ownership - prework simulation acclimation, peer feedback, team contribution as- signment • Other instructional design methodologies and assessments used - team board presentations Students • Graduate level • Number of students: minimum = 12; maximum = 36; ideal class size = 24-36, 5-6 student teams of 4-6 students per team • Number of student teams: minimum = 3; maximum = 6; ideal = 6; use simulation’s auto-competitor team(s) if less than 6 stu- dent teams • Number of students per team: minimum = 4; maximum = 6; ideal = 5 • Instructor determines diverse team composition based on multiple criteria (program, major, concentration, work experience (industry, function), and gender) Instructor • Internal and external; 1 per course section • Simulation facilitation comfort level – very high; has facilitated this course and simulation numerous times in all 3 delivery formats • Roles performed – instructor, facilitator, consultant, and coach Simulation • Discounted pricing for universities; students pay • Best simulation for course – total enterprise, multi-player, off-the-shelf, optional modules (HR, TQM, ethics scenario, 360- degree feedback tool) • Students compete against other student teams (and computer team(s) if less than 6 student teams); benefit – will always have 6 companies in industry • Number of simulation rounds: minimum = 5; maximum = 8; ideal = 6-8 Resources • Technical support available for both instructor and participants – phone, email • Student support materials – student guide, presentations, tutorials, demos, “A Hands-on Introduction to Business Fundamen- tals” (240 pages), “Teamwork Toolkit” • Instructor support materials - student support materials plus instructor guide and university account representative Page 290 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 business simulations in required courses of two of Drexel LeBow’s graduate programs - (1) a TES in the MBA program and (2) a SBS in the MS in Business Analytics program. Managing the Total Enterprise (MGMT 601) is a required course in Drexel LeBow’s MBA program. The course learning objectives are: “(1) develop a holistic mindset of managing the total business enterprise across business disciplines by recognizing the interconnectedness of the various functional areas of an organization, its goals, strategies, and tactical decisions, and the overall impact on stakeholders in its business ecosystem, (2) identify and approach business problems and strategic opportunities using a critical thinking and integrated problem solving perspective and formulate and implement optimal solutions within those constraints, (3) continuously scan the business environment, determine and manage strategic direction, balance short-term, long-term, and functional tradeoffs, make decisions aligned with your strategy, and measure results, (4) apply project/quality management, communication, team dynamics, feedback, decision making, and control processes and tools to improve team and individual productivity and effectiveness, optimize operations, and attain common business goals, and (5) demonstrate information literacy by integrating quantitative and qualitative analyses to implement financial and operational decisions that support the achievement of financial, marketing, strategic, and stakeholder goals” (Caruso, 2017, p/3). The course integrates various assessment tools along with business acumen and leadership content using a total enterprise team-based simulation (Capsim Foundation, 2018). The author has taught this course in three different delivery formats – a 4-day residency, 11 weeks face-to-face, and 11 weeks online. The course develops skills for two of the university’s student learning priorities (information literacy and leadership) by integrating finance, accounting, strategic planning and implementation, marketing segmentation, and operations with teamwork, managing conflict, feedback, influence, alignment, and communication and presentation skills. The course uses several self-assessment, team, and instructor assessment tools. Collectively, they enable students to see the correlation between a high performing team and business (financial and operational) performance. Some of the tools that assess each team’s business performance include a balanced scorecard, tailored success measures, and an analyst report. The TeamMATE evaluation tool (Capsim Teammate, 2018) is built into the simulation and helps information assessors evaluate individual team behavior and team performance and students diagnose their own behaviors and overall team functionality in real time to allow for corrective and developmental action. Students receive this 360-degree feedback on their individual performance, reflect on their ‘blind-spots’, and practice giving and receiving feedback. Additionally, students also complete a EXHIBIT 4 APPLYING THE SIMULATION IMPLEMENTATION FRAMEWORK TO A COURSE USING A TOTAL ENTERPRISE SIMULATION – PART 2 MGMT 601 - Managing the Total Enterprise Total enterprise simulation – Capsim Foundation Simulation (Capsim Foundation, 2018) Course Delivery Format: 4 days, face-to-face 11 weeks, face-to-face 11 weeks, online (virtual) Course / Program Design When taken during program (MBA) Start of program Anytime Anytime When taken during program (other) n/a Elective course Content taught and applied High level breadth due to 4- day residency Breadth; depth in certain areas (planned and student preferences); more time for reflection Course / class session length 4 days (Thu. – Sun.) 11 weeks; one 3-hour class per week 11 weeks; recorded lectures Simulation rounds Days 1 and 2 – 3 rounds per day; Day 3 – 2 rounds 1 round per week - 2 practice rounds, 8 competition rounds 1 round per week - 2 practice rounds, 6-8 competition rounds Students Student profile MBA – face-to-face and online programs Business – MBA, MS, BS/MBA; Non-credit Certificate Non-business – MS, PhD Face-to-face and online programs Simulation Use synchronously / asynchronously Synchronously – face-to-face Synchronously - face-to-face (and virtual) Asynchronously and synchronously (virtual and face-to-face) Total run time Briefing = 30 min.; Team decision-making rounds = 90-120 min.; Round results debrief = 30 min. Briefing = 30 min. (recorded and reusable lecture); Team decision-making rounds = over the course of 1 week; Round results debrief = 30 min. (recorded) Page 291 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 communication style assessment. Finally, student teams conduct a three-part AAR after each round where they assess and reflect on what they intended to do, what actually happened, and what they need to do to improve performance the next round. The Simulation Implementation Framework (Exhibit 1) is applied to this management course in Exhibit 3 and its three delivery formats in Exhibit 4. Due to fast pace of the 4-day immersion residency, students individually complete simulation prework prior to the residency and content can only be covered at a very high level. Given the fast turnaround time between simulation rounds during the residency, it is recommended to provide students with at least one printed copy of the industry and their team reports. The 11-week face-to-face version provides opportunities for flipped classroom variations. The instructor can decide which simple Bloom taxonomy level activities (Cannon & Feinstein, 2005) can be conducted asynchronously outside the classroom such as basic knowledge content, and which complex Bloom taxonomy level activities can be conducted synchronously inside the classroom such as role-playing, applying, and demonstrating mastery (Harris, Harris, Reed, & Zelihic, 2016). Instructional activities included simulation round decisions and debriefs and lectures. Students also have the option of viewing the online version’s recorded lectures before a class for acclimation or after a class for reinforcement. Due to the additional time it takes students to get acclimated to the simulation both individually and as a team, fewer that eight simulation competition rounds can be implemented as well as using less content and assignments for the 11-week online version. To provide an incentive for students to contribute equally relative to their teammates, graded peer-to-peer feedback was also incorporated during the middle of the course in addition to the end. The 11-week face-to-face version provides the instructor the most flexibility given its 11-week term and face-to-face classroom delivery method. The instructor has the option to add more content depth or complexity such as optional simulation modules and an industry recession. During the decision-making rounds in the team breakout rooms for a class session, students also have the flexibility to participate remotely with their team due to unforeseen circumstances preventing them from attending in person. Depending on where they are placed in a curriculum sequence, TES provide an excellent opportunity to teach program content in the current course, highlight upcoming content, or reinforce and apply content previously learned. During an MBA program, students learn about the real-world business challenges of driving performance to improve revenue, growth, profitability, productivity, return on investment, shareholder value, and customer and employee satisfaction. For success in any course with a TES, it is critical that students get up to speed as quickly as possible on the simulation, both individually and with their team. The first assignment provided the structure for this to occur. A pre-course survey served multiple purposes. It gathered each student’s comfort level in the functions of a business such as finance, strategy, marketing, and operations. This information was then used by the instructor to set up diverse teams, including insuring that there is at least one student that is comfortable with financial statements. In addition, the student’s course goals, company/industry business challenge, and executive suite question obtained by the survey were periodically linked to the course content and simulation industry situations. Students learned about business goals, stakeholder value, and the strategic planning process and they typically chose a high volume or high margin strategy. The three major financial statements are taught from both a finance and accounting perspective along with the mix of financial analysis tools such as common size, percent and dollar change, and ratio. Financial statement analysis was linked to both descriptive (what happened) and diagnostic analytics (why and how it happened). Sales and production forecasting were linked to predictive analytics. Content was scaffolded so students first learned the income statement components, its primary goal of profitability, and its drivers of price, volume, cost, and mix. Cost of goods produced and cost of goods sold were linked to the revenue recognition and matching principles. Then, students obtained first-hand experience about asset and cash flow efficiency as their inventory and plant, property, and equipment tied up their cash on the balance sheet. As their companies grew each year, students saw the linkages to the cash flow statement as they identified the sources and uses of cash in operating, investing, and cash flows as well as comprehended the concept of free cash flow. The cash flow statement tells a story of each company over multiple years and students directly understood what happened to their company since they made the decisions. Since they are making long-term investments in plant capacity, students can be exposed to return on investment calculations. This can be further linked to their own company’s capital expenditure approval and strategic planning processes. Once students learned business goals, stakeholder value, strategy formulation and financial statements, this knowledge was then applied to metrics and their drivers such as the DuPont Model and the Balanced Scorecard (Kaplan & Norton, 1996). In addition, they saw the interdisciplinary connections of finance and strategy. A high-volume strategy will typically have lower margins and a high-margin strategy will typically have lower unit market shares. Once students had multiple products in their portfolio, they obtained a holistic view of their company when analyzing how much each product contributed to their company’s total revenues and total profit. Finally, the instructor linked this to a company’s annual report that provided business segment analysis. There were several methods used to develop students’ critical thinking and decision-making skills. Instead of immediately answering the student’s question, the instructor facilitated the Socratic teaching method by asking such questions as ‘what is your strategy’, ‘what are the two things you want to accomplish this round?’, ‘why is that important to you’ and ‘what do you think?’ Students used the simulation software’s proforma planning tool to perform sensitivity and scenario analyses. Here, they saw what the impact on projected financial and operational results were when they changed one or more variables such as price, volume, units produced, and debt issued. By changing days sales outstanding, they saw the dollar per day impact that it has on cash flow. By reducing price by one dollar, they calculated how much additional volume they needed to sell to attain the same amount of profit. Then, they considered if that additional unit market share was realistic in the industry. Since this a multi-player TES with multiple student teams, many team dynamics and processes teaching points were made. These included interpersonal skills, giving and receiving feedback, managing conflict, conducting an AAR, and preparing and delivering final presentation to the board of directors. Each team had the option of selecting a CEO or a facilitator for each round. These learnings were all made possible by creating a safe learning environment where students could make and learn from their mistakes. During a year in which a company incurred a significant loss, they received an emergency loan so that their company Page 292 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 did not go bankrupt. Finally, they communicated openly by both providing and receiving feedback. Remember, it is not just the simulation, it is the instructor’s knowledge and experience that make using a simulation successful (Goosen, 2002). The author has successfully bridged the gap between industry and academia by teaching similar business acumen and leadership concepts in corporate education programs where participants have applied these concepts to their own company such as business goals and challenges, strategy, financial statements, and capital expenditures. This same experiential application can be used in the academic classroom. Using a TES and building other instructional tools around it such as company financial statements and a case study, students can apply concepts to their current or desired company or industry. This makes the theoretical content more practical and relevant to the students. This paper now looks at how a SBS was used in a required course in Drexel LeBow’s MS in Business Analytics program, as described in Caruso (2018). The program focuses on statistics, operations research, mathematical modeling and management information systems. The experiential learning curriculum enables students to solve business problems through data analysis (LeBow Analytics, 2018). One of the required courses in Drexel LeBow’s MS in Business Analytics program is Aligning Information Systems and Business Strategies (MIS 612). The learning objectives for the course are to “(1) gain an understanding of how managers use business analytics to formulate and solve business problems and to support managerial decision making, (2) drive value through the alignment of analytic and strategic thinking, (3) become familiar with the processes to move from data to insights, and (4) learn how to use and apply business analytics software such as SAS text miner to extract insight into unstructured data” (Anandarajan, 2016, p/1). “Business Question to Story” and “Computerized Thematic Analysis” are the two modules in the course (Anandarajan, 2016). An “organization’s analytics climate within which strategic decisions are made” is included in the first module (Anandarajan, 2016, p/1). Organizational speed and agility, fact-based decision making, competitive advantage, and business strategies and information systems alignment are the topics covered (Anandarajan, 2016). A web-based simulation was implemented in the first module’s “Strategic Decision Making - Quantitative Approach” topic. It helped achieve these learning objectives: “(1) integrate EXHIBIT 5 APPLYING THE SIMULATION IMPLEMENTATION FRAMEWORK TO A COURSE USING A SCENARIO-BASED SIMULATION MIS 612: Aligning Information Systems and Business Strategies Scenario-based simulation: Harvard Business Publishing’s Data Analytics Simulation: Strategic Decision Making (Davenport, 2016) Course / Program Design • Required graduate course for MS-Analytics program; first term • Content (course) – business analytics, business problem solving, text analytics • Content (via simulation) – business goals; business acumen (high level finance and strategy); descriptive, diagnostic, and pre- dictive analytics • Simulation used during 1.5 classes of 11-class course Students • Graduate level • Number of students: minimum = 12; maximum = 50; ideal class size = 24-36 Instructor • Course – 1 internal; Simulation (1.5 classes) – 1 external; 1 per course section • Simulation facilitation comfort level: overall – very high; this simulation - moderate • Simulation roles performed – instructor, facilitator, consultant, and coach Simulation • Discounted pricing for universities; students pay • Best simulation for course – this data analytics simulation or a total enterprise, multi-player, off-the-shelf (for this class or MGMT 601) • Number of simulation rounds: minimum = 1; maximum = 4; ideal = 4; option to allow students to conduct multiple scenarios (repeat 4 rounds) Resources • Technical support available for both instructor and participants – phone, email • Student support materials - simulation navigation, participant guide • Instructor support materials - student support materials plus instructor guide, debriefing slides, Harvard case study, teaching note, and instructor videos Page 293 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 AEE, (2011). Association for Experiential Education, 2011, para 4. Retrieved 5/17/14 from http://www.aee.org/ Anandarajan, M. (2016). MIS 612: Aligning Information Systems and Business Strategies course syllabus. Philadelphia, PA: Drexel University LeBow College of Business. Anderson, J. R. (2005). The relationship between student perceptions of team dynamics and simulation game outcomes: An individual-level analysis. Journal of Education for Business, 85-90. Anderson, P. H. & Lawton, L. (2003). The optimal timing for introducing business simulations, Developments in Business Simulation and Experiential Learning, 30, 1- 4. REFERENCES analytics and business acumen, (2) turn around the lagging performance of product in a competitive market and (3) make strategic decisions using simulated data vs. intuition” (Anandarajan, 2016 p/4). The instructor incorporated Harvard Business Publishing’s Data Analytics Simulation: Strategic Decision Making (Davenport, 2016). This versatile single-player simulation can be used to teach business analytics, business intelligence, marketing strategy, marketing research, and business decision-making concepts. The simulation was facilitated over two class sessions. The engaging simulation provided a more realistic context for the entire course content to follow. A flipped classroom approach (Harris et. al., 2016) was incorporated here as students used the time outside of class to prepare for and reflect on the simulation. Before the first session, students became acclimated to the industry, their company, and four years of financial and market data (descriptive analytics). This enabled the students to begin to see the relationships between variables and to inquire about why and how things happened (diagnostic analytics). Based on this data, students chose their strategy and target segment(s), and planned their decisions. The instructor covered content at a high level such as financial and business management goals, business strategy, and the income statement and its drivers. This business acumen content provided the right context for the upcoming simulation experience. These business acumen concepts are the same ones in the MBA course mentioned earlier in this paper. However, for this course and program, the objective is to apply these key business concepts to business analytics. The students made decisions for four simulation years and individually managed their product (Blue Detergent). They made pricing, advertising, production, R&D, and forecasting decisions and applied a Monte Carlo forecasting tool (predictive analytics) to get more guidance with forecasting. The instructor delivered key points to the entire class and responded to individual student questions. Students were able to make adjustments each year since they made decisions and analyzed data for four simulated years. Actual results data from each new year was added to the cumulative descriptive data, enabling students to perform a gap analysis and AAR after each simulated year. This repeated process enabled them to reflect on planning/forecasting and budgeting by analyzing the actual results. Students reflected on their simulation experience using several questions provided by the instructor in-between the two simulation classes. When the students returned for the second simulation class, the instructor conducted a debrief of the simulation results and experience and led a discussion of the major lessons learned. The Simulation Implementation Framework (Exhibit 1) is applied to this business analytics course and accompanying simulation (Davenport, 2016) in Exhibit 5. The simulation experience comprises 1.5 classes of 11 over an 11-week course. The For the first time the simulation was used in this course, students participated in the simulation experience during the fifth and sixth classes. Subsequently, students engaged with the simulation during the first and second classes. Since this simulation has only been used a few times, the simulation facilitator’s comfort level with this specific simulation is moderate. The author feels that this simulation, combining business acumen and analytics, works best for this course within the context of the MS in Business Analytics program given its current required and elective courses. If the program wants to provide more business acumen content, a total enterprise simulation could be incorporated into this course or a separate course (Managing the Total Enterprise) could become a required course. Business simulations and case studies share common benefits (Mitchell, 2004; Gomide & dos Santos, 2013). They are interdisciplinary and involve thorough analysis, critical thinking, decision making, and experiential learning. These active learning methods are embedded in business schools and involve groups cooperating (Gomide et. al., 2013). Based upon the first pilot run of the simulation and industry input, the author recommends using the simulation’s accompanying case study (Harvard Business School – Managing with Analytics at Procter & Gamble). The author believes that the Harvard Business Publishing’s Data Analytics Simulation: Strategic Decision Making provides an excellent opportunity to highlight both the technical and managerial aspects of business analytics, enabling students to make better business decisions quicker. The author recommends incorporating industry guest speakers who can talk about current consumer analytic trends such as the impact that weather has on consumer purchasing decisions. Finally, the author suggests further incorporating analytics competencies into MS-Business Analytics program. This can include the skills needed today and in the future (workforce planning) for such career tracks as manager (leader), analyst (technical), and other functional career paths in analytics. CONCLUSION Business simulations help students obtain workforce skills by enabling them to practice thinking critically, making better decisions, and solving business problems. This paper has described what business simulations are, their types, benefits and limitations, and how they are used. It has also discussed how to select and use a simulation in a course integrating interdisciplinary content and using a Simulation Implementation Framework. The author strongly believes that business simulations are a powerful instructional tool and their learning impact is maximized when considering the framework’s five primary design considerations. http://www.aee.org/ Page 294 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 Baldissin, N., Greco, M., Nonino, F., & Wolfe, J. (2018). The Serious Games Typology Project. Retrieved 10/21/18 from http://seriousgames.online. Beaudry, S. L. (2016). Summiting Mount Everest: An experiential learning application for organizational behavior, Developments in Business Simulation and Experiential Learning, 43, 23-26. Bennis, W. & O’Toole, J. (2005). How business schools lost their way. Harvard Business Review, 83(5), 96-104. Bos, N. & Gordon, M. (2005). A simulated consulting project with a deregulating utility company, Simulation & Gaming, 36(1), 91-113. Byers, C. & Cannon, H. M. (2007). The programming game: An exploratory collaboration between business simulation and instructional design, Developments in Business Simulation and Experiential Learning, 34, 259-265. Cannon, H. M., Cannon, J. N., Koksal, A., & Johnson, A. (2014). Customer transaction costs and marketing simulations: Modeling a new relationship marketing approach, Developments in Business Simulation and Experiential Learning, 41, 36-46. Cannon, H. M. & Feinstein, A. H. (2005). “Bloom beyond Bloom: Using the revised taxonomy to develop experiential learning strategies,” Developments in Business Simulation and Experiential Learning, 32, 348-356. Capsim Foundation (2018). Capsim foundation simulation (software). Retrieved 10/21/18 from http:// www.capsim.com/simulations/foundation/. Capsim Teammate (2018). Capsim teammate assessment (software). Retrieved 10/21/18 from http:// www.capsim.com/assessments/teammate/. Caruso, J. V. (2017). MGMT 601: Managing the Total Enterprise course syllabus. Philadelphia, PA: Drexel University LeBow College of Business. Caruso, J. V. (2018). Integrating Business Acumen and Analytics: A Simulation-Based Approach, Developments in Business Simulation and Experiential Learning, 45, 231-233. Davenport, T. H. (2016). Data analytics: Strategic decision making simulation (software). Retrieved 10/21/18 from https://cb.hbsp.harvard.edu/cbmp/product/7050- HTM-ENG. Dickinson, J. R. (2003). The feasibility of the balanced scorecard for business games, Developments in Business Simulation and Experiential Learning, 30, 90 -98. Gibbons, A. S., Fairweather, P. G., Anderson, T. A., & Merrill, (1997). Simulation and instruction: A future view. In C. R. Dills and A. J. Romiszowski (Eds.) Instructional Development: State of Art. (772-783). Englewood Cliffs, NJ: Instructional Technology Gold, S. (2017). Beat the market simulation: A self-study business economics game, Developments in Business Simulation and Experiential Learning, 44, 284-285. Gold, S. (2018). Beat the market online: A microeconomics game (software). Retrieved 10/21/18 at http:// www.goldsimulations.com/economics-simulation- game/economic-concepts-self-learned. Gold, S. (2003). The design of a business simulation using a system-dynamics based approach, Developments in Business Simulation and Experiential Learning, 30. Gold, S. (2015). With simulations, it is not the wand but the magic in the magician: Pilot study enhancing and assessing topic-specific student learning using an economic simulation, Developments in Business Simulation and Experiential Learning, 42, 69-75. Gomide, T. R. & dos Santos, J. A. (2013). Merging the case method and simulation in management education: Is it possible? Developments in Business Simulation and Experiential Learning, 40, 61-67. Goosen, K. R. (2002). A critical examination of the “experimental” premise underlying business simulation usage, Developments in Business Simulation and Experiential Learning, 29, 308-315. Goosen, K. R., Jensen, R., & Wells, R. (1999). Purpose and learning benefits of business simulations: A design and development perspective, Developments in Business Simulation and Experiential Learning, 26, 133-145. Granland R., Bergland, E., & Eriksson, H. (2000). Designing web-based simulation for learning. Future Green, J. C. & McQuaid, B. (2003). The use of a simulation in an integrated MBA curriculum, Developments in Business Simulation and Experiential Learning, 30, 113-119. Hall, J. J. S. B. (2007). Computer business simulation design: Novelty and complexity issues, Developments in Business Simulation and Experiential Learning, 37, 393-403. Harris, B. F., Harris, J., Reed, L., & Zelihic, M. (2016). Flipped classroom: Another tool for your pedagogy tool box, Developments in Business Simulation and Experiential Learning, 43, 325-333. Heinich, R., Molenda, M., Russell, J., Smaldino, S. (1999). Instructional Media and Learning. (6th ed.). (14-15, 290-291, 319-324). Upper Saddle River, NJ: Kaplan, R. S. & Norton, D. P. (1996). Using the balanced scorecard as a strategic management system, Harvard Business Review, Vol. 74 (January-February), 75-85. LeBow Analytics, (2018). MS in Business Analytics. Retrieved 10/21/18 from http://www.lebow.drexel.edu/ academics/graduate/master-of-science/ms-business- analytics Lunce, L. M. (2004). Computer simulations in distance education, International Journal of Instructional Technology and Distance Learning, 1(10), 29-40. Magnuson, R. A. & Good, D. C. (2017). It’s more than just a simulation: Deepening and broadening student learning by using a business enterprise simulation as a platform, Developments in Business Simulation and Experiential Learning, 44, 95-105. Min, R. (2001). Designing dynamical learning environments for simulation: Micro-worlds applets on the world wide web. 6th Proceedings of EARLI, SIG, June 27-29, 2002, in Erfurt, Germany. Mitchell, R. C. (2004). Combining cases and computer simulations in strategic management courses. Journal of Education for Business, 198-204. NIU, (2014). Faculty development and instructional design center. Retrieved 3/26/14 from http://www.niu.edu/ facdev/_pdf/guide/strategies/experiential_learning.pdf. Roberto, M. & Edmondson, A. (2011). Leadership and team simulation: Everest v2 (software). Retrieved 8/16/17 from https://cb.hbsp.harvard.edu/cbmp/product/7000- HTM-ENG. http://seriousgames.online http://www.capsim.com/simulations/foundation/ http://www.capsim.com/simulations/foundation/ http://www.capsim.com/assessments/teammate/ http://www.capsim.com/assessments/teammate/ https://cb.hbsp.harvard.edu/cbmp/product/7050-HTM-ENG https://cb.hbsp.harvard.edu/cbmp/product/7050-HTM-ENG http://www.goldsimulations.com/economics-simulation-game/economic-concepts-self-learned http://www.goldsimulations.com/economics-simulation-game/economic-concepts-self-learned http://www.goldsimulations.com/economics-simulation-game/economic-concepts-self-learned http://www.lebow.drexel.edu/academics/graduate/master-of-science/ms-business-analytics http://www.lebow.drexel.edu/academics/graduate/master-of-science/ms-business-analytics http://www.lebow.drexel.edu/academics/graduate/master-of-science/ms-business-analytics http://www.niu.edu/facdev/_pdf/guide/strategies/experiential_learning.pdf http://www.niu.edu/facdev/_pdf/guide/strategies/experiential_learning.pdf https://cb.hbsp.harvard.edu/cbmp/product/7000-HTM-ENG https://cb.hbsp.harvard.edu/cbmp/product/7000-HTM-ENG Page 295 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 Schiflett, S. G., Elliott, L. R., Salas, E., & Coovert, M. D. (Eds.) (2004). Scaled Worlds: Development, Validation, and Application. Surrey, England: Ashgate Publishing Limited. Scoresby, J. & Shelton, B. (2014). Reflective redo from the point of error: Implications for after action review, Simulation & Gaming, 1-31, DOI: 10.1177/1046878114549426.10/17 Sternberg, R. J. Cognitive Psychology. (1999). (2nd ed.). (pp. 17, 186, 446-449). Fort Worth, Texas: Harcourt Brace College Publications. Teach, R. & Patel, V. (2007). Assessing participant learning in a business simulation, Developments in Business Simulation and Experiential Learning, 34, 61-67. Willis, J., Hovey, L., Hovey, K.G. (1987). Computer Simulations. A Sourcebook to Learning in an Electronic Environment. New York: Garland. 1.