A Competitive Business Ethics Simulation Game Page 155 - Developments in Business Simulation and Experiential Learning, volume 42, 2015 ABSTRACT This article describes a competitive business ethics simulation game I developed for business ethics courses. The simulation game, inspired by the 2010 BP Gulf oil spill, has been played by over 800 undergraduate and graduate students at ten universi- ties. In this article I describe the simulation, how it works, the model behind it, and explain how the simulation functions as an experiential exercise supporting the teaching of business ethics. Based on my experience with the simulation, I make some rec- ommendations about guiding student discussion of simulation results and on grading ethically-oriented simulations. Finally, I present some quantitative data on the effectiveness of the simu- lation. The contribution of this article is to support and encour- age business ethics instructors to add an important experiential teaching method to the existing repertoire of readings, lectures, group projects and case studies. INTRODUCTION Computer-based simulation games are widely used to teach marketing, strategy, operations and management (Faria, Hutchinson, Wellington & Gold, 2009; Mayer, Dale, Frac- castoro & Moss, 2011, p. 65). These simulations have been shown to be effective teaching tools (Lu, Hallinger & Showana- sai, 2014; Wellington, Faria, Hutchinson & Gowing, 2014; Ti- wari, Nafees & Krishnan, 2014; Faria, 2001; Wolfe, 1997; for a contrary view see Gosen & Washbush, 2004, p. 286). Two of the assumptions that underlie the use of these simulation games are 1) that practice improves one's ability to perform and 2) that simulations provide students with opportunities to practice mak- ing management decisions in a safe environment (Sims, 2002, pp. 179-180; Scherpereel, 2005, p. 389; Hofstede, de Caluwé & Peters, 2010; Mayer, Dale, Fraccastoro & Moss, 2011, p.66). This suggests that a business simulation explicitly designed to confront students with ethical challenges in business might provide useful learning opportunities (LeClair, Ferrell, Montuo- ri & Willems, 1999, pp. 284-286; Fritzsche & Rosenberg, 1989, p. 47). Ethically-oriented simulations might give students prac- tice recognizing the ethical aspects of a management situation, identifying relevant stakeholders, balancing competing interests and making responsible decisions. Moreover, the immediacy of the simulation experience might promote in students an aware- ness of the pressures on managers making morally complicated decisions (Schumann, Anderson & Scott, 1997; Scott, Schu- mann & Anderson, 1998; Wolfe & Fritzsche, 1998; LeClair & Ferrell, 2000; Schumann, Scott & Anderson, 2006.) This article describes a competitive business ethics simula- tion game that I developed for use in my upper level undergrad- uate business ethics course. I developed the simulation to pro- vide an intensive, semester-long, experiential learning exercise for students. I have used it since the Spring 2011 semester. It first became available to instructors at other institutions in Spring 2013. To-date, it has been used by over 800 students at ten universities. In what follows I discuss a) how the simulation works, b) the computer-based business model behind the simulation, c) the ethical aspects of the simulation, d) grading student simula- tion performance, and e) data on the simulation's effectiveness. HOW THE SIMULATION WORKS The simulation game is called "Deepwater." It was inspired by the disaster in the Gulf of Mexico in April 2010, when the Deepwater Horizon offshore oil platform exploded, killing 11 workers. In the three months it took to seal the well, over 200 million gallons of crude oil flowed into the Gulf, creating the worst environmental disaster in U.S. history and damaging the Gulf economy at a cost of billions of dollars (National Commis- sion on the BP Deepwater Horizon Oil Spill and Offshore Drill- ing, 2011). Deepwater is played completely on the web and requires no software downloads or plug-ins. It can be played using a desk- top computer, laptop, tablet or smart phone. It has been incorpo- rated into online courses by several instructors, but to-date has been used primarily in face-to-face classroom settings. In Deepwater, students manage a simulated oil exploration and production company. The company operates a production oil platform (or rig) located in deep water far offshore in the Gulf of Mexico. Students can play individually, in teams or in a combination of the two. Game revenue is generated by extract- ing crude oil from under the Gulf and selling it on the interna- tional oil market. Expenses include operating costs, labor, worker training, maintenance and safety, and fines (if any) for accidents or spills. Students compete against each other, not a computer, to maximize profits responsibly. The simulation game is played in a series of rounds usually beginning with one or more practice rounds (for the importance of practice in experiential exercises, see Snow, Gehlen & Green, 2002, p. 526). The instructor decides how many rounds in the game and how often rounds occur. Typically instructors play the game in one or two rounds a week with two practice rounds and a total of eight to twelve non-practice or "regular" rounds. Simulation rounds do not represent any specific period of time such as a month, quarter or year. Rather, the simulation clock tracks only the number of rounds in the game and the cur- rent round number. Each round has a deadline, by which time students need to enter their management decisions. The round closes once the deadline passes. The simulation engine process- es students' decisions, calculates results for the round, and gen- A COMPETITIVE BUSINESS ETHICS SIMULATION GAME Wayne F. Buck Eastern Connecticut State University buckw@easternct.edu Page 156 - Developments in Business Simulation and Experiential Learning, volume 42, 2015 erates reports that are available to students online. The next round then opens and students can enter decisions for that round up until its deadline. For each round, students make a variety of operational and strategic management decisions. These decisions affect the re- sults of that particular round, and need to be made again in the next round, even if the student wants to make the same decision round after round. With a few exceptions, there are no "set and forget" decisions. For each round, students first need to decide how much crude oil to produce. They can set a target production volume of any amount of oil up to the rig's maximum physical capacity. For a point of reference, students are given a "baseline" produc- tion volume. This is presented to students as the production the rig should be able to produce round after round with no prob- lems – provided of course that other decisions such as spending on maintenance and repair are also "baseline" decisions. The crude oil is sold on the open oil market. Instructors have the option in each round of using the actual market price or setting their own oil prices. An offshore oil rig is a dangerous place to work. Accidents with heavy and powerful equipment can injure or kill. Given that hydrocarbons are being extracted from highly pressurized underground reservoirs, oil rigs are also subject to fires and explosions. The most important piece of equipment preventing explosions is the blowout preventer or BOP. This sits on the ocean floor. When activated in an emergency, it functions as a gigantic valve, shutting off the flow of flammable oil and gas out of the well. A rig's BOP has a recommended service life measured in rounds. Once the BOP has been in service longer than the rec- ommended number of rounds, it should be overhauled. An over- haul costs millions of dollars and requires the oil rig to be "shut in" or stop producing for one or more rounds. Students have the option every round to shut in and overhaul their BOP. They can overhaul early, or they can push their BOP beyond its recom- mended service life and hope that it holds up. Maintenance and repair are important factors in making sure dangerous equipment functions as expected. The more an oil rig's pumps, valves, motors, engines and electrical systems are worked, the greater the chances of a breakdown. Students who produce more than the baseline amount of crude oil round after round can expect their equipment to wear out sooner. To compensate, students decide each round how much to spend on maintenance for that round. Maintenance spending above the baseline value lowers the chances of a mechanical breakdown, spending below the baseline increases the chances of a break- down. Lowering production volumes also reduces the chances of a blowout, while raising production increases the likelihood of a blowout. Together production and maintenance decisions determine the risk of a catastrophic accident. Students can reduce the chances of a worker injury or fatal- ity by spending for safety programs and for additional worker training. Spending on safety affects the chances of a worker accident for the current round only, so the rig's safety program is an ongoing expense. Advanced training requires that workers be sent off the rig to the mainland. They are gone for two rounds, but when they return they work more safely than work- ers who have not benefited from the additional training. Ad- vanced training costs money, and short-term production is im- pacted, but the more workers a student sends for training, the lower the chances of an accident. Students can also fire workers if they believe they have too many or to reduce expenses. There is an immediate, short-term cost to fire a worker, but that cost is quickly made up by the reduced payroll. Weather is also a factor students must consider in making their decisions. Instructors have the option to tie the simulation to the actual weather in the Gulf of Mexico, or to set their own weather. In either case, hurricanes are a serious threat to contin- ued operation of an offshore oil rig. Although the rigs are built to withstand heavy weather, trying to operate in the midst of a hurricane is a risky proposition. Students can shut down their rig in the event of a hurricane to be safe, or attempt to continue operations and risk a blowout or other accident. Students can also invest in pollution control equipment for their rig. The cost of the equipment is capitalized, so the finan- cial impact is limited to reducing the amount of cash on hand and additional depreciation expense each round. There is no requirement to install this equipment, but doing so lowers the amount of pollution emitted by the rig and lowers the compa- ny's impact on the environment and society as a whole. The energy sector of the economy creates very large nega- tive externalities, none more so than extraction industries such as oil production. Crude oil is refined into fuels such as gaso- line, diesel and jet fuel. When these fuels are burnt pollutants and large amounts of CO2 are released into the environment. The extractive operations themselves consume large amounts of fossil fuel energy and inevitably result in spills and other forms of water and air pollution. Local businesses such as fisheries, resorts and restaurants are affected. These negative externalities are estimated and monetized by the Deepwater simulation model and reported to students as the "social costs" of their operations. These do not represent direct costs to the student's simulation company, but they do provide students with information about the negative impact of their business on society. THE SIMULATION MODEL Unlike other business simulations, Deepwater does not model product demand, company sales or changes in market share. In fact, Deepwater assumes that players are able to sell all the oil they produce at the going market rate. Instead, Deep- water is designed as an operations-oriented simulation game. The simulation engine calculates revenues and expenses based on players' operating decisions, and outputs operating, financial and market reports for each player. These reports indi- cate whether the player's rig has experienced a blowout and the number (if any) of worker injuries, fatalities and safety viola- tions. Other operating metrics include actual production, equip- ment condition and the number of hours worked by the rig crew. The financial report includes an income statement and balance sheet. The market report ranks competitors by profita- bility. Also unlike other business simulations, the simulation model is not a system of deterministic functions. Instead, the mathematical functions connecting inputs and outputs are prob- abilistic. A player's operating decisions do not completely deter- mine whether she is cited for a safety violation, experiences an Page 157 - Developments in Business Simulation and Experiential Learning, volume 42, 2015 accident or suffers a blowout. Rather decisions about how much oil to produce and how much to spend on maintenance and safe- ty shift the probabilities of something going wrong. Producing more oil or spending less on maintenance and safety increases the probability of a blowout or an accident, while producing less oil or spending more decreases those probabilities. Deepwater's functions are designed for either diminishing returns or increasing risks at the margin. Moreover, since most of the functions are asymptotic, the probabilities output by these functions never decrease to zero or increase to 100%. Probabilistic functions are a distinctive aspect of the Deep- water engine. As in real life, it is possible for a player to pro- duce the right amount of crude oil, spend the right amount on maintenance and safety, and still have an accident or blowout – unlikely, but possible. On the other hand, it is possible for a player to work her equipment and crew until they are ragged, skimp on maintenance and safety, and have no accidents and no blowout – again, unlikely, but possible. ETHICAL ASPECTS OF THE SIMULATION The simulation does not confront students with explicit ethical dilemmas, and hence the ethical aspects of Deepwater may not be immediately obvious. Students are not, for example, forced to choose between an expensive repair or bribing an in- spector, between losing market share or lying about a product, between a falling stock price or fraudulent accounting. Instead of these kinds of dilemmas (where the challenge is not to determine what is right but to actually to do the right thing) students playing Deepwater are faced with what I call ethical "conundrums." In Deepwater students confront the daily challenge of making responsible tradeoffs between profits, on the one hand, and other values such as worker safety, social impacts and the environment, on the other. These tradeoffs are part-and-parcel of every manager's day-to-day decision making. Consider, for example, the question of how much a chemi- cal plant manager should spend on maintenance, repair and safety training. In general (although there are diminishing re- turns), the more spent, the safer the plant will be. Yet a serious accident could still happen no matter how much is spent. How much spending is enough? How safe is safe enough? These challenges are not ethical dilemmas with obvious right and wrong answers, but conundrums. Like all conun- drums, they present managers with "confusing or difficult prob- lems" ("Conundrum," Merriam-Webster, n.d.). Moreover, be- cause the eventual outcome of these decisions is not knowable in advance, good judgment plays an important role in respond- ing to these conundrums. In the midst of having to make a deci- sion, reasonable, well-informed, well-intentioned people can come to very different "good" judgments. Not only the right solution to the problem is contested, but even how to think about the problem is contestable. (This makes ethical conun- drums much like so-called "wicked" problems [see Rittel and Webber 1973].). Deepwater presents students with a very specific kind of ethical challenge: how, when managing morally perilous busi- ness activities, to strike a responsible balance between benefits to themselves, on the one hand, and harms to others, on the oth- er. Fundamentally, the question each Deepwater player must answer is: how much risk is it acceptable to expose others to in pursuit of my own interests? The decisions required for every Deepwater round (viz., how much to produce, how much to spend on maintenance and safety, how many workers to hire, train or fire, and when to overhaul the BOP) confront students with different variations of the challenge of balancing benefits to oneself against harms to others. To expand the variety of challenges and to keep students engaged in longer games, instructors can select from a number of special, round-specific ethical challenges. In what follows I discuss one such challenge – the BOP Testing challenge –in order to illustrate the nature of these special ethical challenges. Every business decision must be made in the face of uncer- tainty. Most business students are familiar with one kind of un- certainty – uncertainty about outcomes. Will the new product capture significant market share? Will the new hire actually EXHIBIT 1 ASSESSMENT SURVEYS USED WITH DEEPWATER Survey Number When Administered Number of Questions (Individual / Team Implementations) Topics 1 Pre-Practice Rounds 11 / 17 Prior simulation experiences, concerns about individual performance, attitudes about relevance and expected learning benefits. 2 Post-Practice Round 9 / 11 Readiness to play, familiarity with game rules, self- confidence, engagement and level of concern/ anxiety. 3 Simulation Midpoint 9 / 11 Attitudes about relevance, learning benefits, engagement, self-confidence, satisfaction, and expectations. 4 Final Rounds 5 / 7 Satisfaction, engagement, and stressfulness. 5 Simulation Wrap-up 8 / 17 Satisfaction and engagement, recommendations for improvements. 6 Learning Outcomes 12 / 13 Perceived learning outcomes. Page 158 - Developments in Business Simulation and Experiential Learning, volume 42, 2015 deliver the value promised on the resume? Will we get the new plant up and producing on time? But there is also always considerable uncertainty about what is happening today: Are customers really as satisfied with our service as surveys suggest? Do we actually have two of a given item in stock as the inventory system reports? Is market- ing actually collaborating as effectively as it could with sales? The challenge in dealing with uncertainty about the present arises because a manager is likely to have considerable infor- mation, but much of it is likely to be incomplete, ambiguous or inconsistent. A pressure gauge registers an acceptable value, but some employees suspect that the gauge is malfunctioning. A team leader reports that the team is working well together, but perhaps she is keeping quiet about some serious team conflicts in the hope of resolving them herself. A salesman claims that a prospect will sign "within the week" but actually knows that contract negotiations are likely to drag on much longer. What should a manager do with "information" such as this? In the absence of clear, definitive information, competing perspectives on the significance and characteristics of a problem naturally arise. These competing perspectives often reflect com- peting interests among stakeholders. Decision makers must not only deal with the uncertainty, but also navigate through a thicket of conflicting interpretations. The result is that decisions made under uncertainty tend not to be made solely on the tech- nical merits of one option versus another, but in many case on the basis of other factors – simply because of the lack of good information. The BOP Testing ethical challenge provides students with the opportunity to experience the difficulty of making decisions in the face of uncertainty and creates an opportunity to practice balancing conflicting values such as revenue generation and protecting the environment. In Deepwater, players have no direct information about a very critical piece of equipment – their blowout preventer (BOP). The BOP is five thousand feet underwater and hidden from day-to-day observation. Players know that the manufactur- er estimates the average service life at a certain number of rounds. They know that the chances of a blowout increase if their BOP is left in service after the expected service life, but they have no idea how rapidly those chances increase. Over- hauling a BOP is expensive, not only because of the direct cost but because of the high opportunity costs of being shut-in for a round or two. In the BOP Testing ethical challenge, students receive two documents: a maintenance bulletin from the company's quality control engineers and input from the company's budget office. The engineering bulletin reports the discovery that a critical component of the BOP may be defective. This uncertainty is worrisome because the BOP is the last line of defense against a blowout. It is arguably the most important safety device on the entire rig. To complicate matters further, the engineering bulletin hints at past organizational conflicts between engineering and management (not an uncommon occurrence). The outcome of this conflict is that it is squarely a management call on how to respond to the uncertainty about the BOP. Engineering, having been reprimanded in the past, is careful not to make a recom- mendation themselves. The budget office, concerned about cost control issues, has jumped in with its own perspective. A memo from them attempts to down play the importance of the mainte- nance bulletin. The BOP Testing ethical challenge requires students to make a decision based on the information contained in the maintenance memo and budget office email. A number of op- tions are available to students. They can test the suspect compo- nent. The test will increase their expenses and also significantly reduce their production (and hence revenues) for the round. Students might decide that the risk is negligible and continue operating as normal, counting on a future BOP overhaul to completely resolve the issue. Alternatively, students might de- cide to shut-in for the round and do a BOP overhaul, which guarantees replacement of the suspect component. As has been often noted, the value of experiential exercises is fully realized only when students have an opportunity to re- flect on and discuss the exercise. The BOP Testing ethical chal- lenge serves as a good example of this principle. Typically, about two-thirds of students decide to forego the test and operate as normal, and one-third test the BOP. After distributing the individual reports to students, I begin the dis- cussion of this ethical challenge by providing summary descrip- tive statistics on the percentage of students who decided to test their BOP. I ask students if, based on their own experience, the ratio of those who tested to those who did not is roughly what happens in the real world in similar situations. The goal is to encourage a discussion between those who view business man- agers as, in general, more cautious and those who believe man- agers, in general, are more willing to accept risk. It is also very helpful to bring the discussion around to what students them- selves, as consumers, taxpayers, shareholders and small busi- EXHIBIT 2 SURVEY RESULTS: "A SIMULATION GAME IS/WAS A GOOD WAY TO LEARN" Pre-Simulation Post-Simulation Strongly Agree 24% 24% Agree 60% 58% Neither Agree or Disagree 14% 12% Disagree 2% 4% Strongly Disagree 0% 2% Totals 100% 100% n 133 186 Page 159 - Developments in Business Simulation and Experiential Learning, volume 42, 2015 ness owners, would prefer from the managers of companies whose behavior affects them. For example, would the students feel comfortable having managers of airlines, food processing companies, medical device manufacturers, and drug companies play it safe or take on more risk? I have found it useful to initiate a class discussion between those who decided to take the risk and those who conducted the test. The purpose of this discussion is not for each side to make its case, much less try to convince the other. The students' rea- soning will very likely display striking parallels to the reasoning of real life managers in choosing to accept more risk rather than less (much of students' reasoning is likely to mirror that of the email from the budget area). I use this as an opportunity to en- gage students in a discussion about how easily such reasoning can turn into rationalizing a decision that has short term bene- fits. It is important with all simulation ethical challenges to ask students if they would make the same decision in real life as they did in the simulation. Typically, a number of students who did not test will say that if they were really in this situation, they would definitely test. When asked why, they likely will say something such as: "Because lives are at stake and in the simu- lation nobody will be hurt by not being safe enough." Such re- marks provide great opportunities to engage students in reflec- tion on why they are so sure they would make a different deci- sion and confront them with evidence suggesting that in the real world managers are quite likely to take risks that, in retrospect, seem excessive. I give students examples of instances where managers, even though lives were at stake, still did not take the "better safe than sorry" option and walk students through the reasoning behind those decisions. GRADING THE SIMULATION As with any experiential exercise, instructors need to de- cide how to incorporate an ethics-oriented simulation into their course requirements. A particularly thorny question is how to grade an exercise intended to support the teaching of business ethics. Instructors are faced with a difficult choice: grade exer- cise performance purely on the basis of business results (e.g., net income) or factor in the morality of students' decisions (e.g., total social costs). In my experience, it would be a mistake to consider any- thing other than business results in grading a student's simula- tion performance. To do so – in effect to reward students for making the ethically right decisions – teaches the wrong lesson. An important learning objective is for students to experience conflict between personal gain and observing ethical norms – and to reflect on their own response to the conflict. A simula- tion which eliminates the conflict cannot achieve that learning objective. In my own implementations, the only measures that deter- mine a student's grade on the simulation are financial metrics. Students know about the harms they’ve caused to others and to the environment, but these do not affect their grade (except in- sofar as they trigger fines, loss of productive capacity, or direct costs to the business). Only the financial results count. Some instructors have taken a different approach, reasoning that grades based solely on business performance downplay the importance of pursuing profits ethically, thereby sending the wrong message to students. These instructors adopt a mixed grading approach, using a combination of financial results and social impacts to determine student grades. In these situations, students of course need to know what counts from a grading perspective as the "ethical" answer to ethical challenges. These students tend, as a result, to make ethical decisions in order to maximize their grade – where "ethical" here means what the instructor thinks is the "right" thing to do. This may be a good way to impart information about what is right and wrong in business, but I would argue this approach diminishes the experi- ential component of the exercise. EFFECTIVENESS I have developed a set of six student surveys to assess the effectiveness of Deepwater. These surveys are administered before the simulation begins, during the simulation and after it concludes. (On the importance of assessment see, among many others, Feinstein & Cannon, 2002; and Gosen & Washbush, 2004.) Exhibit 1 summaries these surveys. Survey data were collected by myself and one other in- structor on 337 students who have played the Deepwater simu- lation game in the last two years. Most, but not all, surveys were anonymous. Many made use of anonymous responder identifiers to support longitudinal analysis. For the sake of con- sistency, the following brief analysis of Deepwater's effective- EXHIBIT 3 SURVEY RESULTS: "HOW ENGAGED ARE YOU WITH THE SIMULATION?" Post-Practice Final Rounds* Very Engaged 20% 30% Moderately Engaged 61% 50% Minimally Engaged 14% 14% Rather Disengaged 4% 4% Very Disengaged 2% 2% Totals 100% 100% n 111 197 * 12 round games with one round per week Page 160 - Developments in Business Simulation and Experiential Learning, volume 42, 2015 ness will be limited to data I collected anonymously from stu- dents enrolled in my 300-level business ethics course. The sam- ple includes 236 students from 13 different classes spread over 5 different semesters (47 students in three classes from Fall 2012, 63 students in three classes from Spring 2013, 42 students in two classes from Fall 2013, 51 students in three classes from Spring 2014 and 33 students in two classes from Fall 2014). The survey data contains 11,387 data points. For the purposes of this analysis, the 236 students are treat- ed as members of a single group. The analysis that follows thus in effect uses a pre-experimental, one-group, pretest-posttest design. I believe this simplifying approach is justified, at least for this relatively high-level analysis, because: a) each class drew from the same student population (upper level business major students who had completed all core business courses); b) all simulation games involved two practice rounds and ran for 12 rounds, 1 round per week, extending over nearly an entire 14 week semester, c) each game was configured the same and in- cluded the same set of ethical challenges; d) essentially the same course syllabus was used in all 11 classes; and e) all clas- ses were taught by the same instructor. STUDENT EXPECTATIONS ABOUT LEARNING OUT- COMES Students approach any learning opportunity, including an exercise, with certain expectations about the value of that op- portunity. They may anticipate that they will actually learn from the opportunity, they may be uncertain or indifferent, or they may be convinced that they will learn nothing. These attitudes can impact how much the student learns (Gosen & Washbush, 1997; Feinstein & Cannon, 2002; Snow, Gehlen & Green, 2002). Overall, students' expectations about the value of Deep- water as a learning experience were initially high and their actu- al simulation experience did not disappoint them in that respect. Students participating in Deepwater are presented with the following two questions, the first before the simulation begins and the second after it concludes: 1. "Playing a simulation game is a good way for me to learn" 2. "Playing this simulation has been a good way for me to learn" Results show that students, both before their simulation began and after it concluded, believed the simulation was a good way to learn. Prior to the simulation 84% believed in the learning value of simulations. That percentage was virtually unchanged at the end of the simulation, when 82% expressed a belief in the usefulness of Deepwater as a learning tool. It ap- pears that the initial, relatively high expectations of the learning value of the simulation were for most students unchanged or only modestly reduced over the course of the experience (see Exhibit 2). (A Kolmogorov-Smirnov test yields a D of 0.049, less than the critical value of 0.100, indicating the differences in distribu- tions pre- and post-simulation are not statistically significant at the 0.05 level. ) STUDENT ENGAGEMENT Active involvement in the simulation is an important condi- tion for a positive and productive learning experience (Sims, 2002, p. 195). Several times during the simulation, students were asked "At this point, how engaged would you say you are with playing the simulation?" Their responses, immediately after completing the practice rounds and at the close of the sim- ulation are presented below (see Exhibit 3). Before their simulation began 81% of students were very or moderately engaged. That proportion was essentially unchanged (80%) in the final rounds. This suggests that the simulation was successful in holding students' interest over the entire semester. The percentage of disengaged students begin low and remained low: 6% in both cases. Interestingly, the proportion of very en- gaged students increased over the course of the simulation, while the proportion of moderately engaged students fell. There were, on the other hand, only very small changes in the propor- tion of responses in the other three categories (minimally en- gaged, rather disengaged and very disengaged). This suggests that engagement with the simulation increased over time. (Indeed, a Kolmogorov-Smirnov test yields a D of 0.106, great- er than the critical value of 0.097, indicating the differences in distributions pre- and post-simulation are statistically significant at the 0.05 level.) PERCEIVED RELEVANCE Ideally, students would recognize even before the simula- tion begins that it is relevant to business ethics and not simply a game or even just a business management or strategy simula- tion. In any case, unless students recognize the simulation's rel- evance to business ethics at some point in the game, the experi- ence is unlikely to benefit them (Feinstein & Cannon, 2002, p. 434). Students participating in Deepwater were presented with the following two questions, the first before the simulation be- EXHIBIT 4 SURVEY RESULTS: PERCEIVED RELEVANCE OF SIMULATION TO BUSINESS ETHICS Pre-Simulation Mid-Simulation Believe simulation relevant 75% 83% Uncertain or skeptical 25% 17% Totals 100% 100% n 134 148 Page 161 - Developments in Business Simulation and Experiential Learning, volume 42, 2015 gan and the second at the halfway point: 1. "Given what you know about the simulation so far, what is your view today about the relevance of a simulation to business ethics?" Possible responses: "Seems very rele- vant," "Not sure of the relevance," "Don't see how it could be relevant," "It's not relevant." 2. "Based on my experience so far, I believe the simulation is relevant to business ethics." Possible responses: "Strongly agree," "Agree," "Neither agree nor disagree," "Disagree," "Strongly disagree." For the purposes of analysis to determine changes in per- ceived relevance over the course of the simulation, responses to the two somewhat different questions were combined into two categories:  Believe simulation relevant: This combines from question 1 "Seems very relevant" and from question 2 "Strongly agree" and "Agree."  Uncertain or skeptical: This category collects all other re- sponses to these two questions: from question 1 "Not sure of the relevance," "Don't see how it could be relevant," and "It's not relevant" and from question 2 "Neither agree nor disagree," "Disagree," "Strongly disagree." In the aggregate, student's perceptions of the relevance of the simulation to business ethics strengthened over the course of the simulation: the percentage of student's believing the simula- tion is relevant increased from 75% to 83% (see Exhibit 4). It is not clear that this difference is significant. A chi- squared goodness of fit test indicates that the null hypothesis (H0: no significant change in frequencies over course of simula- tion) should be rejected (chi-squared = 4.77, critical value of 3.84 at alpha = 0.05 and one degree of freedom). However, a Kolmogorov-Smirnov indicates the null hypothesis should be accepted at the 0.05 level of significance (D=0.077 and a criti- cal value of 0.112). SATISFACTION Although there is some controversy about the relationship between student enjoyment and learning (Gosen & Washbush, 2004, p. 277), student satisfaction with their experience is none- theless an important desired outcome of any teaching method. Students playing Deepwater were asked at several points during the simulation about their level of satisfaction with their simula- tion experience. Exhibit 5 presents the results from surveys ad- ministered at the half-way point (after round 6 of a 12 round game), and after the simulation concluded. Direct inspection of Exhibit 5 suggests that student satis- faction with the simulation increased during the experience, driven primarily by a declining number of undecided and an increasing number of those who rated their simulation experi- ence "Very good" or "Good." However, a Kolmogorov-Smirnov test yields a D of 0.078, a bit less than the critical value of 0.107, indicating the differences in distributions mid- and post- simulation are not statistically significant at the 0.05 level. In any event, roughly 3 in 4 students rated their simulation experience "Very good" or "Good" at both the halfway point and after the simulation concluded. Another satisfaction-related question asked for students' opinions about whether the simulation should continue to be used in the course in subsequent semesters. Students were asked whether the instructor should continue using the simula- tion with no changes, minor changes or major changes, or stop using it altogether. Over 9 out of 10 students recommended keeping the simulation: of 186 survey responses, 45% said con- tinue using the simulation as is, 46% recommended keeping the simulation but making some improvements, 3% recommended making major changes, and 3% said the simulation should not be used at all in future classes (3% gave other, non-classifiable responses). OUTCOMES Perceived learning value, engagement, perceived relevance to business ethics and satisfaction with the simulation are im- portant preconditions for a successful learning experience. As I have just shown, results from student surveys indicate that all these conditions are met by Deepwater. But is the simulation able to deliver on its pedagogical objectives? Students were asked several outcomes-related questions at the conclusion of the simulation. The results from two of these are presented here. Changing behavior begins with changing someone's under- standing (Scherpereel, 2005, p. 389). Evaluating Deepwater's effectiveness must begin with determining whether students' understanding of the ethical challenges facing business was changed by their simulation experience. EXHIBIT 5 SURVEY RESULTS: "HOW WOULD YOU RATE YOUR SIMULATION EXPERIENCE SO FAR?" Mid-Simulation Post-Simulation Very good 24% 29% Good 50% 53% Neither good nor bad 22% 14% Bad 5% 3% Very bad 0% 1% Totals 100% 100% n 106 161 Page 162 - Developments in Business Simulation and Experiential Learning, volume 42, 2015 Of 152 students responding, 97% reported that the simula- tion experience improved their understanding of the ethical challenges facing businesses at least to some degree. Sixteen percent reported that their understanding of business ethics in- creased "greatly," 43% that their understanding had increased "significantly," 27% reported a "modest" increase, 11% said "a little bit" and only 3% "not at all." Understanding is one thing, taking the correct action anoth- er. One cannot know how students who played the simulation game will act in the future, but they themselves are likely to have opinions about whether an experience has changed their perceptions and understanding enough to result in changed be- havior. There are undoubtedly difficulties attendant upon rely- ing too heavily on students' own beliefs about their future be- havior (see Gosen & Washbush, 2004, p. 277). Yet it is imprac- tical to collect data on students' future behavior, so we are forced to rely on information that is available at the time of the simulation experience. Students were asked to predict the effect of the game on their future behavior: "Playing the simulation has improved the chances I will make responsible business decisions when those decisions could harm workers, customers or the public." Seven- ty-six percent of 200 students responding "strongly agreed" (21%) or "agreed" (55%) with the statement that the simulation has improved the likelihood that they will make re- sponsible business decisions in the future. Twenty-one percent responded "neither agree nor disagree," 3% "disagree" and 2% "strongly disagree." Data from surveys of students who have participated in the simulation thus support claims that roughly eight in ten had a good experience and believe they benefited from playing the simulation game: 1. After playing the simulation game, 82% believed it had been a good way to learn. 2. Roughly 8 in 10 reported being very or moderately engaged both early on in the simulation and at the end. 3. The share of students who understood the relevance of the simulation to business ethics increased from 75% before their simulation game began to 83% at the end. 4. By the end of their simulation game, 82% of students said their experience was "good" or "very good." 5. Ninety-seven percent reported an increase in their under- standing of the ethical challenges facing businesses 6. Seventy-six percent of students believe that the experience will change their future behavior. CONCLUSION As with case studies, simulations can bring real-world am- biguity, messiness and uncertainty into the classroom to flesh out textbook abstractions. They improve on case studies by vis- cerally engaging students and drawing them into grappling on a personal level with the challenges of managing a business re- sponsibly. Simulations cannot stand alone in the business ethics classroom, however. Their effectiveness depends on exposing students to the concepts and principles of responsible business management, and to a guided process of reflection. An under- standing of concepts and principles enables students to general- ize their simulation experience to apply it to real world situa- tions. Reflection on the simulation experience unfolds the full meaning of the experience and connects it to students' own work experiences. The Deepwater simulation, built from the ground up as a business ethics simulation, has been effective as a business eth- ics teaching method. It demonstrates the promise and practicali- ty of simulations as tools in support of improving students' abil- ity to make ethical decisions once they entire the work force. REFERENCES Faria, A. J. (2001). The changing nature of business simulation/ gaming research: A brief history. Simulation & Gam- ing, 32(1), 97-110. Faria, A. J., Hutchinson, D., Wellington, W. J., & Gold, S. (2009). Developments in business gaming. Simulation & Gaming, 40(4), 464-487. Feinstein, A. H., & Cannon, H. M. (2002). Constructs of simu- lation evaluation. Simulation & Gaming, 33(4), 425- 440. Fritzsche, D. J., Rosenberg, Richard D. (1989). Business ethics, experiential exercises and simulation games. Develop- ments In Business Simulation & Experiential Exercis- es, 16, 46-49. (Reprinted from Bernie Keys Library (11th ed.)) Gosen, J., & Washbush, J. B. (1997). Antecedents of learning in simulations. Developments in Business Simulation and Experiential Learning, 24, 60-67. (Reprinted from Bernie Keys Library (11th ed.)) Gosen, J., & Washbush, J. (2004). A review of scholarship on assessing experiential learning effectiveness. Simula- tion & Gaming, 35(2), 270-293. Hofstede, G. J., de Caluwé, L., & Peters, V. (2010). Why simu- lation games work: In search of the active substance: A synthesis. Simulation & Gaming, 41(6), 824-843. LeClair, D. T., & Ferrell, L. (2000). Innovation in experiential business ethics training. Journal of Business Ethics, 23 (3), 313-322. LeClair, D. T., Ferrell, L., Montuori, L., & Willems, C. (1999). The use of a behavioral simulation to teach business ethics. Teaching Business Ethics, 3(3), 283-296. Lu, J., Hallinger, P., & Showanasai, P. (2014). Simulation- based learning in management education: A longitudi- nal quasi-experimental evaluation of instructional ef- fectiveness. Journal of Management Development, 33 (3), 218-244. Mayer, B. W., Dale, K. M., Fraccastoro, K. A., & Moss, G. (2011). Improving transfer of learning: Relationship to methods of using business simulation. Simulation & Gaming, 42(1), 64-84. National Commission on the BP Deepwater Horizon Oil Spill and Offshore Drilling. (2011). Deep water: The Gulf oil disaster and the future of offshore drilling. Wash- ington, D.C.: National Commission on the BP Deep- water Horizon Oil Spill and Offshore Drilling. Scherpereel, C. M. (2005). Changing mental models: Business simulation exercises. Simulation & Gaming 36(3), 388 -403. Page 163 - Developments in Business Simulation and Experiential Learning, volume 42, 2015 Schumann, P. L., Anderson, P. H., & Scott, T.W. (1997). Using computer-based simulation exercises to teach business ethics. Teaching Business Ethics, 1(2), 163-181. Schumann, P. L., Scott, T. W., & Anderson, P. H. (2006). De- signing and introducing ethical dilemmas into comput- er-based business simulations. Journal of Management Education, 30(1), 195-219. Scott, T. W., Schumann, P. L., & Anderson, P. H. (1998). Ethi- cal dilemmas to use with business simulations to teach business ethics. Developments in Business Simulation and Experiential Learning, 25, 83-89. (Reprinted from Bernie Keys Library (11th ed.)) Sims, R. R. (2002). Debriefing experiential learning exercises in ethics education. Teaching Business Ethics 6(2), 179- 197. Snow, S. C., Gehlen, F. L., & Green, J. C. (2002). Different ways to introduce a business simulation: The effect on student performance. Simulation & Gaming, 33(4), 526-532. Tiwari, S. R., Nafees, L., & Krishnan, O. (2014). Simulation as a pedagogical tool: Measurement of impact on per- ceived effective learning. The International Journal of Management Education, 12(3), 260-270. Wellington, W. J., Faria, A., Hutchinson, D., & Gowing, M. (2014). An interdisciplinary study of the impact of playing a marketing simulation game on student knowledge of management accounting/finance princi- ples. Developments in Business Simulation and Expe- riential Learning, 38, 320-326. (Reprinted from Ber- nie Keys Library (11th ed.)) Wolfe, J. (1997). The effectiveness of business games in strate- gic management course work. Simulation & Gaming, 28(4), 360-376. Wolfe, J., & Fritzsche, D. J. (1998). Teaching business ethics with management and marketing games. Simulation & Gaming, 29(1), 44-59.