Large-Scale Business Games for Assurance of Learning Purposes Page 185 - Developments in Business Simulation and Experiential Learning, volume 40, 2013 ABSTRACT It has been implied and asserted that business games can play a major role in meeting the AACSB’s Assurance of Learning directives. This study analyzed the online playing behaviors of 836 undergraduate students on 182 teams in twenty six industries playing a very complex game. The top -tier firms exhibited higher degrees of problem-solving Variety, Depth and Endurance, higher levels of within- group business function information gathering than each industry’s bottom-tier companies. The top-performing companies were also more externally-related based on the proportions of information sought. It was concluded a large-scale game requires of its successful players, group problem-solving and decision-making skills the AACSB believes are needed for today’s well-educated business student. INTRODUCTION In 1991 the AACSB took the concepts of the total quality movement that were being taught in its accredited schools and made them part of its mission-based accreditation procedures. Under these new “outcomes assessment” guidelines schools were allowed great flexibility in determining whether their programs were meeting the goals of their mission statements. Schools typically used indirect accomplishment measures in the form of conveniently-gathered student, alumni and employer feedback. Two years later, based on the lack of the creation of true, direct measures of learning, the AACSB called for the use of internally-derived measures of coursework and program success. Most schools responded to this requirement through the use of instruments such as course grades, teacher evaluations, graduation rates and other instructor-controlled methods. Because of halo effects, subjectivity and the lack of comparability between programs and institutions associated with these types of instruments, the AACSB strengthened its guidelines. These were done under its Assurance of Learning (AoL) initiative (AACSB, 2007). These guidelines suggested the use of objective, standardized tests, and rigorous, twice-reviewed analyses of student work for both individually and jointly prepared cases and class exercises and projects. It has subsequently gone even further, in its popular publications (Bisoux, 2008) and in its most-recent guidelines regarding games and active learning methods. As stated in its Eligibility Procedures and Accreditation Standards for Business Accreditation (2011, 57), The most effective learning takes place when students are involved in their educational experiences. Passive learning is ineffective and of short duration. Faculty members should develop techniques and styles that engage students and make students responsible for meeting learning goals. Many pedagogical approaches are suitable for challenging students in this way– problem- based learning, projects, simulations, etc. This paper investigates the degree a large-scale business game rewards those players and teams that engage in the problem-solving and group decision-making skills that now-must be measured according to the AACSB’s action-oriented accreditation standards. It is important to investigate whether games can be used in this fashion. This is not only because of the AACSB’s endorsement of them for such a purpose, but also because the field’s most- popular games supply adopters with examinations that can be used in such a fashion (The Business Strategy Game, n.d.; CAPSTONE, n.d.; Marketplace Live, n.d.). In investigating this matter the paper took an action-oriented perspective by examining the online decision-making behaviors exhibited by those playing a business game. It was reasoned that such a game could be used for assurance of learning purposes if economic success in the game was associated with the higher use of the problem-solving and group decision-making skills the AACSB believes must be practiced in the schools it accredits. LITERATURE REVIEW Since their inception in the late-1950s, business games have been recognized as being problem-solving exercises (Anderson & Lawton, 2004). Depending on the game’s scope, depth and detail, a good business game embraces all the structural elements of a good, teachable problem. As elaborated by Lohman (2002), the exact nature, or even the existence of a problem, should not be clear. The information needed to solve the problem may not exist, or may be not easily captured. If it has been determined there is a problem, there should be different ways to solve it with each solution having different costs, variable outcomes and different degrees of harmful side effects. Based on the amount of ambiguity and indeterminacy associated with each link in the process, there is no single perfect answer or guaranteed result. This amount of problem-solving ambiguity is greatest at the firm’s strategic apex. This ambiguity still exists at the firm’s operating core but decisions taken there have higher levels of instrumentality (Mintzberg, 1979). Large-Scale Business Games for Assurance of Learning Purposes· Joseph Wolfe Experiential Adventures LLC Jwolfe8125@aol.com file:///C:/Users/Darci/Documents/ABSEL/2013/unedited/LargeScaleABSEL2013ProceedingsCopy.docx#_ftn1#_ftn1 mailto:Jwolfe8125@aol.com Page 186 - Developments in Business Simulation and Experiential Learning, volume 40, 2013 Because business games are teaching and learning devices, an authoritative game possesses the types of validities required, which in turn bring about a valid learning experience (Feinstein & Cannon, 2002). Those who play these games, however, bring to the game decision -making skills and tools that may have not been completely understood, used in action-oriented applications, or applied within a larger context. This is especially true for their use in strategic management courses, the most-frequent application of these games (Faria & Nulson, 1996). It is expected that each company’s novice decision-makers will more or less use coursework theories and tools to solve the firm’s problems at both the tactical and strategic levels. Company success and profits ensue if a successful match between strategy and tactics eventually has been obtained (Wolfe & Chanin, 1993). The cognitive and tactile skill needed to solve a Rubik cube’s riddle was chosen as a guide to the problem-solving process required of a business game player. When a business game is played much thought is involved but those thoughts are only revealed by the results of those thoughts. The game player must find a series of continuous solutions to the problems their company faces. If the game is delivered online, physical movements are required for logging on, scrolling screens and inputting decisions. Much of the same is associated with solving the Rubik cube’s riddle. The task is to reassemble a scrambled set of six differently colored faces so that only one of those colors appears on each of the cube‘s six faces. The solution entails a number of face moves, the sequential solving each of the cube‘s three layers must be done without destroying any of the previously constructed layers. The player must have the patience needed to make the number of moves to reach the cube’s solution. Many do not have the patience or endurance to go through this process as the cube’s pieces can be arranged in 519 quintillion different ways. Broken down, however, the problem-solving process requires the following of the decision-maker: 1. Variety— Consider the array of cube faces shown and then proceed to make a series of moves within an overall strategy for solving the cube’s riddle. 2. Depth— Drill down or envision the arraignments of cubes below its original and evolving surfaces within the chosen strategy that will move the arrangements to a final solution. 3. Endurance— Persist in the logical turning of cube faces until the riddle has been solved. This process, when applied to the playing of an online- delivered business game, would entail the following: 1. Variety— Viewing or visiting a wide array of different screens associated with each of the game’s functional decision-making areas. 2. Depth— Digging into the options, and alternative solutions, associated with each functionally-related decision. 3. Endurance— Staying on course until all the game’s strategic and tactical decisions have been made and submitted for processing. HYPOTHESES TESTED If a business game is to be used as a tool for establishing a baseline of the knowledge being delivered by a course or degree program, it must require its players to successfully engage in the problem-solving process. In an operant conditioning sense, those decisions, and the decision-making behaviors that brought about negative results, will be punished. Those decisions, and the decision- making behaviors that brought about positive results, will be reinforced. Those behaviors that appear to have no or little effect will be extinguished (Thorndike, 1913). If a game’s players are fully engaged in the experience they should more or less begin to sense what works and what does not work. This amount of engagement, however, requires from them Variety, Depth and Endurance if they are to succeed. This study employed three hypotheses to see if a relationship existed amongst the amount of Variety, Depth and Endurance enacted by a company was related to its economic success. The hypotheses were: H1: High tier companies will demonstrate more Variety in their decision-making processes than low tier companies. H2: High tier companies will demonstrate more Depth in their decision-making processes than low tier companies. H3: High tier companies will demonstrate more Endurance in their decision-making processes than low tier companies. METHODOLOGY The study’s participants (n=836) were first-year undergraduate business school students majoring in international business. They were attending a west European university. It has an enrollment of about 14,500 students per year. The university’s course assignment system randomly assigned players to teams. This resulted in mostly four and five-member firms. Twenty-six seven- Exhibit 1 Company Team Sizes Members Companies Players 1 1 1 2 15 30 3 23 69 4 41 164 5 52 260 6 38 228 7 12 84 Total 182 836 http://en.wikipedia.org/wiki/Quintillion Page 187 - Developments in Business Simulation and Experiential Learning, volume 40, 2013 firm industries were created. Exhibit 1 indicates the eventual sizes of the teams reviewed by the end of play. Course drops occurred but they were not above the norm. Sixty percent of the course’s grade was based on the firm’s ranked profitability within its industry. The remaining forty percent was based on the grade the team received on their completion of a strategy planning and evaluation form. Peer reviews were conducted within each team. The majority of a player’s reviews had to be an “Acceptable” to pass this part of the course’s grade requirements. During this study’s data-gathering period 99.9% of its participants received an “Acceptable” from their team mates. The game was played intensively over a two-week period excluding an intervening weekend. Each industry ran for two and one-half years or ten simulated business quarters with all running under the same economic conditions. For this study’s purposes a firm’s success or tier assignment was indicated by its final four-quarter profits. The use of each game’s last four quarters has been found to reflect the degree to which each industry’s firms benefitted or suffered for the quality of their decisions. Using the results from these last four periods also recognizes that the rational firm would make no new strategic moves at the game’s end where there would be insufficient time to realize the benefits of such moves (Wolfe, 2013). At the end of play each industry’s top-ranked and bottom-ranked firms were classified as those being in the game’s first or last tier based on their ranked earnings for the game’s last four periods. The problem-solving dimensions of Variety, Depth and Endurance were measured in the following fashion:  Variety—The average number of different screens viewed/visited by players on high versus low tier companies.  Depth—The average amount of time spent viewing/ visiting each screen by players on high versus low tier companies.  Endurance—The average amount of Variety and Depth - time, over the game’s duration, spent by players on high versus low tier companies. RESULTS PROBLEM-SOLVING VARIETY Three ways were used to test the degree that companies had to be engaged in problem-solving Variety if their companies were to be successful. The first analysis looked at the absolute number of screens visited by the average company. It was reasoned, that to make an intelligent decision, the team’s players must view the types of decisions that could be made. The second Variety test dealt with the types of decisions viewed. It could be surmised that better-competing companies concentrated on those areas that were externally related and new-business related, or that the poorer performing companies focused their attention on either internal affairs, or one or two decision areas while sacrificing their attention on other areas. The last test looked at the degree screen visits were evenly distributed within the company’s management group. Ideally all team members would participate equally in their screen visits so they would be equally prepared for their company’s decision-making process. If the company’s screen visits were highly concentrated in one or two managers, the Variety of decision-making inputs, and the power of those inputs would be restricted to the few managers. ABSOLUTE SCREEN VISITS TEST Exhibit 2 displays the average number of screens visited by the players on high tier vs. low tier companies during the time they managed their companies. The average high tier company visited about 70.6% more screens. The individual players on those companies visited about 51.5% more screens. These differences were significant p <=0.003. DIFFERENT SCREEN VISITS TEST This test was conducted based on the assumption that the more-successful companies would obtain greater Variety by viewing more-different proportions of their company’s decision-making areas. Based on the relevant management literature, they would more-often view their company’s boundary spanning screens (Burns & Stalker, 1961; March, 1991). They would do this as these screens would help them scan more features in their company’s environment (Choo, 2001; Daft, Sormunen, & Parks, 1988). The company’s decision-makers would also work more in the funds acquisition area of financial management and strategic alliance prospecting. This would have to be done in order to finance new ventures. It would also be expected that the highly successful companies would make proportionally fewer visits to their company’s “internal affairs” such as accounting, logistics and operations management and more to its marketing strategy design and channel control efforts (Mavondo, 1999). A chi-square test of the proportions of screens viewed by each group is presented in Exhibit 3. There were significant differences between the two groups except for attention paid to viewing the game’s strategic alliances screens. These screens allowed companies to purchase or license patents, become a subcontractor or purchase or buy capacity from other firms in the industry. Accounting was not of lesser importance for the highly successful companies, as had been presumed. All other differences in proportions were as expected. Exhibit 2 Average Screen Visits Decision-Making Level Tier First Fourth Player 111.9 73.8 Company 4,935.8 2,893.2 Page 188 - Developments in Business Simulation and Experiential Learning, volume 40, 2013 WITHIN-TEAM VARIETY VISITS TEST A Lorenz curve, along with its Gini Coefficient, was used to examine the amount of screen-viewing inequality or screen time concentration within companies. It has been found that participation rates fall as the decision-making group gets larger. In teaching applications the “free rider” phenomenon is well known (Latane, Williams & Harkins, 1979; Karau & Williams, 1991). It could be reasoned, however, that the more successful companies were able to elicit more within-company screen visits than was the case for the low-performing companies. This greater degree of participation, and making themselves better-prepared for any face-to-face decision-making sessions, should result in a greater range of inputs than would otherwise be the case. The average Ginis associated with each group are presented in Exhibit 4 after considering all companies with three or more players. Gini Coefficients tending towards 1.0 would indicate a high concentration of screen visits amongst very few team members. A Gini of 0.0, or one tending towards 0.0, would indicate a complete or near-equality of screen visits. Although it was previously found that the high-tier companies visited more screens, there were no differences between the groups regarding the within-team dispersion of screen visits. PROBLEM-SOLVING DEPTH This study’s previous section examined the total number of different screens a company’s players viewed. It is known from that section’s results that the highest- performing companies viewed the same screens more often, but did not look at a wider Variety of them. Moreover, based on their associated Ginis, the dispersion of screen visits was equal between their members. While these are important first steps in the decision-making process, the amount of time, or Depth of each screen visit is also important. There could be many visits but each visit could be a superficial or fleeting event. Other companies could visit fewer screens but could study each one in greater detail. This section presents the amount of association found between a firm’s screen time and the company’s economic success. SCREEN TIME TEST It was hypothesized that high-performing companies would spend more time on their screens than would an industry’s low-performing company. This hypothesis was accepted. As shown in Exhibit 5, after controlling for team size, the average first tier player spent about 1,373.1 total minutes on the game. This amounted to almost 24.0 hours of work. The average fourth tier player spent about one- half of that amount of time or about 12.6 hours. These differences are significant p <0.001). DECISION-MAKING AREA SCREEN TIMES TEST It was previously found that each success group visited a different set of screens. Exhibit 6 shows the total minutes devoted to each decision area by success group. There were no significant differences between the two success-rate groups regarding the proportions of their screen minutes devoted to the seven decision areas available. The high tier companies, however, spent a significantly larger amount of time on all their decision areas (p < 0.002). PROBLEM SOLVING ENDURANCE A longitudinal perspective is added to the problem- Exhibit 3 Decision Area Visit Proportionsa Decision Area Tier First Fourth Scanning 12.0% 9.4% Accounting 13.9% 10.8% Finance 12.9% 9.8% Logistics 5.8% 5.1% Marketing 20.0% 23.1% Operations Management 34.2% 41.8% Strategic Alliancesb 1.1% 0.1% Total 100.0% 100.0% aOverall chi-square 1918.5, df=6, p=0.0001 bChi-square = 8.36, df=1, p=0.004 with Yates’ correction Exhibit 4 Screen Visit Gini Coefficients Average Ginis Tier* First Fourth Company 0.51 0.55 Range Low 0.65 0.71 Range Width 0.30 0.32 Range High 0.34 0.38 *Non-significant differences p = 0.07. Exhibit 5 Average Total ScreenVisit Time (In minutes) Decision-Making Level Tier First Fourth Player 1,373.1 758.0 Company 6,231.8 3,294.5 Page 189 - Developments in Business Simulation and Experiential Learning, volume 40, 2013 solving process when considering the decision-making group’s Endurance. In a game’s use all aspects of the organizational learning phenomenon come into play. The game’s players are assembled by some typically impersonal method, their qualities may or may not be known, and the tasks they must perform, and the skills they can mobilize, have to be learned over time in an operant conditioning fashion. Cangelosi & Dill’s (1965) pioneering work in the field of organizational learning followed the decision- making dynamics of a seven-member company in The Carnegie-Tech Management Game (Cohen, et. al., 1960). Learning was displayed as the team went through the phases of Intuition, Searching, Comprehending and Consolidating. During this process the players’ actions and discussions became less random and grew to be more focused on essentials. In a sense they became more efficient, and spent less time on the game while also becoming more successful. This study attempted to capture this “learning” effect by (1) tracing firm and player Variety and Depth on a period-by-period basis and (2) using split-half tests of those two variables to see if there were changes overlooked when using the grand averages associated with the first two analyses. VARIETY ENDURANCE Exhibit 7 indicates significant differences existed between the two groups in the number of screens visited over time (p < 0.001). Both groups tracked in the same fashion with the high tier companies exhibiting a more variable path than that exhibited by the low tier companies. The peak number of visits occurred around the game’s fourth period. It was at this time the effects of the major moves were being felt. These effects were associated with expansions of American factories, building new greenfield plants or opening new markets via export sales. Once these moves had been made, it was reasonable that fewer screen visits would be needed, or that the firms had exhausted Exhibit 6 Average Screen Minutes by Major Decision Area Decision Area Tier First Fourth Scanning 20,016 8,697 Accounting 28,064 10,014 Finance 14,614 5,471 Logistics 9,736 4,302 Marketing 32,285 18,934 Operations Management 55,022 37,459 Strategic Alliancesb 2,289 779 Exhibit 7 Average Variety Per Decision Round Page 190 - Developments in Business Simulation and Experiential Learning, volume 40, 2013 what they considered to be major endeavors with reasonable paybacks. A split-half test was conducted to see if the emphasis, as indicated by screen visits, changed by major decision area over time. This split-half test examined every company’s first four periods vs. their last four periods. The first periods were chosen as they would be involved with the company’s Intuition and Searching phases, along with making a number of very expensive decisions. The last four periods were chosen as they entailed Cangelosi & Dill’s Consolidating phase where no major decisions would be made as there would not be enough time for any investment to make a return, and the firm would now be able to retire short-term debt, call outstanding Bonds and issue Dividends. An ANOVA indicated that when the time dimension of Endurance was added to the analysis there were overall significant differences between beginning and ending periods. The proportions of these visits are presented in Exhibit 8 which also brings out the fact that both groups changed over time regarding their attention foci (Cyert & March, 1992). While it has been found that none of the teams shared their problem solving loads equally Exhibit 9 tracks the Gini coefficients for the number of screens visited over time. There were higher levels of concentration for the low performing companies at the beginning (p = 0.04) period and ending three periods (p = 0.006) than was the case for the high performing companies. The amount of dispersion and concentration was also more variable than that for the high performing companies. In this case there were significantly different coefficients of determination (p < 0.0001) and the low tier linear trend having lower predictive value. DEPTH ENDURANCE The previous two tests found the amount of Variety exercised by the two groups changed over time but that their trajectories found that less Variety occurred over time. Exhibit 10 displays the trends in the average amount of screen time expended by the average player over time. An ANOVA was performed on the variability of the paths shown in this exhibit. There was a significant variance in the average screen times (F = 29.71, df = 9, p < 0.0001) but both groups’ screen times fell at statistically equal rates. A test of mean between group differences in average screen times found the fourth tier companies always spent less screen time than the first tier companies (p< 0.001). RESULTS SUMMARY H1 stated that an industry’s high tier companies would demonstrate more Variety in their decision-making processes than would low tier companies. This hypothesis was supported when Variety was measured by the number and proportion of screens visited. It was not supported when Variety was measured by the average degree that company members’ visits were more equally dispersed or evenly shared within the team. H2 stated that high tier companies would demonstrate more Depth in their decision-making processes than would low tier companies. This hypothesis was accepted through two measurements. The high tier companies spent more total minutes on their screens on a per player basis. There also was no difference in the proportions of average times spent on company decision areas. The addition of the Endurance or time dimension to the decision-making process highlighted numerous behavioral changes. H3 hypothesized that high tier Exhibit 8 Proportions of Screen Visits by Game Period and Company Success Decision Area Tier First Fourth Game Period Game Period Early Late Early Late Scanning 8.1% 14.1% 11.9% 11.5% Accounting 18.2% 17.9% 15.2% 15.4% Finance 5.9% 14.0% 8.0% 7.3% Logistics 3.0% 7.3% 4.0% 5.5% Marketing 21.9% 18.1% 20.6% 21.0% Operations Management 42.3% 27.l7% 38.6% 38.3% Strategic Alliances 0.5% 1.0% 1.6% 1.0% Total 100.0% 100.0% 100.0% 100.0% Page 191 - Developments in Business Simulation and Experiential Learning, volume 40, 2013 Exhibit 10 Average Player Screen Time by Period Exhibit 9 Average Company Gini Coefficients by Performance Level Page 192 - Developments in Business Simulation and Experiential Learning, volume 40, 2013 companies would demonstrate more Endurance in their decision-making processes than low tier companies. This hypothesis was supported for both their Variety and Depth. The low performing companies also were more concentrated at their company’s beginning and ending three rounds. Overall screen times fell at the same rate for both groups but the high performers always spend more time on a quarterly basis. DISCUSSION For comparative purposes, this study used the polar opposites of success cases. This was done to put the study’s issues in sharp contrast. In reality, in any business game experience and for this application, besides there being a first-place and a seventh-place firm, there were also firms that placed at in-between ranks. Thus there could have been any number of strong second and third-place firms. Additionally, bottom-tier finishers could have been very profitable companies but just did not do as well as others. A smaller analysis was conducted on the dimensions of Variety, Depth and Endurance for firms that ranked third and fifth in their industries. This was done to see if the stark contrasts created by this study’s first vs. last analysis pertain to them. A sample of the Variety, Depth and Endurance was conducted for second and third tier firms. That is teams whose final profits put them at their industry’s third and fifth ranks. The results of the test of problem-solving Variety are presented in Exhibit 11. It can be seen that there was a continuing decline in screen visits. These differences were significant p = 0.008 or better. Thus, there appears to be a continuum of lessening screen visits over the range of economic outcomes obtainedby the game’s average company. Problem-solving Depth was measured earlier where it was found the first tier players spent about twice as much time at their screens than did the fourth tier players. Exhibit 12 indicates that the Depth levels of the typical second and third tier players resided at their industry’s middle ground. There was no significant difference between their minutes of screen time between them but the difference in their screen times between both the first tier and fourth tier companies was respectively significant at the p = 0.05 and p = 0.03 levels. Thus, the continuum associated with screen visits seems to hold true for screen times. The last measures dealt with the Endurance levels applied to problem-solving Variety and Depth. Exhibit 13 shows the linear trends associated with the Variety of screens visited are stacked in their order of average number of screens visited. The high coefficient of determination Exhibit 12 Total Average Minutes Per Player Tier Average First 1,373.1 Second 1,141.8 Third 1,042.7 Fourth 758.0 Exhibit 11 Average Company Screen Visits by Performance Tier Page 193 - Developments in Business Simulation and Experiential Learning, volume 40, 2013 associated with the fourth tier companies was interpreted as being an indication that this group of companies was less- variable in the number of screens visited while the opposite was true for the first and third tier companies. This study sought to determine if a large-scale, total enterprise game required its players to engage in the problem-solving process. It was found that those behaviors that indicate engagement in the decision-making process were more-often associated with high company performance. By the use of screen visits, times per visit, and the rate of decay of both of these visits, the study accessed what the players could do during their visits. As an online game the players could view, edit, save and submit their sessions. They could also print-out various forms as well as transporting them to spreadsheets. This use of screen visits and times per visit, however, does not tap the thought processes associated with those actions. This study’s overall results should bring comfort to those who use business games. Those companies that “did the right thing”, in both strategic and tactical matters, and were dedicated enough to design and implement their strategies and tactics over the long haul, did relatively well. Direct and summary evaluations of the teaching results associated with business games, various authors have been able to state, after reading essentially the same literatures, that games do not produce (Neuhauser, 1976), do produce (Keys & Wolfe, 1990), or produce something that is not known or has not been correctly measured (Anderson & Lawton, 2009). This study, while not measuring learning, found that learning could take place, especially if the game’s players engage in the experience. Unfortunately the game also revealed the bane that has always been associated with teaching/learning situations—an unwillingness to be engaged in the learning experience, whether the experience is being facilitated via cases, lectures, movies or action projects. The measurement of problem is exacerbated even further, by gaming’s use of group decision-making teams, which allows those who wish not to be engaged can do so with little chance for detection or reprimand. CONCLUSION This study’s major data-collection device was an analysis of the online actions taken by the game’s players. There are other ways the players could have interacted with the game and their teammates, and these ways were not tapped. In examining the logs generated by the game, however, no player ever printed out their screen outputs, for possible dissemination to players who did not log on to the game. Additionally, no player ever used the game’s blogging feature to either chat or pass game-related material with each other although they could have e-mailed or text messaged their partners. Further research should also be conducted using other large-scale games. Games simpler than the one used in this study could be just as useful for assurance of learning purposes, while being less-burdensome on its adopters and players. Further research should also be conducted using other measures of player engagement in the learning process. User diaries, extended debriefings and recorded sessions have been used in past studies although all are subject to either the Hawthorne Effect or after-the-fact rationalizations. The AACSB (2010) proposes that “students achieve knowledge and skills for successful performance in a complex environment requiring intellectual ability to organize work, make and communicate sound decisions, and react successfully to unanticipated events.” In its publications it has provided or referenced testimonial-type evidence of the efficacy of computer-based games in Exhibit 13 Average Screens Visited by Companies by Period Page 194 - Developments in Business Simulation and Experiential Learning, volume 40, 2013 Assurance of Learning initiatives (Bisoux, 2008; Martell & Calderon, 2005). Because a game of the type used here appears to have created an appropriate decision-making environment to assess whether some of the AACSB’s proposals have been achieved, the next step would be to tie each team’s actual decisions to the school’s Mission Statement. The AACSB itself recommends that instructors have input into the types and levels of questions posed in any internally-derived instrument and therefore any game- related questions should test whether the student recognizes and correctly applies theory and tools supposedly taught. To this end, the publishers and authors of the field’s three major larger-scale games, have recognized their games’ potential and have created instruments they suggest can be used for assurance of learning purposes (CAPSTONE, n.d.; Marketplace Live, n.d.; The business strategy game, n.d.). A brief review of the questions posed by their instruments, however, reveals their questions pertain to game specifics rather than the theory and tools the players should have had to employ to both answer the question and successfully play the game. This observation assigns even greater credibility to the AACSB’s recommendation that the instructors should have input into the questions that supposedly detect whether learning has occurred. The questions that are posed should be theory- driven, tool-kit oriented and problem-solving in nature. Moreover they should be supportive of the school’s AACSB submitted mission statement (AACSB, 2011) rather than serving as indications that the players were technically engaged in the simulation itself as a product or teaching technology. Those that have created game test instruments should engage in research that objectively shows that a strong link exists between the business game firm’s results and the correct use of the course’s theories and tools as applied to the game’s problem-solving situation. When such links have been established the promises of business gaming’s earliest years will be realized and that a new use has been found for them as assessment devices. REFERENCES AACSB (2007). AACSB Assurance of Learning Standards: An Interpretation. Retrieved from www.aacsb.edu/ publications/whitepapers/AACSB_Assurance of_Learning.pdf. AACSB (2011). Eligibility Procedures and Accreditation Standards for Business Accreditation. Tampa, FL: AACSB International. Anderson, P. H., & Lawton, L. (2004). The effectiveness of a simulation exercise for integrating problem-based learning in management education. In Developments in Business Simulation and Experiential Learning, 32, 10 -18. . (Reprinted from Bernie Keys Library (12th ed.)) Anderson, P. H. & Lawton, L. (2009). Business s simulations & cognitive learning: Developments, desires, and future directions. Simulation & Gaming, 40(1): 193-216. Bisoux, T. (2008). Measures of success. BizED, (March/ April): 22-30. Burns, S., & Stalker, G. M. (1961). The management of innovation. London: Tavistock Publications. Cangelosi, V. E., & Dill, W. R. (1965). Organizational learning: Observations toward a theory. Administrative Science Quarterly, 10(2): 175-203. CAPSTONE (n.d.). Available from http:// www.capsim.com. Choo, C. W. (2001). Environmental scanning as information seeking and organizational learning. Information Research, 7(1):1-14 Cohen, K. J., Cyert, R. M., Dill, W. R., Kuehn, A. A., Miller, M. H., Van Wormer, T. A., & Winters, P. R. (1960). The Carnegie Tech management game. The Journal of Business, 33(4): 303-321. Cyert, R., & March, J. G. (1992). A behavioral theory of the firm. Hoboken NJ: Wiley-Blackwell. Daft, R. L., Sormunen, J., & Parks, D. (1988). Chief executive scanning, environmental characteristics and company performance: An empirical study. Strategic Management Journal, 9(2): 123-139. Faria, A. J., & Nulsen, R. (1996). Business simulation games: Current usage levels: A ten-year update, In Developments in business simulation and experiential exercises, 23, 22-28. (Reprinted from Bernie Keys Library (12th ed.)) Fienstein, A. H., & Cannon, H. (2002). Constructs of simulation evaluation. Simulation & Gaming, 33(4): 425-440. Green, D., & Faria, A. J. (1995). Are good strategies consistently good? In Developments in Business Simulation and Experiential Learning, 22: 31-37. (Reprinted from Bernie Keys Library (12th edition.)) Harrigan, K. R. (1988). Managing maturing businesses: Restructuring declining industries and revitalizing troubled operations. Lexington MA: D. C. Heath & Company. Harrigan, K. R., & Porter, M. E. (1983). End-game strategies for declining industries. Harvard Business Review 64(4): 111-120. Hamermesh, R. G., & Silk, S. B. (1979). How to compete in stagnant industries. Harvard Business Review, 57 (5): 161-168. Karau, S. J. & Williams, K. D. (1993). Social loafing: A meta-analytic review and theoretical integration. Journal of Personality and Social Psychology, 65(4): 681-706. Keys, B. J., & Wolfe, J. (1990). The role of management games and simulations in education and research. Journal of Management, 16(2): 307-336 Latane, B., Williams, K., & Harkins, S. (1979). Many hands make light the work: The causes and consequences of social loafing, Journal of Personality and Social Psychology, 37(6): 822-832. Lohman, M. C. (2002). Cultivating problem-solving skills through problem-based approaches to professional development. Human Resource Development Quarterly, 13(3): 243-261. March, J. G. (1991). Exploration and exploitation in organizational learning. Organization Science, 2(1): 71 -87. http://www.aacsb.edu/publications/whitepapers/AACSB_Assurance%20of_Learning.pdf http://www.aacsb.edu/publications/whitepapers/AACSB_Assurance%20of_Learning.pdf http://www.aacsb.edu/publications/whitepapers/AACSB_Assurance%20of_Learning.pdf http://ir.stthomas.edu/ocbmgmtpub/9/ http://ir.stthomas.edu/ocbmgmtpub/9/ http://ir.stthomas.edu/ocbmgmtpub/9/ http://www.capsim.com http://www.capsim.com http://faculty.babson.edu/krollag/org_site/soc_psych/latane_soc_loaf.html http://faculty.babson.edu/krollag/org_site/soc_psych/latane_soc_loaf.html http://faculty.babson.edu/krollag/org_site/soc_psych/latane_soc_loaf.html http://faculty.babson.edu/krollag/org_site/soc_psych/latane_soc_loaf.html Page 195 - Developments in Business Simulation and Experiential Learning, volume 40, 2013 Marketplace Live (n.d.). Available from http:// www.marketplace-live.com. Martell, K., & Calderon. T. (2005), Assessment of student learning in business schools: Best practices each step of the way, 1: 167-183. Tallahassee, FL: Association for Institutional Research. Mavondo, F. T. (1999). Environment and strategy as antecedents for marketing effectiveness and organizational performance. Journal of Strategic Marketing, 7(4): 237-250 Mintzberg, H. (1979). The structuring of organizations: A synthesis of the research. Englewood Cliffs NJ: Prentice-Hall. Neuhauser, J. J. (1976). Business games have failed. Academy of Management Review, 1(4): 124-129. The business strategy game (n.d.). Available from www.bsg-online.com Thorndike, E. L. (1913). Educational psychology: The psychology of learning. NY: Teachers College, Columbia University. Wolfe, J. (In press). A large-game test of early-determined game finishes. In Developments in Business Simulation and Experiential Learning. (To be reprinted in the Bernie Keys Library (13th ed.)) Wolfe, J. & Chanin, M. (1993). The integration of functional and strategic management skills in a business game learning environment. Simulation & Gaming, 24(1): 34-46. Wolfe, J., Biggs, W.D., & Gold, S.C. (In press). A replication and expansion of Teach and Patel’s discovery of early-determined endgame finishes. Simulation & Gaming. http://www.marketplace-live.com http://www.marketplace-live.com Table of Contents Volume 40, 2013 Leo Investor SERIOUS PLAY: ON-LINE PEPULATOR SERIOUS PLAY:ON-LINE GAME "MANAGE-ART" The Game Within the Simulation Game the Research Method Concept and Project With Game Design Implementation Managing Human Resources Simulation Rising Stars: A Study of Creativity as Experiental Learning in County Government Why do Gen Y Students Study Abroad? The Relationship Between Individual Growth and the Intent to Study Abroad Our Favorite Assignment: The In-Basket Activity Mirror-Mirror: How Can We Better Understand Our Consumption Behavior? Reflective Thinking, Writing and Imagery as Tools to Better Understand Abstract Marketing Concepts Increasing Intuitive Decision Making Speed and Accuracy by Further Understanding Intuitive Decision Making Using Emotional Means Group Exams: Are they Relevant and Reliable as a learning Tool? Integrative Learning: Exploring Opportunities in Business Simulations Simulated Tabletop Exercise for Risk Management - Anti Bio-Terrorism Multi Scenario Simulated Tabletop Exercise Simulated Tabletop Exercise for Electric Power Saving Management in a Small Organization Merging the Case Method and Simulation in Management Education: Is it Possible? Experiential Learning in Accelerated Human Resource Management Courses Digital Game Building as Assessment: A Study of Secondary Students' Experience Using an Investment Fund Simulation Integrated with a Business Game Creating Simulation Conditions Based On Real World Data Simulation Games in Training New Management Methods A Summary Overview of Cultural Differences in Higher Education Innovations And Future Directions In Education: Case Review For Best Online Teaching Practices An Implicit Measure of Forecasting Accuracy Immediate Feedback and Assessment Technique (IF-AT) testing forms: An overview of the tool and uses Teaching Secondary Mathematics: Pre-service Teachers' Digital Game Design, Pedagogy and 21st Century Skills Follow The Leader II Business Simulations: From Punch Cards to Web-Based University Engagement: Good Neighbors Budget Allocation Exercise Customizing Business Simulations: An Exploration and a Hierarchy A Review of the Simulation Research in the Academy of Management Journal: Suggestions for Strengthening the Research Conducted by ABSEL Members ABSEL Reflections: 40 Years Of Excellence, Now Going Forward How Many Options do Multiple-Choice Questions Really Have? The Management/Accounting Simulation Now 100% Web-based Design, Server Side Programming, and Implementation A Large-Scale Game Test of Early-Determined Game Finishes Large-Scale Business Games for Assurance of Learning Purposes The Gamification Of Education OB Simulation: A Hands-On Demonstration Converting Simulations for the Online Environment: The New Ginseng Game Repositioning Brands with the Web-Based Product Positioning Map Graphics Package Implications of Regulatory Focus Theory for Simulation and Experiential Learning Riskware: A Game for Teaching Software Project Risk Management Teamwork for Decision-Making Through Games: The Case of On-Line "Manage-Art" Virtual Pepulator, a Model for Teaching Negotiation Small-Scale Business Games for Assurance of Learning Purposes Incorporating Intellectual Property Issues into a Business Simulation© Nationalization and Privatization in a Computer-Assisted Business Game An Investigation of the Relationship of Plan Quality, Forecast Accuracy and Earnings Performance Under Equal and Unequal Starting Positions in Marketing Positions in Marketing Simulations Measuring the Performance Ranking Curve in Marketing Simulation Games The Effectiveness of SDM Method in Business Simulation Game 40 years of Brazilian S&G - Analysis and Perspectives Accounting For Externalities: Harnessing The "Face-In-The-Mirror" Phenomenon The Role Of Simulations In Organizational Learning: Building Individual Absorptive Capacity Assessing Project Management as an Academic Learning Outcome (ALO) At the Inflection Point: Designing Army Assessments "Economics in Practice" A Simulation Game for High School Students Teaching the Basics of Economics and Entrepreneurship Fueling the Force: Exploring Leader Priorities Implementing Mental Models: Extending Insight and Whole Person Learning Using Business Simulations To Introduce Business Concepts