Small-Scale Business Games for Assurance of Learning Purposes Page 258 - Developments in Business Simulation and Experiential Learning, volume 40, 2013 ABSTRACT Recent AACSB literature has suggested and cited the use of business games for Assur­ance of Learning (AoL) purposes. The study of 182 teams with 836 undergraduates playing a large-scale game found theoretical and practical support of this endorsement. This paper repli­cates the large-scale game's environment using a small-scale and equally interfaced game to see if similar AoL conclusions can be drawn. The small-scale game experience placed 62 undergrad­uate students on 22 teams competing in four identical industries. It was able to duplicate the problem solving situation’s need for Variety and Endurance but was equivocal regarding Depth. Further research should be conducted using games that possess other degrees of validity and complexity. INTRODUCTION Since 1991 the AACSB has slowly placed more pressure on its member schools to assess the outcomes produced by their programs. Under its "outcomes assessment" guidelines schools were encouraged to write mission statements that stated their goals. They were given great flexi­bility in how goal-accomplishment was measured. Most AACSB schools employed student course evaluations and alumni and employer surveys to substantiate that their goals were being accomplished. Two years later, the AACSB, recognizing the inadequacies of these measures, asked for more-direct indications of goal accomplishment. Most AACSB schools answered this call through the use of teacher rather than course evaluations, course grades, graduation rates and various instructor- controlled methods. Although schools tried to demonstrate goal achievement and teaching results by using course evaluations, course grades and graduation rates, these either do not directly measure learning, or suffer from halo effects and subjectivity. Therefore the AACSB issued an additional set of guidelines in 2007 in the form of its Assurance of Learning initiative (AACSB, 2007). These guidelines could also have been made in response to the findings and recommendations of the Spellings Commission (U.S. Department of Education, 2006). The AACSB initiative clearly endorsed the use of active learning methods, and in fact, stated these methods are the best ones for preparing graduates for their business careers as stated in its Eligibility Procedures and Accreditation Standards for Business Accreditation (2011, 57). The most effective learning takes place when students are involved in their educa­tional experiences. Passive learning is ineffective and of short duration. Faculty members should develop techniques and styles that engage students and make students responsible for meeting learning goals. None of this is news to ABSEL, JASAG, ISAGA and the Organizational Behavior Teaching Socie­ty. They have been advocating business games and experiential methods throughout their lifetimes. Although these organizations have often had to preach to those who were already be­ lievers, there are indications that the AACSB's recognition of the business gaming approach is having an effect on business school administrators. Bisoux (2008), in the AACSB's publication BizED, cited the use of CAPSTONE in the AoL programs associated with Montclair State Uni­ versity, Temple and Kennesaw State University. The Business Strategy Game's July 3, 2010 Assurance of Learning Report (The Business Strategy Game, n.d.) reported its use by 32,600 stu­dents making decisions for 10,500 companies at 360 schools in 25 countries. Other games, such as Marketplace at the University of Texas at Arlington, are also used for such purposes. Based on the growing use of business games for learning assurance efforts, as well as the AACSB's endorsement of them, a recent study by Wolfe (In press) examined whether a game actually creates the type of environment that demanded the skills, insights and tools needed business school graduates to solve real-world business problems. Using a mix of the program assessment objectives for Old Dominion's College of Business and Public Administration and those exampled by the AACSB (2007, 2012), the ideal game should present problem- solving situations where success is obtained if the following learning goals have been accomplished:  Ethical issues—Players can recognize and analyze the various ethical dilemmas presented to them and select the resolution that is fairest for all concerned.  Communications— Players can communicate any number of the issues facing their com­pany either in written or oral form.  Analytical problem solving— Players will be able to recognize and apply the appropriate model, theory or tool to understand and solve the various problems their firm faces. Small-Scale Business Games for Assurance of Learning Purposes Joseph Wolfe Experiential Adventures LLC jwolfe8125@aol.com mailto:jwolfe8125@aol.com Page 259 - Developments in Business Simulation and Experiential Learning, volume 40, 2013  Group participation— Players will engage in the collaborative behaviors necessary for the accomplishment of group task.  Global perspective— Players will recognize international opportunities and the risks asso­ciated with dealing with differing cultures, market structures and currencies. The Wolfe (In press) study used a very complex and elaborate business game to test whether business games can be used for an AoL initiative (The Global Business Game: World Edition, n.d.). It was reasoned that only a game of such a size would present the very rich and comprehensive decision-making environment demanded by an accredited AACSB program review. There are those who argue, however, that smaller or simpler games (1) can teach, (2) are easier to administer, or (3) their simplicity makes them better teaching environments by avoiding the complexity paradox (Cannon, 1995; Frazer, 1983; Teach & Murff, 2008, 2009). For these reasons, research on smaller-sized business games should be initiated. This is especially true as the market's most-popular games (The Business Strategy Game, n.d.; CAPSTONE, n.d.; Marketplace Live, n.d.) are smaller than the one used in the large-scale game study, and therefore their validation would have the greatest impact on using games for AoL purposes. LITERATURE REVIEW It is natural that the AACSB would cite business games as being the type of action-oriented teaching method and philosophy it advocates. The business gaming movement began in the late-1950s as a business school experiment. It combined systems theory, a student-centered approach to teaching/learning and fast-processing computers (Cohen, et. al. 1964; Graham & Gray, 1969). More recently, Anderson & Lawton (2004) noted that business games epitomize the problem-solving approach. They observed a good business game has all the elements associated with a teachable problem-solving situation. Given the game's complexity and breadth, its players must first determine whether a problem exists. If there is more than one problem they need to prioritize them and then search for possible solutions. In a business school environment the players should have mastered, at least at the theoretical level, any number of discipline-related tools that could solve the chosen problem. A particular tool is applied based on the expected results of using the tool. Once this has been done, the players should examine whether the expected result occurred. If it did not they either mis-applied the tool or did not understand all the ramifications associated with using the tool. At this point learning has occurred and the players can move onto any subsequent problems more efficiently if the same tool can Exhibit 1 Demographic Comparisons Demographic Statistic Class size 76 Game participants 62 Course drop rate 13.2% Average age 21.0 ACT/SAT percentile 56.6 High school GPA 3.25 Male 64.5% Female 35.5% Caucasian 72.4% Afro-American 7.9% Hispanic 5.3% Asian 2.6% American Indian 2.6% Foreign 3.9% Unknown 3.9% Other 1.3% Page 260 - Developments in Business Simulation and Experiential Learning, volume 40, 2013 E x h ib it 2 C o m p a ra ti v e G a m e S iz es a n d C o m p le x it ie s D ec is io n S m a ll G a m e L a rg e G a m e P ro d u ct s O n e — M o to r sc o o te rs T w o — 2 5 ” an d 2 7 ” te le v is io n s et s ta il o re d f o r si x m ar k et s M ar k et s T w o O n e to s ix P ri ce s O n e to t w o b as ed o n m ar k et s en te re d S ix t o t w e n ty -f o u r b as ed o n m ar k et s, p ro d u ct s an d p ri v at e la b el b id s F ac to ri es O n e o r tw o O n e to s ix F ac to ry s c h ed u li n g T w o s h if ts p er f ac to ry , fo u r d ec is io n s b y s h if t T w o s h if ts p lu s o v er ti m e p er f ac to ry , ei g h t d ec is io n s b y s h if t F ac to ry m ai n te n a n ce d ec is io n s O n e to t w o b y f ac to ri es u se d T h re e to e ig h te e n b as ed o n t ec h n o lo g ie s a n d f ac to ri es i n o p er at io n F ac to ry c ap ac it y a ct io n s O n e to t w o b y f ac to ri es u se d E ig h t to f o rt y -e ig h t b as ed o n f ac to ri es i n o p er at io n Q u al it y C o n tr o l N o T w o t o t w el v e b as ed o n f ac to ri es i n o p er at io n R es ea rc h a n d D ev el o p m en t N o O n e to s ix b as ed o n m ar k e ts e n te re d F ac to ry w o rk er s k il l le v e ls O n e F o u r to t w e n ty -f o u r b as ed o n p ro d u ct s an d f ac to ri es i n o p er at io n F ac to ry f o re m a n N o O n e to s ix b as ed o n f ac to ri es i n o p er at io n R o b o ti cs d ec is io n s N o S ix t o e ig h te e n b as ed o n t y p es i n st al le d R a w m a te ri al s d ec is io n s O n e to t w o b as ed o n f ac to ri es s u p p li ed S ix t o t h ir ty -s ix b as ed o n f ac to ri es i n o p er at io n F u n d s tr an sf er s N o O n e to s ix b as ed o n m ar k e ts e n te re d S al es p ro m o ti o n d ec is io n s T h re e to s ix b as ed o n m ar k et s en te re d T w o t o t w el v e b as ed o n p ro d u ct s o ff er ed E x ch a n g e ra te s T w o F iv e S al es o ff ic es d ec is io n s O n e to t w o b as ed o n m ar k et s en te re d T w o t o t w el v e b as ed o n m ar k et s en te re d S al es f o rc e d ec is io n s T w o .t o t o f o u r b as ed o n m ar k et s S ix t o t h ir ty -s ix b as ed o n m ar k et s a n d p ro d u ct s E m p lo y ee t ra in in g T w o b u d g et s p er m ar k et T h re e to e ig h te e n b as ed o n m a rk et s an d f ac to ri es D is tr ib u ti o n C e n te rs O n e to t w o b as ed o n m ar k et s en te re d T w o t o t w el v e b as ed o n c o u n tr ie s an d f ac to ri es W h o le sa le /R e ta il er d ec is io n s T w o t o f o u r b as ed o n m ar k et s en te re d T w o t o t w el v e b as ed o n n u m b er o f m ar k et s en te re d S h ip p in g d ec is io n s O n e to t w o b as ed o n m ar k et s T w e n ty t o s ix ty b as ed o n p ro d u ct s sh ip p ed a n d m ar k et s e n te re d B o n d a ct io n s O n e T w o t o t w el v e b as ed o n m ar k et s S to ck a ct io n s O n e T h re e S h o rt -t er m l o an s N o n e O n e to s ix b as ed o n m ar k e ts e n te re d S h o rt -t er m i n v e st m en ts N o n e O n e to s ix b as ed o n m ar k e ts e n te re d R es ea rc h r ep o rt s N o O n e to t w el v e a s p u rc h a se d E th ic al i ss u es N o n e T h re e C ri ti ca l In ci d e n ts w it h f o u r al te rn at iv es p er i n ci d en t S tr at eg ic a ll ia n ce s N o n e T h re e p er c o m p an y w it h m u lt ip le a ll ia n ce s A v er ag e te a m s iz e 2 .8 p la y er s 4 .6 p la y er s D ec is io n R an g e 2 6 -4 8 1 0 3 -4 6 8 C o m p an ie s in p la y 6 6 Page 261 - Developments in Business Simulation and Experiential Learning, volume 40, 2013 be used again, or they can use other tools for unfamiliar problem-solving situations. The cognitive and tactile skills needed to solve a Rubik cube's riddle was chosen in the original study as the method for tapping the problem-solving processes associated with a busi­ness game. When the game begins the typical game scenario presents the player(s) with a situa­tion that possess both short-term and long-term "goods" and "bads". Some type of situational analysis should be employed to understand the situation's nature and whether an inventory of so­lution-solving tools is available. There should be much thought involved as the company's man­agers drill down and move ahead. The task of solving the Rubik's cube riddle entails reassembling a scrambled set of six different colored faces so that the final cube has each face with one color. The player must study the cube, make a number of superficial face and sub-level moves that do not destroy the progress that has been made by previous moves. By the time the solution has been obtained the problem solver has made a Variety of moves, envisioned the Depths associated with each cube face, and has mani­fested the Endurance to patiently move through and overcome any frustrations involved. This same process needs to be used if a business game company is to obtain any result, and hopefully, optimal results. A business game team should do the following, for instance, if they are playing an online-delivered game: 1. Engage in Variety— View or visit a wide array of different screens associated with each of the game's functional decision-making areas. 2. Engage in Depth— Dig into the options, and alternative solutions made possible by the game's structure or complexity level by functional area. 3. Manifest Endurance— Generate and sustain high levels of Variety and Depth until the game's tactical and strategic problems have been addressed. METHODOLOGY This paper repeats the method used by Wolfe (In press) to examine whether the degree firms playing a large- scale game engaged in the previously described problem- solving pro­cess. It was reasoned that if the AACSB wants business school students to be engaged in a business game so that they will learn how to be better real-world decision makers, the simulation used by those schools must reward those students who solve problems correctly. This study's participants (n=62) were first-year undergraduate business school students. They were attending an eastern Tennessee university with an annual enrollment of about 5,000 students. Players were randomly as­signed to 3-member teams. Course drops occurred but they were not significantly different from the previous semester’s four non-game playing course sections. Four students failed to purchase a license to play the game. A comparison of the demographic differences by age, sex, ethnicity, high school GPAs and ACT/SAT scores of this group found no significant between them versus those who pur­chased licenses. Exhibit 1 presents the demographics associated with those playing the games. Sixty percent of the course's grade was based on the firm's ranked profits with­in its industry. A combination of peer performance evaluations and a term paper on their company's performance accounted for the remaining grade weight. A one-hour game overview presentation was made and the game then proceeded for six consecutive weeks with one game period or business quar­ter played each week. The game itself was a scaled down edition of the large- scale game played in the Anonymous3 (In review) study. The small-scale game used the same interface structure but was simplified regarding the number of decisions possible and the amount of Depth associated with each decision. Exhibit 2 contrasts the types and numbers of decisions possible for each game. The small-scale game's graphics and texts were also presented in a relaxed and congenial fashion through the use of cartoons. On the other hand, the large-scale game's appearance was austere and text-laden The Flesch Kincaid Grade Levels and Gunning Fog Indexes for the two games are presented in Exhibit 3. Exhibit 3 Screen Text Reading Scores For this study's purposes a firm's relative economic success was based on its last three-period earnings. Compa­ ny endgame periods were chosen as it has been found in two studies that a company's final standings are not known, or are not highly predictable, until the game's last decision periods (Biggs & Fritzsche, 2010; Wolfe, Biggs, & Gold, In press). These cumulative earnings were then ranked to generate companies that were in their industry's top level or bottom levels. Given the tier assignments obtained by their ranked end-game earnings, all top and bot­tom-tier firms were examined along the problem-solving dimensions of Variety, Depth and Endurance. These dimensions were measured as follows:  Variety—The average number of different screens viewed/visited by players in high versus low tier companies.  Depth—The average amount of time spent viewing/ visiting each screen by players in high versus low tier companies.  Endurance—The average amount of Variety and Depth - time, over the game's dura­tion, spent by players in high versus low tier companies. Reading Index Game Size Small Large Flesch Kincaid Grade Level 9.7 10.5 Gunning Grade Reading Ease 10.5 12.0 Page 262 - Developments in Business Simulation and Experiential Learning, volume 40, 2013 HYPOTHESES TESTED This study has posed its hypotheses in the form of the findings of the large-scale study, i.e., are the relationships between a player and team's amount of Variety, Depth and Endurance and company success the same as those that were found in the large-scale game study. The hypotheses were: H1: High tier companies will demonstrate more Variety in their decision-making process­es than will low tier companies. H2: High tier companies will demonstrate more Depth in their decision-making processes than will low tier companies. H3: High tier companies will demonstrate more Endurance in their decision-making pro­cesses of Variety and Depth than will low tier companies. RESULTS This paper’s section presents the array of tests used in the large-scale game study that examined whether such a game presented its players with an environment that required and rewarded those who engaged in problem- solving Variety, Depth and Endurance. As best as possible, given this paper’s small sample size, it will first test the hypothesis, and then indicate the degree the small-scale game produced results that were the same as those produced by the large-scale game. PROBLEM-SOLVING VARIETY EXAMINATION The large-game study tested problem-solving Variety three ways. Its first analysis looked at the absolute number of screens visited by the average company. It was reasoned that to make an intelligent decision, the team’s players must view the screens that presented the types of decisions that could be made. The second Variety test dealt with the Exhibit 4 Decision Area/Support Screen Decision Area/Support Screen Performance Level High Low Costs, GDP Forecasts and Bulletin Board 35 5 Currency Exchange Rates 18 2 Firm Summaries 37 6 Performance Indicators 35 10 Firm Operations Report 79 38 Firm Income Statement 119 37 Cash Flow 0 0 Firm Balance Sheet 32 6 Merlin Studies 30 21 Marketing Operations 354 131 Marketing/ Sales Promotion 244 111 Production/ Factory Capacity 363 236 Production/ Factory Operations 412 146 Production Subassembly Kits 234 77 Finance/ Stock 121 45 Finance/ Bonds 84 7 Pro Forma Results 155 51 Pro Forma Results/ Cash Flow Report 108 35 Pro Forma Results/ Income Statement 0 0 Note: F = 4.21, df=18, p=0.002. Page 263 - Developments in Business Simulation and Experiential Learning, volume 40, 2013 range of decisions viewed. It could be surmised that better- competing companies would look at more of their company’s options, or that the poorer performing companies would focus on one, or very few decision- making areas and disciplines. The large-scale study also looked at the degree screen visits were evenly distributed within the company’s management group. This analysis could not be performed in the small-scale study as it used three-member firms while the large-scale study’s team sizes ranged from 2-7 that allowed for a greater range of decision-making sharings. ABSOLUTE SCREEN VISITS TEST This test found that the average game screen was visited 5.39 times by those on the industry’s top-tier companies. The average screen was visited 2.11 times by the bottom-tier company player. The average high tier company visited almost three times more screens. The difference between the average number of screens visited by players was significant p <0.0001 in a two-tail test. DIFFERENT SCREEN VISITS TEST This test was based on the assumption that the more- successful companies would obtain greater within-team functional Variety by absolutely viewing more screens. Their attention to screens would be more varied or more- qually-equally distributed. The small-scale game presented players with eighteen decision-area screens. We already know that screens were visited more-often in total by the high-performing companies. The original study’s next test looked at Variety in an additional fashion. It examined whether there were any differences in the proportions of screens visited. Exhibit 4 presents the results of this test for all companies based on their economic performances. There were significant differences (F=4.21, df=18, p=0.002) between the two groups regarding the decision areas they visited. The major sources of this variation was associated with the low-tiered group dedicating a 16.1% greater proportion of their screen visits to manufacturing concerns and the high-tiered group spending a 54.5% greater proportion of visits to alternative ways to finance their company’s operations and 25.8% more visits to their company’s accounting information. The fact that overall differences existed between the two performance groups conforms to the findings associated with the large-scale game study. PROBLEM-SOLVING DEPTH This study’s previous section examined the total number of different screens a company’s players viewed. It is known from that section’s results that the highest- performing companies viewed the same screens more often and differed between the decision areas visited. While these are important first steps in the problem- solving process, the amount of time, or Depth of each screen visit is also important. There could be many superficial visits or fewer visits in great Depth and interest. On the other hand, one group’s screen navigating skills may have been better which would allow them to view Exhibit 5 Screen Visits Per Player Over Time Page 264 - Developments in Business Simulation and Experiential Learning, volume 40, 2013 more screens in less time. This section presents the degree of association found between a firm’s screen time and the company’s economic success. SCREEN TIME TEST It was hypothesized that high-performing companies would spend more time on whatever screens they visited. This hypothesis was rejected as there was no statistically significant difference between the total time spent by players by performance group. The high tier companies spent 1,817 total minutes on the game and the low tier companies spent 2,544 minutes making decisions for their companies. This wide difference, however, was not significant given the wide within-group period-by-period variances. Another test was conducted on the amount of time spent on the game on a per player basis. In this test the low- tier companies spent a significantly higher amount of time per screen than did the high-tier companies. The respective minutes spent was 3.18 vs. 0.71 (significant p=0.001 in a one-tail test) and is counter to the hypothesis that was tested. DECISION-MAKING AREA SCREEN TIMES TEST It was previously found that each of the two success groups visited a different set of screens. In the case regarding Depth, there were no significant differences between the two groups regarding the proportions of their screen minutes devoted to their decision areas. PROBLEM SOLVING ENDURANCE The analyses that have been conducted thus far used summary data. When Endurance is added a longitudinal perspective enters into the analysis of the problem-solving process. This sequential perspective is especially relevant when judging the value of using a business game. This perspective captures a team’s need to change and adapt to its evolving situation. The team’s within team dynamics also entails different learning rates that can have negative affects on teamwork and outputs. The large-scale game study’s Endurance dimension captured the organizational learning and adaptive problem-solving cycle and process. If companies were to be successful they would have to sustain the requisite amount of Depth and Variety over an extended period of time. Simply put, it was hypothesized that the more-successful companies would express more Depth and Variety over a longer period of time. VARIETY ENDURANCE Exhibit 5 indicates that the average number of screen visits by each performance group fell over time. This would be a natural event because a company’s players begin to learn which screens are important to them, and many of their large, complicated strategic decisions have been made early in the game. Exhibit 6 shows that, although both groups made fewer visits to their decision- making screens, the low-tier group made significantly fewer visits in periods one and two both significant p=0.04 level in a one-tail test. The large-game study next used a split-period test which was conducted to see if the emphasis on the particular Variety decision-making areas changed over time. It could be reasoned that the areas dealing with beginning finances and factory capacity would be most- often visited during the game’s early periods but also that a more-comprehensive view should be taken to make sure all areas and eventualities were covered. In a company’s later rounds there should be an overall decline in visits but the range of decision areas should remain at relatively high levels due the firm’s broader outlook on affairs. For this study, the game’s first two and last two periods were used to capture the evolutionary process detected by the work of Cangelosi & Dill (1965) which investigated the learning/behavioral changes associated with the large-scale Carnegie Tech Management Game (Cohen, et. al., 1964). They found in the team they tracked that its early periods could be characterized as being Intuitional and Searching phases. The team’s last periods were its Consolidating phase. During this ending phase, no major decisions were made and, because of its high profitability, had the capital retire short-term debt, call outstanding Bonds and issue Dividends. An ANOVA indicated, that when the time dimension of Endurance was added to the analysis, there were overall significant differences between beginning and ending period decision area visits. The results of these tests are presented in a comparative fashion in Exhibit 7. All differences between groups, either between their early vs. late periods, or high tier vs. low tier performance levels were significant at least p <0.04. If one follows Aristotle’s quotation “Well begun is half done”, an examination of the decision area concentrations in the game’s first two periods is revealing. The high performing companies showed greater Variety regarding their visits to the available decision making areas. They made almost four times as many visits to company finances and almost twice as many to accounting and cash flow Exhibit 6 Average Visits *Significant p=0.04 Period Tier High Low 1 111 34* 2 36 9* 3 37 36 4 17 8 5 16 4 6 14 13 Average 38 17 Page 265 - Developments in Business Simulation and Experiential Learning, volume 40, 2013 matters. The low performing companies were significantly more concentrated in fewer decision-making areas as indicated by their average Gini Coefficient of 0.78 versus a significantly lower (p = 0.0007) coefficient of 0.59 for the high performing firms. The low performing firms early-on concentrated on marketing and factory operations decisions to the exclusion of visits to their firm’s other areas. This comparatively high degree of low Variety by the low performing firms was carried through the rest of their decisions. At the game’s end, their Gini Coefficient was 0.76 versus a 0.63 coefficient for the high performing teams. This difference was significant p = 0.007. DEPTH ENDURANCE The previous two tests found the amount of Variety exercised by the two groups changed over time but that their similar ending trajectories had both groups practicing the same amount of Variety. Exhibit 8 displays the trends in the average amount of screen time expended by the average player over time. Both groups spent less Depth over time but with the high tier group spending significantly less Depth in the game’s second period. RESULTS SUMMARY This section summarizes the study’s results by hypothesis before being discussed in the next section. H1: High tier companies will demonstrate more Variety in their decision-making process­es than will low tier companies. This hypothesis was supported and agrees with the findings produced in the large-scale game study. The high performing companies visited almost three times more screens than did the low performing companies. The high tier companies also visited a wider array of decision making areas. H2: High tier companies will demonstrate more Depth in their decision-making processes than will low tier companies. Exhibit 7 Decision Area Endurance Difference by Performance Group Decision Area Low Tier High Tier Low Tier Early High Tier Early Low Tier Late High Tier Late Early Late Early Late Costs, GDP Forecasts and Bulletin Board 0.8% 0.0% 1.6% 0.6% 0.8% 1.6% 0.0% 0.6% Currency Exchange Rates 0.4% 0.0% 1.0% 0.3% 0.4% 1.0% 0.0% 0.6% Firm Summaries 0.4% 1.0% 1.6% 0.9% 0.4% 1.6% 1.0% 0.9% Performance Indicators 0.2% 1.5% 1.3% 2.4% 0.2% 1.3% 1.5% 2.4% Firm Operations Report 1.6% 8.2% 3.2% 3.7% 1.6% 3.2% 8.2% 3.7% Firm Income Statement 1.6% 7.7% 3.8% 10.1% 1.6% 3.8% 7.7% 10.1% Cash Flow 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% Firm Balance Sheet 0.2% 1.0% 1.3% 1.5% 0.2% 1.3% 1.0% 1.5% Merlin Studies 1.2% 1.5% 1.0% 1.8% 1.2% 1.0% 1.5% 1.8% Marketing Operations 14.1% 23.6% 12.7% 9.8% 14.1% 12.7% 23.6% 9.8% Marketing/ Sales Promotion 16.0% 10.3% 6.6% 13.1% 16.0% 6.6% 10.3% 13.1% Production/ Factory Capacity 31.6% 4.6% 18.1% 8.8% 31.6% 18.1% 4.6% 8.8% Production/ Factory Operations 14.1% 6.7% 16.0% 19.5% 14.1% 16.0% 6.7% 19.5% Production Subassembly Kits 8.0% 5.1% 10.4% 9.5% 8.0% 10.4% 5.1% 9.5% Finance/ Stock 2.3% 14.4% 6.4% 2.7% 2.3% 6.4% 14.4% 2.7% Finance/ Bonds 0.4% 2.1% 4.6% 0.6% 0.4% 4.6% 2.1% 0.6% Pro Forma Results 3.9% 7.7% 5.8% 10.1% 3.9% 5.8% 7.7% 10.1% Pro Forma Results/ Cash Flow Report 3.3% 4.6% 4.8% 4.6% 3.3% 4.8% 4.6% 4.6% Pro Forma Results/ Income Statement 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% Page 266 - Developments in Business Simulation and Experiential Learning, volume 40, 2013 This hypothesis was rejected at the summary company level. Companies in each industry spent a statistically similar amount of total time. This hypothesis was also rejected on a per player basis. The average player on the low tier companies spent more time on their screens than did those on high performing teams. Additionally, there was no difference between the groups regarding total time spent on company decision-making areas. H3: High tier companies will demonstrate more Endurance in their decision-making pro­cesses of Variety and Depth than will low tier companies. Both groups spent less Depth and Variety over time. This hypothesis was accepted, however, as the low tier companies exercised less Variety during the game’s early periods. The Endurance, or time dimension also revealed that the low performing companies always demonstrated less Variety regarding the range of decisions viewed, and the sharing of decisions made within each team at both their company’s early and late periods. The amount of Depth displayed by all companies was the same throughout the game. Therefore, this hypothesis was accepted regarding Variety but rejected for Depth. DISCUSSION This study used polar opposites to test its hypotheses. This was done to put the issues in sharp relief. This method, however, did not recognize that a wide array of performance results can occur in a game. This study method looked at only “winners” and “losers” without examining those who may also have engaged in the same decision making processes but led them to second, third and fourth place finishes. Research should be done on companies that finish in the relatively in-between rankings in their industries. This study sought to determine if a small-scale, total enterprise game afforded its players the same problem- solving process dimensions associated with a large-scale game. In doing so, and for the hypotheses used here, it was assumed the results obtained by the large-scale game were correct, and that those results should be the standard of comparison. It is possible the large-scale game’s results were an aberration, or that in-between sized games should actually set the standard. Research should be engaged in that tests the nature of the problem-solving environment created by such games as CAPSTONE (n.d.), Marketplace Live (n.d.) and The Business Strategy Game (n.d.). This study’s overall results may or may not bring comfort to those who use business games. Generally observing, those companies that “did the right thing”, in both strategic and tactical matters, and were dedicated enough to design and implement their strategies and tactics over the long haul, did relatively well. The observations of what players did, however, was done via the Rubik’s Cube problem solving process. Other problem-solving schema’s should be employed to determine if confirming or contradictory results accompany the use of different schema. This paper, however, found its Rubik’s cube Exhibit 8 Screen Minutes Per Player Over Time Page 267 - Developments in Business Simulation and Experiential Learning, volume 40, 2013 schema has an analogue schema associated with medical laboratory research. Barry (2004) noted that significant research in the fields of medicine, mathematics, chemistry and physics, come from experimenters who were able to think horizontally (engaged in Variety), vertically (engaged in Depth) and were compulsive (engaged in Endurance) about what they were attempting to discover. It should be noted that this study’s only data-gathering device was the game’s online-generated electronic logs. It is possible that players spent most of their time, or their more-productive time, engaged in face-to-face meetings. They could have also communicated with each other via telephone, direct e-mails and BlackBerrys. The virtue of this study’s method, nonetheless, is that it detected exactly who was doing what, although the method could not detect the mental processes going on which the player viewed and moussed through the available screens. On the other hand, no player ever used the game’s blogging feature to either chat or pass game-related material with each other, none ever printed out any of the game’s screen materials, made hard-copies of outputs, or exported those results to spreadsheets which could be shared with each other. Further research should also be conducted using other games of different sizes and complexity levels. Games simpler than the one used in this study could be just as useful for assurance of learning purposes. Further research should also be conducted using other measures of player engagement in the learning process. User diaries, extended debriefings and recorded sessions, could be useful methods although all are subject to either the Hawthorne Effect or after-the-fact rationalizations. CONCLUSION Partial, but sometimes conflicting results were found in this study when its results were compared to those produced by a prior large-scale game. These refutations of the large-scale game’s results could have be caused by using the large-scale game as the standard of comparison instead of using some other less-ambitious game. Additionally the lack of size and scale, or its opposite smallness and simplicity, could have a power of its own. Based on this study’s results, however, it is believed that a more-ambitious, and thus more-challenging game, is the proper vehicle for engaging in an AoL effort. REFERENCES AACSB (2007). AACSB Assurance of Learning Standards: An Interpretation. Retrieved from www.aacsb.edu/ publications/whitepapers/AACSB_Assurance of_Learning.pdf. AACSB (2011). Eligibility Procedures and Accreditation Standards for Business Accreditation. Tampa, FL: AACSB International. AACSB (2012). Learning goals examples. http:// www.aacsb.edu/accreditation/business/ standards/aol/ learning, goals examples.asp. Accessed September 29, 2012. Anderson, P.H., & Lawton, L. (2004). The effectiveness of a simulation exercise for integrating problem-based learning in management education. In Developments in Business Simulation and Experiential Learning, 32: 10 -18. (Reprinted from Bernie Keys Library (12th ed.)). Anderson, P.H. & Lawton, L. (2009). Business s simulations & cognitive learning: Developments, desires, and future directions. Simulation & Gaming, 40(1): 193-216. Barry, J.M. (2004). The great influenza: The epic story of the deadliest plague in history. NY: Viking. Biggs, W.D., & Fritzsche, D.J. (2010). Using Accumulated Profits to Assess Performance in Simulations. Developments in Business Simulation and Experiential Learning, (pp. 183-189). Reprinted from Bernie Keys Library (10th ed.)). Bisoux, T. (2008). Measures of success. BizED, (March/ April): 22-30. Cangelosi, V.E., & Dill, W.R. (1965). Organizational learning: Observations toward a theory. Administrative Science Quarterly, 10(2): 175-203. Cannon, H.M. (1995). Dealing with the complexity paradox in business simulation. In Developments in business simulation and experiential exercises, 22: 96- 102. (Reprinted form Bernie Keys Library (12th ed.)). CAPSTONE (n.d.). Available from http:// www.capsim.com. Choo, C.W. (2001). Environmental scanning as information seeking and organizational learning. Information Research, 7(1):1-14 Cohen, K.J., Cyert, R.M., Dill, W.R., Kuehn, A.A., Miller, M.H., Van Wormer, T.A., & Winters, P.R. (1964). The Carnegie Tech management game: An experiment in business education. Homewood IL: Richard D. Irwin, Inc. Cohen, K.J., Cyert, R.M., Dill, W.R., Kuehn, A.A., Miller, M.H., Van Wormer, T.A., & Winters, P.R. (1960). The Carnegie Tech management game. The Journal of Business, 33(4): 303-321. Faria, A.J., & Nulsen, R. (1996). Business simulation games: Current usage levels: A ten-year update. In Developments in business simulation and experiential exercises, 23: 22-28. (Reprinted from Bernie Keys Library (12th ed.)). Feinstein, A.H., & Cannon, H. (2002). Constructs of simulation evaluation. Simulation & Gaming, 33(4): 425-440. Frazer, J.R. (1983). A deceptively simple business strategy game. In Developments in Business Simulation and Experiential Learning, 10: 98-100. (Reprinted from Bernie Keys Library (12th ed.)). Graham, R.G., & Gray, C.F. (1969). Business games handbook. New York: American Management Association, Inc. Green, D., & Faria, A J. (1995). Are good strategies consistently good? In Developments in Business Simulation and Experiential Learning, 22: 31-37. (Reprinted from Bernie Keys Library (12th ed.)). Harrigan, K.R. (1988). Managing maturing businesses: Restructuring declining industries and revitalizing troubled operations. Lexington MA: D. C. Heath & Company. http://www.aacsb.edu/publications/whitepapers/AACSB_Assurance%20of_Learning.pdf http://www.aacsb.edu/publications/whitepapers/AACSB_Assurance%20of_Learning.pdf http://www.aacsb.edu/publications/whitepapers/AACSB_Assurance%20of_Learning.pdf http://www.aacsb.edu/accreditation/business/ http://www.aacsb.edu/accreditation/business/ http://ir.stthomas.edu/ocbmgmtpub/9/ http://ir.stthomas.edu/ocbmgmtpub/9/ http://ir.stthomas.edu/ocbmgmtpub/9/ http://www.capsim.com http://www.capsim.com Page 268 - Developments in Business Simulation and Experiential Learning, volume 40, 2013 Harrigan, K.R., & Porter, M.E. (1983). End-game strategies for declining industries. Harvard Business Review 64 (4): 111-120. Hamermesh, R.G., & Silk, S.B. (1979). How to compete in stagnant industries. Harvard Business Review, 57(5): 161-168. Keys, J.B., & Wolfe, J. (1990). The role of management games and simulations in education and research. Journal of Management, 16(2): 307-336 Latane, B., Williams, K., & Harkins, S. (1979). Many Hands Make Light The Work: The Causes and Consequences of Social Loafing, Journal of Personality and Social Psychology, 37(6): 822-832. Lohman, M.C. (2002). Cultivating Problem-Solving Skills through Problem-Based Approaches to Professional Development. Human Resource Development Quarterly, 13(3): 243-261. Marketplace Live (n.d.). Available from http:// www.marketplace-live.com. Martell, K., & Calderon, T. (2005), Assessment of student learning in business schools: Best practices each step of the way, 1: 167-183. Tallahassee, FL: Association for Institutional Research. Neuhauser, J.J. (1976). Business games have failed. Academy of Management Review, 1(4): 124-129. Old Dominion University. AACSB Maintenance Report. http://bpa.odu.edu/bpa/AACSB-Maintenance-Reports/ Consolidated_Assessment_Plans_and_Results_For_SP 06_FA06.pdf. Accessed September 29, 2012. Teach, R., & Murff, E.J.T. (2008). Are the simulations we play too complex? In Developments in Business Simulation and Experiential Learning, 35: 205-211. (Reprinted from Bernie Keys Library (12th ed.)). Teach, R., & Murff, E.J.T. (2009). Learning inhibitor in business simulations and games. In Developments in Business Simulation and Experiential Learning, 36: 191-197. (Reprinted from Bernie Keys Library (12th ed.)).. The business strategy game (n.d.). Available from www.bsg-online.com. The global business game: World edition (n.d.). Available from www.onlinegbg.com. The global business game: Business basics edition (n.d.). Available from www.onlinegbg.com. Wolfe, J. (In press). Large-scale business games for assurance of learning purposes. In Developments in Business Simulation and Experiential Learning. (To be reprinted in Bernie Keys Library (13th ed.)). Wolfe, J. (In press). A large-game test of early-determined game finishes. To be reprinted in the (Bernie Keys Library (13th ed.)) Wolfe, J. & Chanin, M. (1993). The integration of functional and strategic management skills in a business game learning environment. Simulation & Gaming, 24(1): 34-46. Wolfe, J, Biggs, W.D., & Gold, S.C. (In press). A replication and expansion of Teach and Patel’s discovery of early-determined endgame finishes. Simulation & Gaming. http://faculty.babson.edu/krollag/org_site/soc_psych/latane_soc_loaf.html http://faculty.babson.edu/krollag/org_site/soc_psych/latane_soc_loaf.html http://faculty.babson.edu/krollag/org_site/soc_psych/latane_soc_loaf.html http://faculty.babson.edu/krollag/org_site/soc_psych/latane_soc_loaf.html http://www.marketplace-live.com http://www.marketplace-live.com http://bpa.odu.edu/bpa/AACSB-Maintenance-Reports/Consolidated_Assessment_Plans_and_Results_For_SP06_FA06.pdf http://bpa.odu.edu/bpa/AACSB-Maintenance-Reports/Consolidated_Assessment_Plans_and_Results_For_SP06_FA06.pdf http://bpa.odu.edu/bpa/AACSB-Maintenance-Reports/Consolidated_Assessment_Plans_and_Results_For_SP06_FA06.pdf http://www.bsg-online.com http://www.onlinegbg.com http://www.onlinegbg.com Table of Contents Volume 40, 2013 Leo Investor SERIOUS PLAY: ON-LINE PEPULATOR SERIOUS PLAY:ON-LINE GAME "MANAGE-ART" The Game Within the Simulation Game the Research Method Concept and Project With Game Design Implementation Managing Human Resources Simulation Rising Stars: A Study of Creativity as Experiental Learning in County Government Why do Gen Y Students Study Abroad? The Relationship Between Individual Growth and the Intent to Study Abroad Our Favorite Assignment: The In-Basket Activity Mirror-Mirror: How Can We Better Understand Our Consumption Behavior? Reflective Thinking, Writing and Imagery as Tools to Better Understand Abstract Marketing Concepts Increasing Intuitive Decision Making Speed and Accuracy by Further Understanding Intuitive Decision Making Using Emotional Means Group Exams: Are they Relevant and Reliable as a learning Tool? Integrative Learning: Exploring Opportunities in Business Simulations Simulated Tabletop Exercise for Risk Management - Anti Bio-Terrorism Multi Scenario Simulated Tabletop Exercise Simulated Tabletop Exercise for Electric Power Saving Management in a Small Organization Merging the Case Method and Simulation in Management Education: Is it Possible? Experiential Learning in Accelerated Human Resource Management Courses Digital Game Building as Assessment: A Study of Secondary Students' Experience Using an Investment Fund Simulation Integrated with a Business Game Creating Simulation Conditions Based On Real World Data Simulation Games in Training New Management Methods A Summary Overview of Cultural Differences in Higher Education Innovations And Future Directions In Education: Case Review For Best Online Teaching Practices An Implicit Measure of Forecasting Accuracy Immediate Feedback and Assessment Technique (IF-AT) testing forms: An overview of the tool and uses Teaching Secondary Mathematics: Pre-service Teachers' Digital Game Design, Pedagogy and 21st Century Skills Follow The Leader II Business Simulations: From Punch Cards to Web-Based University Engagement: Good Neighbors Budget Allocation Exercise Customizing Business Simulations: An Exploration and a Hierarchy A Review of the Simulation Research in the Academy of Management Journal: Suggestions for Strengthening the Research Conducted by ABSEL Members ABSEL Reflections: 40 Years Of Excellence, Now Going Forward How Many Options do Multiple-Choice Questions Really Have? The Management/Accounting Simulation Now 100% Web-based Design, Server Side Programming, and Implementation A Large-Scale Game Test of Early-Determined Game Finishes Large-Scale Business Games for Assurance of Learning Purposes The Gamification Of Education OB Simulation: A Hands-On Demonstration Converting Simulations for the Online Environment: The New Ginseng Game Repositioning Brands with the Web-Based Product Positioning Map Graphics Package Implications of Regulatory Focus Theory for Simulation and Experiential Learning Riskware: A Game for Teaching Software Project Risk Management Teamwork for Decision-Making Through Games: The Case of On-Line "Manage-Art" Virtual Pepulator, a Model for Teaching Negotiation Small-Scale Business Games for Assurance of Learning Purposes Incorporating Intellectual Property Issues into a Business Simulation© Nationalization and Privatization in a Computer-Assisted Business Game An Investigation of the Relationship of Plan Quality, Forecast Accuracy and Earnings Performance Under Equal and Unequal Starting Positions in Marketing Positions in Marketing Simulations Measuring the Performance Ranking Curve in Marketing Simulation Games The Effectiveness of SDM Method in Business Simulation Game 40 years of Brazilian S&G - Analysis and Perspectives Accounting For Externalities: Harnessing The "Face-In-The-Mirror" Phenomenon The Role Of Simulations In Organizational Learning: Building Individual Absorptive Capacity Assessing Project Management as an Academic Learning Outcome (ALO) At the Inflection Point: Designing Army Assessments "Economics in Practice" A Simulation Game for High School Students Teaching the Basics of Economics and Entrepreneurship Fueling the Force: Exploring Leader Priorities Implementing Mental Models: Extending Insight and Whole Person Learning Using Business Simulations To Introduce Business Concepts