Can Action Complexity be Used to Measure the Effectiveness of an Educational Game? Page 216 - Developments in Business Simulation and Experiential Learning, Volume 43, 2016 ABSTRACT We distinguish between design complexity, which is an attribute of an exercise, and action complexity, which is the extent to which participants of an exercise act in a variety of ways. We propose that the action complexity of an exercise be measured by calculating the χ2 of the difference between the minimum- action and actual-action states of key decisions by the participants of the exercise. We suggest applications, and consider if action complexity might be used to measure the effectiveness of an educational game. We describe the design of our multinational business game, and find in a 235- undergraduate, one-semester administration of the game to two subpopulations that, as hypothesized, (a) the action complexity of participants’ nationality in the game corresponds with the participants’ personal performance scores at the midpoint and endpoint of the games’ duration, and (b) the same measure also corresponds with the relative personal performance scores of the two subpopulations. We conclude that action complexity may be a better measure of the effectiveness of an educational game than conventional surveys and test scores, and may also be better than the number of decisions per period, an established design complexity measure. INTRODUCTION Complexity is a common word in everyday use and in the literature of simulation, gaming, and experiential exercises. As with most common words, the meaning of the word varies with user and context. In an early exposition on the subject, Keys (1977, 1980) breaks down a computerized game’s complexity into two components: game variable complexity and computer- model complexity, and asserts that the best overall measure of complexity is the number of individual decision inputs per round of game play. Consistent with this exposition, Wolfe (1978) also sees two components to complexity, which he calls playing complexity and program complexity. For Wolfe, “playing complexity entailed the sophistication and intricacy faced by the player …, [whereas] program complexity entailed the relative size of a game’s program as determined by the number of executable statements” (p. 144). In later work, Wolfe (1990) associates a game’s complexity with “the number of discrete decisions made for each of the simulation’s iterations, the number of functions and sub- functions modeled in the game, and the degree of abstraction possessed by the concepts employed” (p. 280), but he also apparently agrees with others (Burns, Gentry, & Wolfe, 1990) that the measurement of complexity can be simplified to a single variable, namely “the number of decisions required per episode” (p. 269), a position in agreement with Keys (1977, 1980). More recently, Cannon, Friesen, Lawrence, and Feinstein (2009) maintain that complexity has heretofore been discussed only along the information-load dimension, ignoring the uncertainty dimension. Their idea, apparently, is that as more information is presented to players, players employ simplifying mechanisms to manage the information, so the information load that they actually experience does not invariably increase with the quantity of information that is presented to them. As we see it, the essence of Cannon et al.’s (2009) argument is that complexity should be defined in terms of the players’ experience rather than in terms of the game’s design, because learning is the purpose of the exercise, and what the players learn from the exercise must arise from their experience with the exercise. We agree with the argument, and would extend it one step further by arguing that what the players experience must give rise to actions if the experience is one of consequence. Accordingly, we distinguish between two kinds of complexity: design complexity and action complexity. Design complexity incorporates both playing complexity and program complexity, so it is synonymous with information-load complexity. Design complexity is an attribute of the exercise, irrespective of how players experience the exercise. On the other hand, action complexity is the extent to which participants of an exercise act in a variety of ways, so action complexity accounts, in an observable manner, for what players actually experience. EXAMPLE Consider, for example, a game that allows players to choose to produce one and only one of three quality levels (high, mid, or low) of a product. If three such games are each administered to players of a single population, then the results Can Action Complexity be Used to Measure the Effectiveness of an Educational Game? Precha Thavikulwat Towson University pthavikulwat@towson.edu Jimmy Chang The Hong Kong Polytechnic University jimmy.chang@polyu.edu.hk Bosco Wing Tong Yu The Hong Kong Polytechnic University wtongyu@speed-polyu.edu.hk mailto:pthavikulwat@towson.edu mailto:jimmy.chang@polyu.edu.hk mailto:wtongyu@speed-polyu.edu.hk Page 217 - Developments in Business Simulation and Experiential Learning, Volume 43, 2016 of administering the three games might be as follows: In Game A, all players opt to produce the high-quality product; in Game B, half of the players opt to produce the high-quality product and the other half opt to produce the mid-quality product; and in Game C, one third of the players opt to produce the high-quality product, one third opt to produce the mid-quality product, and one-third opt to produce the low-quality product. The players’ actions are least varied in Game A and most varied in Game C, so by our definition, we would rate Game A’s action complexity to be the lowest and Game C’s action complexity to be the highest, irrespective of differences in design complexity among the three games. Action complexity can be quantified in a straightforward and statistically advantageous manner by calculating the χ2 (Equation 1) of the difference between the minimum-action state and the actual-action state for the k options of the states. We define the minimum-action state as the state of the game when all players take the path of least resistance, which may be the default choice on a form, or the choice of the previous period if the game begins with a history, or the choice suggested by an administrator or decision-support system, among other possibilities. The minimum action state will usually contain zero-frequency options. So, to avoid division by zero in applying Equation 1, we assign fi to the minimum-action frequencies and Fi to the actual-action frequencies. For example, suppose that the minimum-action state for players of the three-game example is for the players to choose the high-quality product of all three games. After 18 players have each played all three games, we find that their choices are as shown in Table 1. In Game A, all players chose the minimum action state, so the χ2 action-complexity measure of Game A is zero. The χ2 Yates’s-corrected action-complexity measure of Game B at 9.48 is significantly higher than that of Game A, p = .002; and the χ2 action-complexity measure of Game C at 36.00 is even higher than that of Game B, χ2 (2) = 9, p = .011. APPLICATION Besides its utility for classifying and selecting games, action complexity can be used to pinpoint dead periods in a game and to identify periods when game activity is especially hectic. For example, if a game reaches its moderate action- complexity level in period 4, stays at the same action- complexity level through period 8, and advances to the high action-complexity level by period 10, we would surmise that periods 5 through 8 are dead periods whereas periods 9 and 10 are hectic periods. In this case, the game designer might smooth out the challenge of the game by raising the difficulty level of the dead periods and lowering the difficulty level of the hectic periods, or the game administrator may accomplish the same result by allowing players less time to work through the dead periods and more time to work through the hectic periods. Even so, the most intriguing question for us is the title of this paper: Can action complexity be used to measure the effectiveness of an educational game? As with all games, educational games can be effective in two ways, as instruments for assessing the capability of players and as tools for enhancing those capabilities (Anderson, Cannon, Malik, & Thavikulwat, 1998). In both cases, however, the utility of the proffered effectiveness measurement depends upon the correspondence between the effectiveness measurement and the players’ performance measurement. As action must precede outcome, the action-complexity measurement would seem to be a viable candidate. In the discussion that follows, we describe the design of a multinational business game and the issues it addresses, and we explain how we applied the χ2 action-complexity measurement to the participants’ choice of nations. We hypothesize that action-complexity will rise throughout the duration of the game in conformity with the rise in participants’ performance scores. We also hypothesize that action-complexity will be greater for the higher-performing sub-population than for the lower- performing sub-population. Finally, we present supportive results, note limitations, and consider implications. GAME DESIGN We are unable to locate the source of the astute observation that the principal purpose of higher education is to prepare students for their last job, not their first job. Their first job will be the job they acquire upon graduation; their last job will be the job they relinquish upon retirement. To the extent that their last job is similar to their first job, higher education fails them, because a college graduate’s first job is typically one that requires only health, character, and a good high-school . (1) TABLE 1 PLAYERS’ PRODUCT-CHOICES IN THREE GAMES DIFFERING IN ACTION COMPLEXITY Product quality Minimum Action Game A low complexity Game B mid complexity Game C high complexity High 18 18 9 6 Mid 0 0 9 6 Low 0 0 0 6 *χ2 0 9.48 36.00 df 0 1 2 p 1.000 .002 .000 *χ2 computed with reference to the minimum-action column, where the theoretical frequency is that of the result column. Page 218 - Developments in Business Simulation and Experiential Learning, Volume 43, 2016 education. For this reason, the primary focus of a business game used in higher education, especially one that is costly to develop and time-consuming to play, should be on the job the students may attain just before their retirement. Commonly, business games peg that job as the job of a business manager or chief executive. In the work presented here, we go higher. We peg the job as the job of an entrepreneur who may found several firms, employ executives of those firms, and merge, acquire, and divest of firms, all in an international setting of many nations with different products, production advantages, and national policies. Our game is GEO, a multinational business game that enables participants to experience decision-making at society’s highest level. Participants of GEO do not enter the game in the common fashion, as executives of a company. Rather, they enter as individuals, who may found firms for which they will employ executives, who also are participants in the game. Executives include the firm’s seller, buyer, and manager, as illustrated in the organizational chart of Figure 1. The seller sets the firm’s sales policy, the buyer sets the firms purchasing policy, and the manager handles banking operations and supervises both the seller and the buyer. Founders may employ any combination of themselves and others to the executive positions. The goal of participants in the game parallels the apparently common goal of all animal life, which is to extend the duration of one’s life. Participants extend that duration by consuming products that are produced by the firms of the game. These products have a defined utility values, such that products with higher utility values extend life more than products with lower utility values, but the extent to which increased consumption of utility values extends life diminishes with the quantity consumed in each period, which is the measure of time in the game. Figure 2 shows the industries of the game, together with the supply-chain relationship among the five nonbanking industries (service, material, energy, clothing, and food) and the utility values, in utils, of the product of each nonbanking industry. The game allows participants to found firms in any industry, including the banking industry. A banking firm has only one executive position, the manager’s position. A service firm has two executive positions, the manager’s and seller’s position. All other firms have three executive positions, the manager’s, buyer’s, and seller’s positions. The supporting software of many business games are web based, which requires a browser. GEO’s supporting software, however, is Internet-based (Pillutla, 2003). From the user’s standpoint, the primary difference between the two kinds of software is in program responsiveness. The web-based software is generally less responsive, because the software, working through the browser, does not access the computer’s operating system directly. In contrast, the Internet-based software is more responsive because it accesses the operating system directly. Better responsiveness means that users spend less time interacting with the software, allowing them to spend more time interacting with each other and thinking about what they should be doing. Applying the four-quadrant classification system of Crookall, Martin, Saunders, and Coote (1986), we classify our game as a computer-assisted game, rather than a computer- directed, computer-based, or computer-controlled game, because our game supports extensive participant-participant interaction and leaves participants in control of outcomes. To maximize participant control, we applied a constructivist approach to game design that includes simple rules, smart algorithm, and participant role-playing of key processes (Thavikulwat & Pillutla, 2010). Participants choose their nationality and the nationality of the firms that they found. The choice involves three issues prominent in courses on multinational management and international business, namely, competitive advantage, comparative advantage, and trade policy. COMPETITIVE ADVANTAGE Competitive advantage is the advantage that one firm has over another firm in the same industry and nation, which enables the more advantaged firm to be more profitable than the less advantaged firm. Notable sources of competitive advantage in the everyday-world are intellectual property, economy of scale (EOS), learning-curve effects, and management competency, issues that economists place in the category of new -trade theory, also known as strategic-trade theory (Brander & Spencer, 1985; Krugman, 1981; Lancaster, 1980). GEO omits intellectual property. All products produced by the firms of the game are commodities, without the protection of patents or brands. The other notable sources of competitive FIGURE 1 ORGANIZATIONAL CHART FIGURE 2 RELATIONSHIP AMONG INDUSTRIES Service (1 util) Material (4 utils) Energy (4 utils) Clothing (20 utils) Food (24 utils) Banking Page 219 - Developments in Business Simulation and Experiential Learning, Volume 43, 2016 advantage are incorporated into the game. EOS can be decreasing, constant, or increasing. Decreasing EOS means that the fixed cost per unit of output increases as more items are produced by the same firm. Constant and increasing EOS means that the fixed cost per unit of output stays unchanged and falls, respectively, under the same condition. The implication for an industry characterized by decreasing EOS is that the entrepreneur maximizes profit by founding many firms rather than by expanding the production of one firm. The implication is reversed for an industry characterized by increasing EOS. GEO models all three forms of EOS with step functions based on the size of each firm’s executive team. To keep the model simple, fixed costs in GEO consist only of the salaries and stock options paid to executives, and variable costs consist only of the inputs into the production process, as shown in Figure 2. The parameters of the step-function model are presented to the participants on command in a digital panel that has the form of a table, as shown in Figure 3. In Figure 3, the tabs are labeled with the names of the nations (Alpha-FT to Zeta-MT) and the columns are headed by the names of the nonbanking industries. The base capacity figures (rows 11 through 14) are the numbers of product units that a firm of the industry is capable of producing in its first period of production. These figures show (a) decreasing EOS in the service industry between the one-executive and two- executive staffing levels and in the material and energy industries from the one-executive staffing level to the three- executive level, (b) constant EOS in the clothing and food industries between the one-executive and two-executive staffing levels, and (c) increasing EOS in the clothing and food industries between the two-executive and three-executive staffing levels. To forestall monopolies, some production at the no-executive staffing level is allowed, so if limited competition gives rise to prices high enough for a firm to be a good investment at the no-executive level, then more firms should enter the industry, giving rise to more competition and lower prices. Service firms are characterized by decreasing EOS. So, if the players can each found two service firms, then two players should each found two firms. Each player minimizes fixed cost per service unit by being a manager of one firm and a seller of the other, for a combined two-player base production capacity of 2 × 2 × 20 = 80 service units. The inferior alternative is for each player to found one firm and to swap executive positions, such that each player is the manager of one firm and the seller of the other firm, for a combined two-player base production capacity of only 2 × 1 × 35 = 70 service units. These employment arrangements presume that executives can moonlight (i.e., a person is employed in two different positions in two different firms, such as in a manager’s position in one firm and a seller’s position in another firm), which the game allows, without double dipping (i.e., a person is employed in the same position, such as the manager’s position, in two firms), which the game disallows. Thus, the rational action in an industry characterized by decreasing EOS is for players to found many partially staffed firms, rather than fully staffing a smaller number of firms. Clothing and food firms are characterized by increasing EOS at the highest executive staffing levels. In this case if production at the no-executive level is too low to be worth the investment each player entering the industry should found only one firm. The players minimize fixed cost per food unit by swapping executive positions, such that each player is manager of one firm, buyer of a second, and seller of the third, for a three -player, three-firm combined base production capacity of 3 × 35 = 105 food units. This level of production cannot be exceeded by any other arrangement among three players that excludes firms without executives. Thus, the rational action in an industry characterized by increasing EOS is to found the fewest number of firms consistent with employing the greatest number of executives in each firm, rather than founding many partially staffed firms. GEO models learning-curve effects classically, based on a fixed experience coefficient (φ = .95), such that the effect of learning on production capacity rises, and the effect of learning FIGURE 3 TECHNOLOGY OF PRODUCTION Page 220 - Developments in Business Simulation and Experiential Learning, Volume 43, 2016 on resource utilization falls, by 1 - φ fraction of the previous period’s value for every doubling of production experience. Specifically, the learning-curve effect on production capacity is modeled by Equation 2, where x stands for production experience in number of items produced; q, for production capacity; and q0, for production capacity with no experience. The learning-curve effect on resource utilization is modeled by Equation 3, where x, again, stands for production experience in number of items produced; u, for resource utilization; and u0, for resource utilization with no experience. As illustrated by the graphs of the two equations in Figure 4, most of the advantage of learning is realized within 50 periods of production. The model’s implication for mergers and acquisitions is elaborated upon by Thavikulwat, Chang, and Stanford (2013). Avoiding models, GEO games management competency, which is to say that the executive team of the firms in our game consists of the participants that the owners of the firms employs. Thus, the participants, being people, have naturally varying competencies. COMPARATIVE ADVANTAGE Comparative advantage is the advantage that one nation has over another nation in producing the same product. GEO models comparative advantage by differences in base capacities, as shown in Table 2. The base capacities in the service, clothing, and food industries range from 5 units a period for a firm at the no-executive staffing level to 35 units for a firm at the maximum staffing level, irrespective of nation. Thus, no nation has a comparative advantage in producing service, clothing, and food products. Nations Alpha-FT, Chi- EP, and Eta-ST have a comparative advantage in producing material products and nations Beta-FT, Delta-EP, and Gamma- ST have a comparative advantage in producing energy products. These comparatively advantaged production capacities are bolded in the table. TRADE POLICY Trade policy in our game is confined to import tariffs and export subsidies, and divided into four categories: free trade, import substitution, export promotion, and strategic trade. Of these four, free trade is the policy of non-governmental intervention in trade. GEO operationalizes free trade by setting the import tariffs and export subsidies of all products to zero, as shown in Table 3. Import substitution is the policy of hindering imports. GEO operationalizes import substitution by setting 50% tariffs on all imported products. Export promotion is the policy of assisting exports. GEO operationalizes export promotion by setting 50% subsidies on all exported products. Strategic trade is the policy of giving overwhelming assistant to industries with the highest potential for long-term profitability, so that competitors would be unprofitable if they reside in a nation without the policy. GEO operationalizes strategic trade by setting 100% subsidies on clothing and food products, which have the highest consumer utility values and are characterized by increasing EOS at the highest executive staffing levels. A problem in operationalizing trade policy in a game is the necessity of assuring that the game is fair to all participants. Fairness is not assured if different participants are assigned to nations with different trade policies, because different trade policies are likely to be differentially advantageous. We resolve the fairness problem by assigning all participants to the import- substitution nation (Kappa-IS), and then allowing the participants to migrate to nations of their choosing. MEASUREMENT Complexity is an area-specific concept as well as a general concept. For example, one may refer specifically to the complexity of a total-enterprise game’s marketing area, or one . (2) . (3) FIGURE 4 LEARNING-CURVE EFFECTS ON PRODUCTION CAPACITY (A) AND RESOURCE PRODUCTIVITY (B) A B Page 221 - Developments in Business Simulation and Experiential Learning, Volume 43, 2016 may refer generally to the complexity of the total-enterprise game as whole. In the research reported herein, we investigate the action complexity of the nationality area of GEO. We measure action complexity at only two points, the midpoint and endpoint of the exercise, because we seek merely to prove the concept of action complexity as a measure of effectiveness, not to explore the limits of the concept. We address nationality at two levels, the personal level and the firm level. At the personal level, participants choose the nation of their citizenship; at the firm level, participants choose the resident nation of the firms that they found. The former choice is reversible, for participants are allowed to migrate up to four times by the end of the exercise. The latter choice is irreversible, but a regretted choice for a nonbanking firm can be mitigated, because a firm of one nonbanking industry can be converted to a firm of another nonbanking industry. Accordingly, the choice of nation, nominally area specific, has broad consequences in the game, so it also serves as the dominant measure of the game’s general complexity. RESEARCH HYPOTHESES For action complexity to be useful in measuring the effectiveness of a business game, the action-complexity value must move in conformity with performance scores over the duration of the game, and the action-complexity value must be greater for the higher-performing subpopulation. Accordingly, our hypotheses are as follows: H1: The action complexity values of participants’ nationality at the midpoint and endpoint of the exercise correspond with their personal performance scores at those two points. H2: The action complexity values of participants’ nationality between subpopulations correspond with the relative personal performance scores of the subpopulations. H3: The action complexity values of firms’ nationality at the midpoint and endpoint of the exercise correspond with the participants’ personal performance scores at those two points. H4: The action complexity values of firms’ nationality between subpopulations correspond with the relative personal performance scores of the subpopulations. METHOD We administered the game for one semester jointly to two undergraduate subpopulations: 135 Hong Kong (HK) students enrolled in a senior-level course on strategic management and 100 U.S. (US) students enrolled in a junior-level course on international business. The game advanced at an accelerating pace, from about two weeks to a period at the beginning of the semester to about six hours to a period by the end of the semester. The HK students began their semester two weeks earlier than the US students, so the game started earlier for them. The HK students ended the game four weeks earlier than TABLE 2 RANGE OF BASE PRODUCTION CAPACITIES OF FIRMS BY NATION Nation Service Material Energy Clothing Food Alpha-FT, Chi-EP, Eta-ST 5-35 15-60 5-35 5-35 5-35 Beta-FT, Delta-EP, Gamma-ST 5-35 5-35 15-60 5-35 5-35 Kappa-IS … Zeta-MT 5-35 5-35 5-35 5-35 5-35 TABLE 3 TARIFF AND SUBSIDY RATES OF TRADE POLICIES Free trade Import substitution Export promotion Strategic trade Import tariff Service 0% 50% 0% 0% Material 0% 50% 0% 0% Energy 0% 50% 0% 0% Clothing 0% 50% 0% 0% Food 0% 50% 0% 0% Export subsidy Service 0% 0% 50% 0% Material 0% 0% 50% 0% Energy 0% 0% 50% 0% Clothing 0% 0% 50% 100% Food 0% 0% 50% 100% Page 222 - Developments in Business Simulation and Experiential Learning, Volume 43, 2016 the US students, at period 85, to allow them to focus on completing a required senior project. Over the 85 periods in which the HK and US students were active, the game progressed through eight phases. The parametric conditions of the phases are tabulated in Table 4. We chose round numbers for the midpoint and endpoint of the exercise, Periods 40 and 80, respectively. Both subpopulations could work with each other to found and own shares in firms. The game’s program designates as controlling shareholder the shareholder who receives the majority of the share-based votes, each share entitling its owner to one vote. Unless a shareholder acts to change the assignment, the game’s program automatically casts the votes of new shareholders for the first director, the shareholder who most recently received the greatest number of votes. Thus, unless the votes are tied, the first director is the controlling shareholder, able to employ the firm’s manager and execute its general policies, such as those on dividend payments, industry, mergers and acquisitions, and share sales and buybacks. If the greatest number of votes is held by two or more shareholders, then no shareholder has control, so neither the manager’s employment nor the general policies of the firm can be changed. A firm is assigned to a subpopulation if the first director of the firm is a member of that subpopulation. Firms in the game can acquire shares in other firms, so if the first director is a firm rather than a participant, the firm is assigned to a subpopulation if the chain of first directors of parent firms leads to a first director who is a member of that subpopulation. Otherwise the firm is unassigned. Unassigned firms are controlled by neither subpopulation. We configured the game to enable shareholders to sell their shares in the digital stock market of the game at any time. Shares sold can be purchased by participants or firms, including the firm that issued the shares. The government of the nation in which the firm is resident assumes the role of market maker. It is represented by an algorithm that maintains a bid to buy all outstanding shares at slightly below the firm’s book value and to sell all shares it acquires on the same basis, but never below the price of zero in both cases. Thus, shareholders of bankrupt firms can always dispose of their shares at the price of zero or higher, which effectively limits their liability to the amount of their investment. Accordingly, unassigned firms are firms whose outstanding shares have all been either re-purchased by the issuing firm or purchased by the government. RESULTS The number of unassigned firms rose from none in Period 0, to 86 by Period 40, and to 215 by Period 80. By Period 80, the 235 participants had founded an average of 1.54 firms per person, although by then they owned only about 0.62 firms per person, which are the assigned firms. The number of participants and of assigned firms for both the HK and US subpopulations in Periods 0, 40, and 80 are tabulated in Table 5. The ratio of the number of assigned firms relative to the number of participants is significantly greater for the HK subpopulation in Period 0, HK = 0.921, US = 0.475, χ2 (1) = 8.67, p = .003, reflecting the fact that the HK students had more time to found firms before the game advanced to Period 1, inasmuch as their semester began earlier. The same ratios are almost identical for both subpopulations in Period 40, HK = 0.963, US = 0.970, χ2 (1) = 0.003, p = .957, indicating that participants of both subpopulation were equally active by about midpoint in the duration of the exercise. The same ratios are significantly different again in Period 80, HK = 0.170, US = 1.230, χ2 (1) = 62.79, p = .000, indicating that US participants were much more active than HK participants towards the end of the exercise. HK participants are required to successfully complete a final-year project unrelated to the game, so work on the project diverts their attention from the game, which explains their lower level of activity towards the end of the exercise. We should expect that lower activity would give rise to lower performance scores. It does. Figure 5 is a graph of the average personal performances of the two subpopulations. The divergence in average personal performance between the two subpopulations is consistently significant beginning in Period 44, t (232) = 2.06, p = .041, which is about midpoint in the duration of the exercise. Table 6 shows the nationalities of the entire population at Period 40 and Period 80. The minimum-action columns, which puts all participants in Kappa-IS, are the default states, inasmuch as all participants are assigned to Kappa-IS upon registration. Consistent with the accelerating average personal performance scores of all participants from the start of the exercise until Period 40, as evidenced by the convexity of the TABLE 4 PARAMETRIC CONDITIONS Condition Phase 0 1 2 3 4 5 6 7 Ending Period 0 4 7 34 51 66 81 97 Foreign investments No No Yes Yes Yes Yes Yes Yes Service industry Yes Yes Yes Yes Yes Yes Yes Yes Material industry No No Yes Yes Yes Yes Yes Yes Energy industry No No Yes Yes Yes Yes Yes Yes Clothing industry No No No Yes Yes Yes Yes Yes Food industry No No No Yes Yes Yes Yes Yes Banking industry No No No Yes Yes Yes Yes Yes Company founding limit 1 1 2 3 4 5 6 7 Ownership layer limit 0 1 1 1 1 1 5 5 Emigration limit 0 0 0 0 3 3 4 4 Max. deposit-loan term 0 0 0 0 0 0 0 9 Max. nation deposit/loan 0 0 0 $8,000 $8,000 $8,000 $8,000 $80,000 Page 223 - Developments in Business Simulation and Experiential Learning, Volume 43, 2016 entire population’s average personal performance curve in that region (Figure 5), the action complexity value of χ2 = 65.21 at Period 40 is significantly greater than the minimum-action state, whereas the action complexity value of χ2 = 99.70 at Period 80 shows only a tendency to be greater that the value at Period 40, χ2 (5) = 8.60, p = .126. The results support H1, the action complexity values of participants’ nationality at the midpoint and endpoint of the exercise correspond with their personal performance scores at those two points. Table 7 shows participant nationalities broken down by subpopulations. The finding of interest from the breakdown is that the action complexity value for the HK subpopulation barely changed between Period 40 and Period 80, whereas the same value for the US subpopulation changed significantly between the two periods, χ2 (1) = 3.95, p = .047, confirming that the HK subpopulation, unlike their US counterpart, had low activity between the two periods. The results support H2, the action complexity values of participants’ nationality between subpopulations correspond with the relative personal performance scores of the subpopulations. Table 8 shows the nationality of the assigned firms at Period 40 and Period 80. As with the action complexity values of participant nationality, the action complexity value of firm nationality at Period 40, χ2 = 29.38, is significantly greater than the minimum-action state. Moreover, the action complexity value of firm nationality at Period 80, χ2 = 107.76, is significantly greater than the value at Period 40, χ2 (6) = 183.04, p = .000. This result is not completely supportive of H3, the action complexity values of firms’ nationality at the midpoint and endpoint of the exercise correspond with the participants’ personal performance scores at those two points, because action complexity rises significantly after the midpoint despite the slowdown in the rise of personal performance scores that followed. Breaking down assignable firms by subpopulation, the action complexity of both the HK and US subpopulations increases significantly from Period 40 to Period 80 (Table 9); the action complexity of the US subpopulation is significantly higher than that of the HK subpopulation at Period 40, χ2 (1) = 18.97, p = .000; and the action complexity of the two subpopulations are not significantly different at Period 80, χ2 (1) = 0.34, p = .560. This result also is not completely supportive of H4, the action complexity values of firms’ nationality between subpopulations correspond with the relative personal performance scores of the subpopulations, because the action complexity of HK assigned firms is not significantly lower than that of US firms in Period 80 even though the personal performance scores of the HK subpopulation is significantly less in that period. CONCLUSION The results completely support H1 and H2, but do not completely support H3 and H4. So, the action complexity of participants’ nationality is a better measure of effectiveness in this game than the action complexity of firm’s nationality. We surmise that the action complexity of firm’s nationality did not fare as well because the HK participants who stayed active in the game past its midpoint attended to their activities in the game with no loss in dedication greater than that of the US participants past the midpoint. Thus, the HK participants who found the game interesting stayed interested despite their final project’s demand on their time. Other HK participants dropped out to focus on their final project. Yet, to conclude that one action complexity measure is better than another measure of the same kind is to raise the question as to how action complexity compares with conventional effectiveness measures of a different kind, such as surveys of participants’ observations and opinions, and pre- and TABLE 5 NUMBER OF PARTICIPANTS AND ASSIGNED FIRMS HK US Period 0 Period 40 Period 80 Peri- od 0 Peri- od 40 Peri- od 80 Participants 12 7 13 4 13 5 9 9 1 00 1 00 Assigned Firms 11 7 12 9 23 4 7 9 7 1 23 FIGURE 5 AVERAGE PERSONAL PERFORMANCES OF PARTICIPANTS Page 224 - Developments in Business Simulation and Experiential Learning, Volume 43, 2016 post-test scores. Conventional measures, however, have not fared well at all. After 11 studies undertaken over eight years using such measures, Gosen and Washbush (2004) found that “there were few easily interpretable results…. For the vast majority of predictor variables, relationships with learning were not significant” (p. 293). In contrast to Gosen and Washbush’s (2004) findings, Wolfe (1978) and Butler, Pray, and Strang (1979) found that learning is better enhanced with a game of moderate to high design complexity, and that the number of decisions per period (NDP) suffices as the measure of that complexity. So, the collective findings of these researchers suggest that NDP is a better measure of game effectiveness than the conventional measures of learning based on surveys and test scores. Even so, NDP is problematic in two ways. First, NDP is ambiguous when applied to a game where decisions are entered on a dynamic form rather than a static form. On a dynamic form, a decision can give rise to other decisions, such that the number of decisions that follows depends upon preceding decisions, so the number of decisions associated with a game depends on how the decisions are counted. Second, NDP does not account for how the game is actually played—default TABLE 6 NUMBER OF PARTICIPANTS BY NATION IN TWO PERIODS Nation Period 40 Period 80 Minimum action Actual action Minimum action Actual action Alpha-FT 0 16 0 23 Beta-FT 0 19 0 22 Chi-EP 0 6 0 13 Delta-EP 0 5 0 6 Eta-ST 0 2 0 1 Gamma-ST 0 3 0 5 Kappa-IS 234 183 235 165 Total 234 234 235 235 *χ2 (5) 65.21 99.70 p .000 .000 *χ2 computed with Eta-ST and Gamma-ST combined because of low numbers. The theoretical frequency is that of the actual-action column. TABLE 7 NUMBER OF PARTICIPANTS BY NATION AND SUBPOPULATION IN TWO PERIODS Nation HK US Period 40 Period 80 Period 40 Period 80 Alpha-FT 1 1 15 22 Beta-FT 1 1 18 21 Chi-EP 0 5 6 8 Delta-EP 0 0 5 6 Eta-ST 2 1 0 0 Gamma-ST 0 0 3 5 Kappa-IS 130 127 53 38 Total 134 135 100 100 *χ2 (1) 0.76 3.95 p .380 .047 *χ2 computed with Alpha-FT through Gamma-ST combined because of low numbers for the HK subpopula- tion and Yates’s correction. The theoretical frequency is that of the period-80 column. Page 225 - Developments in Business Simulation and Experiential Learning, Volume 43, 2016 TABLE 8 NUMBER OF ASSIGNED FIRMS BY NATION Nation Period 40 Period 80 Minimum action Actual action Minimum action Actual action Alpha-FT 0 5 0 16 Beta-FT 0 3 0 12 Chi-EP 0 3 0 10 Delta-EP 0 9 0 16 Eta-ST 0 1 0 1 Gamma-ST 0 5 0 7 Kappa-IS 226 200 146 84 Total 226 226 146 146 *χ2 (6) 29.38 107.76 p .000 .000 *χ2 computed with the theoretical frequency being that of the actual-action column. TABLE 9 NUMBER OF ASSIGNED FIRMS BY NATION AND SUBPOPULATION IN TWO PERIODS Nation HK US Period 40 Period 80 Period 40 Period 80 Alpha-FT 3 2 2 14 Beta-FT 0 2 3 10 Chi-EP 0 2 3 8 Delta-EP 0 2 9 14 Eta-ST 0 0 1 1 Gamma-ST 1 0 4 7 Kappa-IS 125 15 75 69 Total 129 23 97 123 *χ2 (1) 22.76 9.88 p .000 .000 *χ2 computed with Alpha-FT through Gamma-ST combined because of low numbers for the HK subpopulation, Yates’s correction applied. The theoretical frequency is that of the period-80 column. decisions and actual decisions have equal weight. Both problems are resolved by action complexity. Moreover, action complexity is more closely related to learning, for if an experience affects learning, then action must follow, for “every experience both takes up something from those which have gone before and modifies in some way the quality of those which come after” (Dewey, 1938, p. 27). Essentially, action complexity measures learning by quantifying action, the modification of the quality of the experience that follows the learning experience. Page 226 - Developments in Business Simulation and Experiential Learning, Volume 43, 2016 Anderson, P., Cannon, H. M., Malik, D., & Thavikulwat, P. (1998). Games as instruments of assessment: A framework for evaluation. In J. K. Butler, Jr., N. H. Leonard, and S. W. Morgan (Eds.), Developments in Business Simulation and Experiential Learning, 25, 31-37. Brander, J. A., & Spencer, B. (1985). Export subsidies and international market share rivalry. Journal of International Economics, 18, 83–100. Burns, A. C., Gentry, J. W., & Wolfe, J. (1990). A cornucopia of considerations in evaluating the effectiveness of experiential pedagogues. In J. W. Gentry (Ed.), Guide to business gaming and experiential learning (pp. 253-278). East Brunswick, NJ: Nichols/GP Publishing. [Available from http://www.absel.org] Butler, R. J., Pray, T. F., & Strang, D. R. (1979). An extension of Wolfe’s study of simulation game complexity. Decision Sciences, 10, 480-486. Cannon, H. M., Friesen, D. P., Lawrence, S. J., & Feinstein, A. H. (2009). The simplicity paradox: Another look at complexity in design of simulations and experiential exercises. Developments in Business Simulation and Experiential Learning, 36, 92-98. [Available from http:// www.absel.org] Crookall, D., Martin, A., Saunders, D., & Coote, A. (1986). Human and computer involvement in simulation. Simulation & Games, 17, 345-375. Dewey, J. (1938). Experience and education. New York, NY: Macmillan. GEO. [Developed by P. Thavikulwat] Towson, MD (601 Worcester Road, Towson, MD, USA). [Available from http://pages.towson.edu/precha/geo/] Gosen, J. & Washbush, J. (2004). A review of scholarship on assessing experiential learning effectiveness. Simulation & Gaming, 35, 270-293. Keys, J. B. (1977). Total enterprise games: Computerized. In R. E. Horn (ed.), The Guide to Simulations/Games for Education and Training. Cranford, NJ: didactic Systems, Inc. Keys, J. B. (1980). Total enterprise games: An evaluation. In R. E. Horn and A. Cleaves (eds.), The Guide to Simulations/ Games for Education and Training, 4th Edition (pp. 277- 286). Beverly Hills, CA: Sage. Krugman, P. R. (1981). Intraindustry specialization and the gains from trade. Journal of Political Economy, 89, 959- 973. Lancaster, K. (1980). Intra-industry trade under perfect monopolistic competition. Journal of International Economics, 10, 151-175. Pillutla, S. (2003). Creating a Web-based simulation gaming exercise using PERL and JavaScript. Simulation & Gaming, 34, 112-130. Thavikulwat, P., Chang, J., & Sanford, D. (2013). Mergers and acquisitions in a business game. Simulation & Gaming. 44, 706-731. Thavikulwat, P. & Pillutla, S. (2010). A constructivist approach to designing business simulations for strategic management. Simulation & Gaming. 41, 208-230. Wolfe, J. (1978). The effects of game complexity on the acquisition of business policy knowledge. Decision Sciences, 9, 143-155. Wolfe, J. (1990). The evaluation of computer-based business games: Methodology, findings, and future needs. In J. W. Gentry (Ed.), Guide to business gaming and experiential learning (pp. 279-300). East Brunswick, NJ: Nichols/GP Publishing. [Available from http://www.absel.org] REFERENCES http://www.absel.org http://www.absel.org http://www.absel.org http://pages.towson.edu/precha/geo/ http://www.absel.org