The Invalidity Of Profit=F(Market Share) PIMS Validation Of Marketing Games Developments in Business Simulation and Experiential Learning, Volume 33, 2006 THE INVALIDITY OF PROFIT=F(MARKET SHARE) PIMS VALIDATION OF MARKETING GAMES John R. Dickinson University of Windsor mexperiences@aol.com ABSTRACT Research that purports to validate simulations should itself be valid. A current stream of research puts forth as game validation criteria “laws” drawn from the Profit Impact of Marketing Strategies (PIMS) project. One PIMS-based “law” is that profit is a function of market share. The present study invalidates that PIMS profit=f(market share) criterion based on the PIMS project itself, published conceptual logical reservations against that relationship, published empirical studies against that relationship, and the invalidity in several vital respects of the methodology applying that relationship. INTRODUCTION That business simulation games be valid is sine qua non. Conceptually, Feinstein & Cannon (2002), Peters, Vissers, & Heijne (1998), and Stanislaw (1986) have presented general frameworks for simulation validation. Within those frameworks, many empirical studies have evaluated simulation games per se and the experience of participants in game competitions. Research that purports to validate simulations should itself be valid. An unfortunate–in that it is itself invalid– current stream of research has recently been perpetuated in Faria and Wellington (F&W, 2005), that article being a collection of three papers previously published in Association for Business Simulation and Experiential Learning (ABSEL) Proceedings. The stream of research puts forth as game validation criteria “laws” drawn from the Profit Impact of Market Strategy (PIMS) project. (The term “law” is used incautiously by Schoeffler [1977, p. 109; 1983, pp. 23-1, 23-4], as well as by Green and Faria [1995, p. 32] and F&W [2005, p. 262].) One PIMS-based “law” is that profit is a function of market share (F&W 2004). The case made here for invalidating the PIMS profit=f(market share) “validation” criterion derives from the PIMS project itself, published conceptual logical reservations against that relationship, published empirical studies against that relationship, and the invalidity in several vital respects of F&W’s (2004) methodology. INVALID PRECURSOR STUDY F&W (2004) cite a paper by Green and Faria (1995) that “...examined the results from a simulation competition with regard to another PIMS finding...strategies that are successful in one marketplace/economic environment will continue to be successful in a similar environment even if the firm’s competitors are changed.” (p. 333) Green and Faria’s (1995) “...results strongly supported the view [within the simulated competition utilized] that a fundamentally sound strategy remains a fundamentally sound strategy in a similar environment even if competitors are changes as suggested by the PIMS findings.” (F&W 2004, p. 333) On the face of it, the notion that the effectiveness of a marketing strategy is unaffected by competing strategies seems nonsensical. On abundant and diverse substantive bases, the notion is nonsensical. Twice stated by Green and Faria (1995) is the specific PIMS finding that, “The laws of the marketplace determine about 80 percent of the variance in business performance” (p. 32) and “PIMS findings have suggested that 80 percent of the variance in a company’s performance can be explained by its environment” (p. 34). And, “...the laws of the marketplace determine about 80% of the variance in business performance...” (F&W 2005, p. 262) The very origin of the oft-cited “80 percent” directly and explicitly contradicts, i.e., invalidates, this claimed premise. To wit: “The basic ideas underlying both The PIMS Program and the strategic planning models at the General Electric Company from which PIMS evolved were originally developed by Sidney Schoeffler.” (Buzzell and Gale 1987, p. xii) Here is a more complete quote of Schoeffler: “When we try to understand the variance between [very profitable and very unprofitable businesses], the laws of the marketplace account for up to 80 percent of that variance. This means that the characteristics of the served market, of the business itself, and of its competitors constitute about 80 percent of the reasons for success or failure...” (Schoeffler 1983, p. 23-4, italics added). Schoeffler’s “marketplace” does not exclude competitors, as interpreted by G&F; the PIMS marketplace expressly includes competitors in explaining the profitability of a business. On this basis and additional bases of: • at least 36 published empirical PIMS-based studies, • marketing theory, • market response models, • published empirical studies not based on PIMS, 232 mailto:mexperiences@aol.com Developments in Business Simulation and Experiential Learning, Volume 33, 2006 • published operationalizations of marketing strategy variables (including a majority of PIMS definitions), and • surveys of managerial practice, all incorporating competitor effects in explaining the effectiveness of marketing strategy, Green and Faria (1995) has been shown to be invalid (Dickinson 2003), an invalidation presumably known to F&W prior to the publication of F&W (2005). QUESTIONABLE VALIDITY OF PIMS, SPECIFICALLY MARKET SHARE The PIMS project has not been without its critics, both overall and with respect to the profit=f(market share) relationship specifically. Wensley (1982, p. 149) cites his working paper with Rumelt (1980) as indicating: “...there are very significant methodological problems associated with ascribing significance to individual variable coefficients in complex regression models such as the PAR model with accounting identify relationships, potential specification errors and, probably, measurement errors. In general, it would seem very risky to assume that the reliability of the statistical results could support any general optimizing approach without much further rigorous analysis.” Anderson and Paine (1978) raise 16 “observations” (i.e., criticisms) of PIMS. Their “contingency factors” observation is that, “Conclusions derived from the PIMS analysis may be misleading due to omission of certain contingency factors including inconsistencies across industries and neglect of relevant ranges for variables.” (1978, p. 609) With regard to this criticism, they cite four published sources (including Bloom and Kotler 1975) in stating, “The PIMS conclusions with regard to market share have been criticized most heavily because of their neglect of contingencies.” (1978, p. 609, italics added) Bloom and Kotler (1975) articulate what is essentially a diminishing marginal return from increasing market share. They argue that market share can increase beyond the point of maximum return on investment: “...we feel that an organization’s goal should not be to maximize market share, but rather to attain optimal market share...a company that has exceeded [its optimal market share] should seek to reduce its current share.” (p. 65) The simple validation criterion proposed by F&W does not recognize the potential suboptimality of ever-increasing market share. Woo (1984) contrasted 41 low-performing (pretax ROIs of less than 10%) market-share leaders with 71 high- performing (pretax return exceeding 40%) market-share leaders. She concluded, “Obviously, market-share leadership is far from a sure indicator of superior performance; market share has not given this group of businesses attractive returns.” (p. 53) Most damningly, Jacobson and Aaker (1985) conducted an extensive investigation of the ROI~market share relationship using PIMS data. “The estimate of the market share effect [on return on investment] is 21 standard deviations less than the commonly cited estimate...” (p. 20, bold added) “The results of this study point to the conclusion that the direct impact of market share on ROI is substantially less than commonly assumed and, in fact, relatively minor.” (p. 21) THE ILLOGIC OF PROFIT=f(MARKET SHARE) IN MARKETING GAMES Profit is not inherently related to market share. Very high sales, i.e., market share, may be obtained at very low profit or even very great loss. Is there logic, then, to explain a possible profit=f(market share) relationship? Echoing earlier explanations of why market share may be profitable (Buzzell and Gale 1987, Chapter 5; Buzzell, Gale, and Sultan 1975, p. 98), Jacobson and Aaker (1985) summarize, “Two causal explanations [that] are usually offered for the observed link between market share and profitability.” (p. 11) “First is the related effects of the experience curve and economies of scale.” (p. 11) “A second causal explanation is that large market share can create market power over and above the cost advantage achieved by experience/scale effects. Large share firms may be able to extract favorable concessions from channel members because of their size and importance in the market.” (p. 12) In addition to the above two types of causes, the several authors identify a third explanation: management quality. Management quality is an explanation not of a causal relationship between profit and market share, but an explanation of that relationship being spurious, with both ROI and market share being dependent on management quality. A given marketing game may not incorporate effects of the experience curve or economies of scale or provide for the exercising of market power, thus rendering any profit=f(market share) spurious, not logical. There would be no rationale for validating the game against what is a spurious relationship. SHOULD CAUSAL LOGIC BE BUILT INTO GAMES? Given the logic summarized by Jacobson and Aaker (1985), should experience, scale economies, and market power be built into marketing games? Not necessarily. The purpose of marketing games is not to exactly and 233 Developments in Business Simulation and Experiential Learning, Volume 33, 2006 234 completely emulate the real world. (This is a matter of varying degree across the entire genre of simulations.) The degree of complexity of a marketing game’s design primarily depends on the game’s intended purpose. For introductory educational purposes, relatively simple games are designed. For strategic marketing management, more complex games are designed. As to the real world, it is not the case that somehow experience and scale effects are ever-present there. “Although experience and scale effects have been observed literally thousands of times, they are neither universal nor automatic.” (Jacobson and Aaker, 1985, p. 11, citing Aaker 1984, Chapter 10) PROFIT=f(MARKET SHARE) IS A CONTINGENCY F&W put forth the profit=f(market share) relationship in a single guise. As addressed below, they go so far as to stipulate a bivariate correlation value of 0.50 as an indirectly (and invalidly) PIMS-derived benchmark for the strength of the relationship. F&W do not recognize that if the relationship exists in PIMS and the real world at all that it may exist in different forms and strengths. This universality is patently invalid and is expressly denied by three founders of PIMS: “When one examines the impact of the various explanatory variables on ROI, it quickly becomes apparent that the impact differs from group to group.” (Schoeffler 1978, p. 113) “The importance of share [in determining profitability] does vary considerably from one type of industry or market situation to another.” (Buzzell and Gale 1987, p. 96) EMPIRICAL DEMONSTRATIONS OF CONTINGENCY In several studies it has been empirically demonstrated that any profit=f(market share) relationship is a contingency; the relationship interacts with a great number of additional factors: “Market share interacts with the R & D level of the business.” (Schoeffler 1977, p. 116) “...this impact is far stronger for industrial products than it is for consumer products.” (Schoeffler 1978, p. 113) (Firms in the two games analyzed in F&W market consumer products.) “We have already seen it is more important in stable markets than in unstable markets.” (Buzzell and Gale 1987, p. 96) “So, we conclude that market share is more important in high-tech industries.” (Buzzell and Gale 1987, p. 97) “Market leadership has a greater payoff in industries that are not very investment intensive.” (Buzzell and Gale 1987, p. 97) Anderson and Zeithaml (1984) found market share to contribute more to (i.e., to explain greater variance in) ROI in the maturity stage than in the growth stage of the product life cycle and to not be significantly related in the decline stage. Appendix B of Buzzell and Gale (1987) presents numerous estimated regression models specifying just such contingencies. WITHIN GAME VARIABILITY Four competitions of the MME were mentioned earlier. In the MME companies may compete in any or all of for product-market segments, a large majority of participants usually competing in all four segments. Consider one of the four segments in one of the four competitions. For a given competition, a profit=f(market share) correlation may be calculated for each of the participating companies, the correlation being taken across the nine periods of competition. Suppose this yields 47 correlations, i.e., one for each company. The potential range of the correlations is -1.0 to +1.0 or 2. For one of the segment-competitions the actual range was 1.943. In fact, across all of the 16 segment-competitions the smallest range was 1.26. Against the PIMS validation criterion, the assessed validity of the game would very much depend on which segments x companies x competitions were analyzed. SALES IN GAMES MAY NOT BE COMPARABLE TO SALES IN PIMS In one manner or another, the operationalization of market share is a function of sales. Sales is usually measured in currency, e.g., dollars, or in units sold. The number of units sold (and therefore the amount of dollar sales), however, may not be equal to demand. In marketing games where the decision mix includes inventory management, this inequality of sales and demand may be due to stockouts. Inventory is managed in many real-world companies, too, of course. However, there is no reason to believe that the frequency of stockouts and the extent of unsatisfied demand are comparable between a given simulation game and the multitude of firms involved with PIMS and seemingly no feasible way to assess that comparability. Stockouts are a material factor. Consider four competitions of The Marketing Management Experience (Dickinson 2002), all conducted in graduate-level introductory marketing classes. Across the competitions between 20.9 and 27.3 percent of approximately 6,000 Developments in Business Simulation and Experiential Learning, Volume 33, 2006 235 inventories stocked out. Likely game managers are not equally adept as real- world managers at managing inventory. Game participants are often students who may or may not have business experience, but are even less likely to have had inventory management experience. Even where game participants are managers, as with consulting or professional development programs or other training, there would still be a period of learning the game’s relevant parameters in contrast to PIMS managers who are very likely much further along the learning curve. (PIMS excluded businesses which were less than three years old. [Abell and Hammond, 1979, p. 291]) In short, stockouts almost certainly affect game sales (and profits) differently than PIMS sales (and profits). There seems to be no ready remedy for this incomparability. It is possible that demand-before-stockouts rather than sales-after-stockouts is knowable in a game. Real-world companies might know when they have stocked out, but only very rarely would they know the extent of demand unmet. That is, they do not know by how much they stocked out. Thus, measuring market share using demand-before-stockouts in a game has no counterpart in the real world. INFEASIBILITY OF ESTIMATING PIMS PROFIT=f(MARKET SHARE) If the ostensible profit=f(market share) PIMS relationship or any other PIMS finding is to be used as a validation criterion for marketing games it must be feasible to estimate the game relationship in a way comparable to the way in which it is estimated in PIMS. Otherwise, if the relationship does not exist in the game or if it does exist in the game but in a form different from PIMS then the difference between the game and PIMS may simply due to estimation method, namely specification error, rather than to true invalidity. “The most publicized use of the PIMS data is a regression which contains 37 independent variables...” (Anderson and Paine 1978, p. 602) The PIMS model is actually considerably more complex: “These 37 variables operate in a highly interactive way, being compounded into 58 separate cross-products or other combined terms.” (Schoeffler 1977, p. 111) “This profit level equation includes more than 60 terms composed of various combinations of 37 basic factors.” (Schoeffler, Buzzell, and Heany 1974, p. 140) Simply put, it is infeasible to the point of being inconceivable that any existing or foreseeably future marketing game would feature the same 37 variables and 58-plus terms as the basic PIMS model. In fact, by Strategic Planning Institute policy, the major PIMS models are proprietary: “Our operating rules specify that the major models may not be published, but must remain proprietary to the companies whose experiences are included in the data base.” (Schoeffler 1977, p. 115). The major PIMS models may never enter the public domain to enable games to be validated against them. INVALIDITY OF F&W AND OTHER BIVARIATE ANALYSES F&W’s use of the simple Pearson correlation to estimate profit=f(market share) is an extreme example illustrating the well-known specification error described earlier. F&W’s Pearson correlation analysis is statistically equivalent to a simple bivariate regression model. Thus, F&W estimate the equivalent of a regression model containing one independent variable, not 37 independent variables, and also not containing any of the additional 21- plus interaction terms. Other such bivariate “analyses” have been depicted in the form of 3 x 3 crosstabulations (Abell and Hammond 1979; Schoeffler 1977). However, Anderson and Paine (1978) correctly dismiss these depictions (and, thus, the analyses of F&W) with their eighth observation: “Analysis of independent variables in the absence of remaining model variables may lead to erroneous conclusions primarily due to problems of multicollinearity.” (p. 605) And “The ROI- explaining variables are highly multicollinear.” (Schoeffler 1977, p. 112) The depicted two-variable crosstabulations are just as invalid as the simple correlations of F&W. AN INVALID POPULATION PARAMETER AND AN INVALID STATISTICAL TEST F&W (2004, p. 334) test the following hypothesis: H1: Market share and company earnings will be strongly and positively correlated (Pearson’s r > .5) in a simulation game competition. The invalidity of bivariate, i.e., Pearson’s correlation, analysis has been established earlier. Whence comes the 0.5 correlation population parameter? In the PIMS context, F&W (2004, p. 334) state “...from the assertion by Buzzell and Gale (1987, p. 8) that market share and profitability are strongly correlated...” F&W (2004, p. 334) quantify “strongly correlated” as .5 by quoting “...Cohen and Cohen (1983, p. 61) who state that Pearson’s r values of .50 or more are considered ‘strong effect sizes”...” The logical progression, then, is that (1) Buzzell and Gale use the word “strongly,’ (2) Cohen and Cohen characterized Pearson correlation values of 0.5 or more as strong, (3) therefore by “strongly” Buzzell and Gale meant a Pearson correlation of 0.5. Closer examination of the statement by Buzzell and Gale vis-a-vis the statement by Cohen and Cohen reveals there is no support for the specification of 0.50 as a population parameter or as a benchmark for simulation game validation. In two fatal respects, it is clear that the use of the word “strongly” by Buzzell and Gale (1987, p. 8) is not at all in the context of Cohen and Cohen. Continuing to quote from the same Buzzell and Gale paragraph, “Market share and profitability are strongly related...But when we take quality as well as some 20 other Developments in Business Simulation and Experiential Learning, Volume 33, 2006 236 market factors into account, market share still has a strong impact on profitability.” (1987, p. 8, italics added) Buzzell and Gale use the descriptors “strongly” and “strong” in the context of a 20-plus variable multivariate analysis. Page 61, the page cited by F&F, in Cohen and Cohen (1983) falls in their chapter titled, “Bivariate Correlation and Regression” (pp. 25-78, italics added) and their page 61 presentation, indeed, applies to the bivariate Pearson correlation. By incomplete and inaccurate reference to Buzzell and Gale (1987) and Cohen and Cohen (1983) and by invalid logic, F&W put forth a population parameter correlation value of 0.5 when there is no apparent basis for that value. A second fatal flaw in the invalid logical progression of F&W is that Cohen and Cohen (1983), in fact, do not use the word “strong” at all to describe bivariate correlation values of 0.50 or more; they use the word “large.” It is quite certain that Buzzell and Gale (1987) did not have in mind Cohen and Cohen (1983) when they employed the words “strongly” and “strong.” The point is not incidental. F&W prescribe a criterion against which marketing games might–even should–be validated. The profit=f(market share) relationship itself does not seem sufficiently valid to serve as a criterion. The illusion of rigor in specifying a bivariate correlation of 0.5 to achieve validity furthers what is a misdirection to game designers, users, and participants. INVALID HYPOTHESIS TEST The specific hypothesis posed by F&W (2004, p. 334) stipulates “Pearson’s r > .5” as the population correlation to be tested. F&W (2005, p. 268, note below Table 2) state that all eight of the correlation values are statistically significant with p-values less than 0.01. Yet the actual statistical test conducted by F&W via SPSS P.C. Version 10 (2005, p. 265) was not of a 0.5 population parameter, but of zero. All but one of the eight sample correlation values in Table 2 (2005, p. 268) are less than 0.5. Had F&W conducted the appropriate statistical test–a one-tailed test of a 0.5 population parameter–for seven of the eight tests p- values would have been greater than fifty percent. For seven of the eight tests, the theorized Pearson correlation of at least 0.5 would very conclusively have not been supported. INVALID LEVELS OF AGGREGATION F&W illustrate the application of the profit=f(market share) PIMS criterion using two simulation games: The Marketing Management Simulation (MMS) and Compete (Faria, Nulsen, and Roussos 1994). The MMS comprises four distinct product-market segments, each segment having a different sales-determining algorithm. Compete comprises three separate strategic business units. Yet for their analyses F&W combine data for the four MMS segments and combine data for the three Compete SBUs. This in itself is invalid vis-a-vis the PIMS definition of a business unit: “A business unit is defined as a division of the firm ‘selling a distinct set of products to an identifiable set of customers in competition with a well-defined set of competitors’.” (Jacobson and Aaker 1985, p. 13) And “...we say that each business unit in a company should have its own distinct, separate strategy...” (Buzzell and Gale 1987, p. 32) Even more grossly, for one analysis F&W combined data for two distinct simulation games (2005, p. 335, third column of correlations in Table 1). PIMS-INVALIDITY OF UNIT MARKET SHARE In F&W’s application of the profit=f(market share) to the MMS and Compete, for the former game market share was defined in terms of dollars. For the latter game, “...the unit of analysis for market share in the Compete simulation game was units sold.” (F&W 2005, p. 335) However, PIMS measures the size of the “served market” and, thus, market share, in dollar sales (Abell and Hammond 1979, p. 310) and not in terms of unit sales. F&W attempted to PIMS- validate the Compete game against a criterion–unit market share–that does not exist in PIMS. CONCLUSION There is no denying the ambition and prominence of the PIMS project. At the very least, it has provided numerous provocative conceptual perspectives and a wealth of published empirical studies. Loose nomenclature notwithstanding, though, there is nothing “law like” about the profit=f(market share) relationship. To disqualify the profit=f(market share) relationship as a criterion for simulation game validation it is not necessary to prove that the relationship does not exist. It is only necessary to call into well-founded question its being a “universal law” (Abell and Hammond 1979, p. 275). Abundant and diverse evidence conclusively renders the profit=f(market share) criterion anything but necessary for simulation game validity. That invalid studies purporting PIMS-based validation criteria continue to be published and past invalid published studies left unretracted has major dysfunctional implications. Game designers may plan games toward invalid criteria, users of games may adopt specific games on bases of invalid criteria, and participants in games may gain invalid experience. REFERENCES Aaker, David A. (1984). Strategic Marketing Management (New York: Wiley). Abell, Derek F. and Hammond, John S. (1979). Strategic Market Planning (Englewood Cliffs, NJ: Prentice-Hall, Inc.). Developments in Business Simulation and Experiential Learning, Volume 33, 2006 237 Anderson, Carl R. and Pain, Frank T. (1978). “PIMS: A Reexamination.” The Academy of Management Review, Volume 3, July, 602-612. Anderson, Carl R. and Zeithaml, Carl P. (1984). “Stage of the Product Life Cycle, Business Strategy, and Business Performance.” Academy of Management Journal, Volume 27, March, 5-24. Bloom, Paul N. and Kotler, Philip (1975). “Strategies for High Market-Share Companies.” Harvard Business Review, Volume 53, November-December, 63-72. Buzzell, Robert D., Gale, Bradley T., and Sultan, Ralph G. M. (1975). “Market Share–A Key to Profitability.” Harvard Business Review, Volume 53, January- February, 97-106. Dickinson, John R. (2003). “A Misuse of PIMS for the Validation of Marketing Management Simulation Games.” In Pillutla, Sharma and Feinstein, Andrew Hale (editors), Developments in Business Simulation and Experiential Learning, Volume 30 (Statesboro, GA: Association for Business Simulation and Experiential Learning), 99-106. Faria, A. J. and Wellington, William J. (2005). “Validating Business Gaming: Business Game Conformity with PIMS Findings.” Simulation & Gaming, Volume 36, June, 259-273. Faria, A. J. and Wellington, William J. (2004). “Validating Business Simulations: Does High Market Share Lead to High Profitability?” In Feinstein, Andrew Hale and Potosky, Denise (Eds.), Developments in Business Simulation and Experiential Learning, Volume 31 (Statesboro, GA: Association for Business Simulation and Experiential Learning), 332-336. Feinstein, Andrew Hale and Cannon, Hugh M. (2002). “Constructs of Simulation Evaluation.” Simulation & Gaming, Volume 33, December, 425-440. Green, Donna and Faria, A. J. (1995). “Are good strategies consistently good?” In Overby, John D. & Patz, Alan L. (Eds.), Developments in Business Simulation & Experiential Exercises, Volume 22 (Statesboro, GA: Association for Business Simulation and Experiential Learning), 31-37. Jacobson, Robert and Aaker, David A. (1985). “Is Market Share All that It’s Cracked Up to Be?” Journal of Marketing, Volume 49, Fall, 11-22. Peters, Vincent, Vissers, Geert, and Heijne, Gerton (1998). “The Validity of Games.” Simulation & Gaming, Volume 29, March, 20-30. Rumelt, R. P. and Wensley, J. Robin C. (1980). “In Search of the Market Share Effect.” UCLA Working Paper. Schoeffler, Sidney (1983). “The PIMS Program.” In Albert, Kenneth J. (editor), The Strategic Management Handbook (New York: McGraw-Hill Book Company), 23-1-23-10. Schoeffler, Sidney (1977). “Cross-Sectional Study of Strategy, Structure, and Performance: Aspects of the PIMS Program.” In Thorelli, Hans B. (editor), Strategy + Structure = Performance (Bloomington, IN: Indiana University Press), 108-121. Stanislaw, Harold (1986). “Tests of Computer Simulation Validity,” Simulation & Games, Volume 17, June, 173- 191. Wensley, Robin (1982). “PIMS and BCG: New Horizons or False Dawn?” Strategic Management Journal, Volume 3, April-June, 147-158. Woo, Carolyn Y. (1984). “Market-Share Leadership–Not Always so Good.” Harvard Business Review, Volume 62, January-February, 50-54. Woo, Carolyn Y. (1983). “Evaluation of the Strategies and Performance of Low ROI Market Share Leaders.” Strategic Management Journal, Volume 4, April-June, 123-135. Table of Contents Volume 33, 2006 Learning Assurance Using Business Simulations Applications To Executive Management Education Team Teaching In An Integrated Business Course Using Critical Problem Based Learning Factors In An Integrated Undergraduate Business Curriculum: A Business Course Success Personality Type And Strategic Planning Business Games As Strategic Management Laboratories The Relationship Between Students' Success On A Simulation Exercise And Their Perception Of Its Effectiveness As A PBL Problem Forecasting Accuracy And Learning: The Key To Measuring Simulation Performance The Business Strategy Game: A Performance Review Of The New Online Edition Using The Socratic Method And Bloom's Taxonomy Of The Cognitive Domain To Enhance Online Discussion, Critical Thinking, And Student Learning Using Negotiation Exercises To Promote Critical Thinking Skills Effective Leadership Experiences For Management Majors In A Futures Class The Role Of Learning Versus Performance Orientations When Reacting To Negative Outcomes In Simulation Games Is Pay Inversion Ethical? A Three-Part Exercise Simulations And Experiential Exercises - Do They Result In Learning? Have We Figured It Out Yet? Examining Program Management In Business Simulations: Student And Faculty Views Validating Business Simulations: Do Simulations Exhibit Natural Market Structures? Characterizing Business Games Used In Distance Education Utilizing Games In A Graduate Level Instructional Game Course Employment Interview Preparation: Assessing The Writing-To-Learn Approach Simulations - Bridging From Thwarted Innovation To Disruptive Technology Creating An Authentic Cultural Lens Using Case Dialogue Learning By Fire: Reflections Of A First Time Online Instructor An International Internship With A Service-Learning Focus Learner Participation In The Online Learning Experience: Help Or Hindrance? Any Given Sunday: Intervention In Pursuit Of Simulation Team Parity Beginning With The End: Creating An Experiential Exercise From Assessment Criteria Simulating Life Cycles: Life Span As The Measure Of Performance In Business Gaming Simulations It's Puzzling: Communications, Competition, And Cooperation Balanced Scorecard Implementation For Strategy Management: Variation Of Manager Opinion In Real And Simulated Companies Cases And Business Games: The Perfect Match! Three-Attribute Interrelationships For Industry-Level Demand Equations Using A Web-Based Module To Teach Information Literacy Decision Support System For Demand Forecasting In Business Games The Invalidity Of Profit=F(Market Share) PIMS Validation Of Marketing Games Online Market Test Laboratory With The MINSIM* Program The Gas Mileage Game - A Policy Simulation Delivered Cost And Differentiation Applied To Threshold 3rd Ed. The Effect Of Team-Leadership Modes On Team Performance: A Preliminary Study The Design And Use Of A Macroeconomics Simulation Using Maple Software: A Pilot Study The Instructor's Toolbox: A Meaning-Centered Framework For The Social Construction Of Experiential Learning Incorporating Strategic Product-Mix Decisions Into Simulation Games: Modeling The 'Profitable-Product Death Spiral' Group Composition And Groupthink In A Business Game A Direct Approach To Teaching Business Ethics A Decision Support System For Planning Sales, Production, And Plant Addition With Manager: A Computer Simulation Polish - American Entrepreneurial Business Cooperation Workshop Utilizing The Income/Outcome Simulation Student Leader Training Exercise Student Preference To Mode Of Learning In Hong Kong Experiential Learning For Technology-Based And Management Programme In Hong Kong: A China Study Tour A Price Game With Product Differentiation In The Classroom Discrete Event Modeling In A New Transportation Simulation Supply-Side Modeling In A Total Enterprise Simulation The Quality Game Towards A Massive Multiplayer Online Business Simulation Making The Connection: Improving Virtual Team Performance Through Behavioral Assessment Profiling And Behavioral Cues Individual Learning Producing A Learning Organization: 'Playing Dice With Polar Bears' Narratology and Ludology: Competing Paradigms or Complementary Theories in Simulation Framework For Evaluating Internet Research Using Children's Games To Illustrate Strategy Concepts: Is Less Better?