VALIDATING BUSINESS SIMULATIONS: DOES HIGH MARKET SHARE LEAD TO HIGH PROFITABILITY? Developments in Business Simulation and Experiential Learning, Volume 31, 2004 VALIDATING BUSINESS SIMULATIONS: DOES HIGH MARKET SHARE LEAD TO HIGH PROFITABILITY? A. J. Faria Odette School of Business Univeristy of Windsor ad9@uwindsor.ca William J. Wellington Odette School of Business Univeristy of Windsor r87@uwindsor.ca ABSTRACT Among the nine major strategic influences on profitability reported as part of the ongoing stream of PIMS research is that a business firm’s share of its served market has a strong positive impact on profit and net cash flow. This finding is based on an examination of the performance history of over 3,800 companies contributing data on a yearly basis to the Strategic Planning Institute. A major concern of simulation users through the years is how realistic are business simulation games. Numerous validation studies on business simulations have examined game validity. The market share and profitability levels of 440 simulation companies in 96 industries from two separate simulation games were examined and it was found that market share and profitability levels were significantly and positively correlated but the strength of relationship wasn’t as strong as suggested by the PIMS findings. INTRODUCTION It has now been over 45 years since the first use of a business simulation game in a university class in 1957 (Watson 1981). Since that time, the number of business simulation games and their use in university classes has grown enormously. Presently, over 200 business games are being used at over 1,700 universities and community colleges by approximately 11,000 business teachers in the U.S. alone (Faria 1998). Empirical research in the area has been extensive. Comprehensive reviews can be found in Greenlaw and Wyman (1973), Keys (1976), Wolfe (1985), Miles, Biggs and Shubert (1986) and Randel, Morris, Wetzel and Whitehill (1992). Despite the widespread use of business games, an ongoing issue of concern is whether or not participation in a simulation game is a meaningful experience. This paper introduces another measure that might be used for assessing the relative merit of business game participation that relates to the outcomes reported over the past forty years from the ongoing PIMS project as now administered by the Strategic Planning Institute. Specifically, this study examines the outcomes from two different simulation games to determine, as reported by PIMS, if high market share is positively related to high profitability. PAST RESEARCH Meaningfulness, as applied to the business simulation gaming experience, has taken on a number of interpretations as reflected in past research including: (1) the learning, or skills training, aspects of business games; (2) the relative merit of business games versus other teaching approaches; (3) the external validity of business simulation games; and (4) the internal validity of business games. Research into the skills training or learning aspects of business simulations dates back to the first uses of business games in university classes. The reported types of learning brought about by the use of business games include goal setting and information processing; organizational behavioral and personal interaction skills; sales forecasting; entrepreneurial skills; financial analysis; basic economic concepts; inventory management; mathematical modeling; personnel skills such as hiring, training, leading and motivating; creative skills; communication skills; data analysis; formal planning and report preparation; and much more. Faria (2001) provides a history and complete list of references covering skills training research. The merit of simulation games versus other teaching approaches has been investigated by a number of researchers (Greenlaw and Wyman 1973; Keys 1976; Snow 1976; Waggener 1979; Wolfe 1985; Miles, Biggs and Schubert 1986; Hall 1987; Spect and Sandline 1991; Washbush and Gosenpud 1991; Randle, Morris, Wetzel and Whitehill 1992; Wolfe 1997). Several comprehensive reviews, as cited earlier, have summarized the bulk of these comparative studies. Across all of the reported studies, simulation games were found to be more effective teaching tools, as measured by performance on course final exam performance, than conventional instructional methods (generally cases and lectures) in 75 of the comparisons, conventional methods of instruction were found to be superior in 332 Developments in Business Simulation and Experiential Learning, Volume 31, 2004 27 of the comparisons, while no differences were reported in 58 of the comparisons. The external validity of a business simulation game has generally been viewed as a measure of how well the business game models the real-world industry in which the simulation takes place (Carvalho 1991). In a classroom setting, two approaches have been used to examine the external validity of business games. The first approach has focused on the correlation between a business executive’s simulation game performance and his/her real-world performance. If the simulation game is externally valid, a successful business executive should also be successful when participating in the simulation competition. A number of studies of this nature have supported the external validity of business games. The best of these studies can be found in Wolfe and Roberts (1986). The second approach to measuring external validity employs a longitudinal research design. In this approach, a student’s business game performance is compared to some measure of subsequent business career success (e.g., number of promotions, salary level, etc.). Using this approach, two comprehensive studies have reported such a correlation (Wolfe and Roberts 1986; Wolfe and Roberts 1993). The internal validity of business simulations has also been measured in two ways. The first approach basically states that if a simulation exercise is to be considered internally valid, better students should outperform poorer students. Several studies have supported this view of the internal validity of business games (see Wolfe 1987 for one of the better studies and an overview of other research on the internal validity of business games). A second, and possibly more reasonable view of internal validity, examines whether participant decisions in a simulation competition, over time, conform to the environment of the simulation. While the dynamics of the simulation and the actions of competing companies will influence participants’ decisions, the simulated environment must be considered and, ceteris paribus, participant decisions should adapt to the simulation environment. If this type of adaptive decision- making takes place, the simulation exercise may be considered internally valid. Past research of this type has been only moderately supportive of the internal validity of business games. The most thorough study of this nature, which contains an overview of all past research on internal validity, can be found in Wellington and Faria (2001). MARKET SHARE AND PROFITABILITY The PIMS (Profit Impact of Marketing Strategies) project was initiated in the 1960s within the General Electric Company. In order to expand the program, the project was moved to the Harvard Business School in 1972 and, to facilitate the further expansion of the program, the Strategic Planning Institute was formed in 1975 to administer the project. The PIMS program is a multi-company research project designed to gather marketing and financial information on a number of different business firms for analysis purposes. Each member company of the PIMS project submits information about its business conditions to the Strategic Planning Institute each year. The PIMS staff analyzes the data to search for general laws that seem to govern the business environment (Henderson 1980). Currently, there are over 3,800 businesses contributing data to the Strategic Planning Institute each year. Based on many years of research, and through hundreds of publications on their findings, the Strategic Planning Institute has put forth nine basic findings on business strategy. The one that we are concerned with in this paper is, “Market share and profitability are strongly related” (Buzzel and Gale 1987, p. 8). Buzzel and Gale (1987) go on to state that, “…enterprises that have achieved a large share of the markets they serve are considerably more profitable than their smaller-share rivals. This connection between market share and profitability has been recognized by corporate executives and consultants, and it is clearly demonstrated in the results of our research over the past fifteen years” (p. 70). In the most common format of classroom simulation gaming, participants are grouped into companies, and companies are grouped into industries. Companies within a given industry compete against one another for a share of the served market and the resulting profitability. Given this situation, it would be easy to examine the market share/profitability relationship that occurs in simulated competitions to check whether the outcomes conform to the PIMS findings. In a similar type of study, Green and Faria (1995) examined the results from a simulation competition with regard to another PIMS finding. That is, another conclusion reported as part of the many studies published by the Strategic Planning Institute states that business strategies are successful if their fundamentals are good, unsuccessful if they are not. The implication from this is that strategies that are successful in one marketplace/economic environment will continue to be successful in a similar environment even if competitors are changed (Buzzel and Gale 1987). To test this principle in a simulation environment, Green and Faria (1995) removed the winning companies (highest earning companies) in 25 separate, five team, simulation industries and moved them to a different industry which still contained the remaining four companies. All twelve (three years) of simulation decisions were then re-run. In 18 of the 25 (72%) of the re-runs, the original winning team, once again emerged as the winner, even with four new competitors. In another three industries, the original winning team came in second. These results strongly supported the view that a fundamentally sound strategy remains a fundamentally sound strategy in a similar environment even if competitors are changed as suggested by the PIMS findings. While not the purpose of their research, House and Taylor (1991) reported a number of findings from a review of student performance in two different simulation games. Among the conclusions stated by House and Taylor (1991, p. 137) were that, “It was found that market share and plant expansion were important determinants of profitability in the executive game….In the business game environment, market share has a negative, short term impact on profitability….” This suggests one example of conformity to the PIMS findings and one example of nonconformity across two separate simulation game environments. HYPOTHESIS 333 Developments in Business Simulation and Experiential Learning, Volume 31, 2004 Based on the research cited above, the following general hypothesis is put forth for testing. H1: Market share and company earnings will be strongly and positively correlated (Pearson’s r > .5) in a simulation game competition. Past simulation research has suggested that business simulation games possess external and internal validity. The little available research to date, suggests that selected business simulation games conform to several of the major PIMS findings. Given these findings, and the findings reported from the ongoing PIMS project, it would seem, then, that market share for companies in a simulated business environment would be strongly correlated with company earnings. The selection of Pearson’s r > .5 is based on the assertion by Buzzel and Gale (1987, p. 8) that market share and profitability are strongly correlated and on Cohen and Cohen (1983, p. 61) who state that Pearson’s r values of .50 or more are considered “strong effect sizes”, while r values between .30 and .50 are considered “medium effect sizes”, and r values between .10 and .30 are considered “small effect sizes”. DATA COLLECTION AND ANALYSIS Data were collected from 440 simulation companies competing in 96 industries that participated in two different simulation games, COMPETE: A Dynamic Marketing Simulation (Faria, Nulsen and Roussos 1994) and The Marketing Management Simulation (Faria and Dickinson 1996). The data from the COMPETE competition were collected from 209 companies competing in 46 industries that were involved in 18 competitions operated by six different instructors spanning the time period from January 1996 through April 2003. The type of data available for collection included: cumulative earnings per share data for each company and an average of cumulative unit market share by product for three different products. This data was then used to compute the relative earnings per share for each company in each industry and the relative market share for each company in each industry. In addition, the cumulative market share and cumulative earnings per share data were standardized so that the COMPETE data could be combined with the The Marketing Management Simulation data. The data from The Marketing Management Simulation was collected from 231 companies competing in 50 industries that were involved in 5 competitions operated by two different instructors spanning the time period from January 1991 through June 1997. The type of data available for collection from this competition were cumulative earnings per share for each company and cumulative total sales for each company which were used to compute a total dollar market share value. This data was then used to compute the relative earnings per share for each company in each industry and the relative market share for each company in each industry. In addition, the cumulative market share and cumulative earnings per share data were standardized in the same fashion as the COMPETE data. The data were analyzed using the correlations program from SPSS P.C. Version 10. For The Marketing Management Simulation, correlation coefficients were computed to compare earnings per share versus dollar market share, relative earnings per share versus relative market share and the standardized earnings per share versus the standardized market share. Similarly, for the COMPETE simulation, correlation coefficients were computed to compare earnings per share versus unit market share, relative earnings per share versus relative market share and the standardized earnings per share versus the standardized market share. Finally, the combined databases for both simulations were used to compute correlation coefficients for relative earnings per share versus relative market share and the standardized earnings per share versus the standardized market share. FINDINGS The findings from the data analysis are reported on in Table 1. The results shown in Table 1 indicate that all of the correlations between earnings and market share, for both simulation games, were significant and that the power to detect the resultant correlations with a .01 level of significance was .93 or better in all cases (Cohen and Cohen 1983, p. 528). The findings indicate that for The Marketing Management Simulation, the relationships between earnings and market share, and standardized earnings and standardized market share, were of medium strength (r value of .392) while the relationship between relative market share and relative earnings was strong (r value of .650). For the COMPETE simulation game, the relationships between earnings and market share, and standardized earnings and standardized market share, were of small strength (r values of .213) while the relationship between relative market share and relative earnings was of medium strength (r value of .314). The combined data from the two simulations produced medium relationships for relative market share and relative earnings (r value of .496) as well as for standardized market share and standardized earnings (r value of .309). Given these findings, with only one relationship with an r > .5 for relative market share and relative earnings per share for one simulation game, the hypothesis that market share and earnings per share are strongly related, as would be suggested by the PIMS findings, would have to be rejected with regard to the two simulation games and the data collected. The performance outcomes in the simulation games utilized in this study did not demonstrate a “strong” relationship between market share and earnings per share but rather a “medium” relationship. 334 Developments in Business Simulation and Experiential Learning, Volume 31, 2004 Table 1: Pearson’s r Correlations Between Market Share and Earnings for Two Simulation Games ______________________________________________________________________________________ Combined MMS Compete MMS & Compete (N=231) (N=209) (N=440) ______________________________________________________________________________________ EPS Vs Dollar Share .392** - - EPS Vs Unit Share - .213* - Relative EPS Vs Relative Share .650** .314** .496** Standard EPS Vs Standard Share .392** .213* .309** ______________________________________________________________________________________ * Significant at .00 ** Significant at .000 DISCUSSION AND CONCLUSIONS Based on the findings from this study, the performance outcomes from the participant teams in two simulation games examined did not provide evidence of a “strong” relationship between market share and earnings per share. However, the findings do support the notion that there was a “medium” relationship between market share and earnings per share. In particular, The Marketing Management Simulation demonstrated a medium to strong relationship while the COMPETE simulation game demonstrated a small to medium relationship. A possible explanation for the variance in these findings could stem from two measurement issues. Firstly, the unit of analysis for market share in The Marketing Management Simulation game was dollars while the unit of analysis for market share in the COMPETE simulation game was units sold. The comparison of earnings to market share was dollars to dollars in The Marketing Management Simulation while it was dollars to units in the COMPETE simulation. Secondly, the market share percentages for The Marketing Management Simulation companies was total cumulative company sales divided by total cumulative industry sales for all companies combined. This combined the dollar sales of two products in two regions for all periods of simulation play. However, the market share values computed for the COMPETE firms were based on the average of the cumulative unit market shares for three products in three different regions. Unfortunately, the data available for the COMPETE simulation did not provide cumulative dollar sales that would allow the investigators to deal with the unit of analysis issue for the whole simulation. However, quarterly dollar sales and quarterly earnings data for the final period were available. In order to understand the potential impact of these measurement issues, an additional analysis was undertaken for the COMPETE data. Using final period quarterly sales data, dollar market shares were computed for the COMPETE simulation companies. This quarterly dollar market share data was then correlated with the quarterly earnings data to measure the relationship between earnings and market share to see whether there was a difference in relationship between unit market shares and dollar market shares. It was found that the correlation between dollar earnings and dollar market share for the final quarter in the COMPETE simulation was .381 and the correlation between relative dollar market share and relative earnings for the quarter was .319. In both instances, these correlations were higher than the unit share values reported and both relationships are of medium strength. This finding suggests that unit market share measures provided lower estimates of the relationship between market share and profitability in the data gathered for this study. A more thorough examination of a complete set of data over the entire competition is needed to determine the validity of this assertion. In conclusion, although the simulation games examined in this study do not “fully” fit the PIMS findings of a “strong” relationship between market share and profitability, they do demonstrate a “medium” strength relationship that is statistically significant. As such, although not “ideal” models by the PIMS standard, they can be assessed to be “acceptable” models of the business environment and once again, we have a strong validation of the use of business simulation games for teaching purposes. REFERENCES Buzzel, R.D. & Gale, B. T. (1987). The PIMS Principles: Linking Strategy to Performance. New York: The Free Press. Carvalho, T. A. (1991). Evaluating Computerized Business Simulators for Objective Learning Validity. Simulation & Gaming, 22, (2), 328-347. Cohen, J. & Cohen, P (1983). Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences, 2nd Ed., Erlbaum:Hillsdale, N.J., 25-78. Faria, A. J. (1998). Business Simulation Games: Current Usage Levels. Simulation & Gaming, 29, (3), 295-308. Faria, A. J. (2001). The Changing Nature of Business Simulation/Gaming Research: A Brief History. Simulation & Gaming. 32, (1), 97-110. Faria, A. J., Nulson, R. O. and Roussos, D. S. (1994). COMPETE: A Dynamic Marketing Simulation, Fourth Edition, Boston, Richard D. Irwin, Inc. Green, D. & Faria, A. J. (1995). Are Good Strategies Consistently Good? Developments in Business Simulation 335 Developments in Business Simulation and Experiential Learning, Volume 31, 2004 & Experiential Learning, Association for Business Simulation and Experiential Learning, 31-37. Washbush, J. B. & Gosenpud, J. J. (1991). Student Attitudes About Policy Course Simulations. Developments in Business Simulation and Experiential Learning, Walt Wheatley and J. Gosenpud (eds.), Association for Business Simulation and Experiential Learning, 105-110. Greenlaw, P.S. and Wyman, F. P. (1973). The Teaching Effectiveness of Games in Collegiate Business Courses. Simulation & Games, 4, (2), 259-294. Watson, H. J. (1981). Computer Simulation in Business, New York: John Wiley & Sons. Hall, D. R. (1987). Developing Various Student Learning Abilities via Writing, the Stock Market Game, and Modified Marketplace Game. Developments in Business Simulation and Experiential Learning, Lane Kelley and Pat Sanders (eds.), Association for Business Simulation and Experiential Learning, 84-87. Wellington, W. & Faria, A. J. (2001). An Investigation of the Environmental Awareness Attained in a Simple Business Simulation Game. Developments in Business Simulation and Experiential Learning, Association for Business Simulation and Experiential Learning, 239-249. Henderson, Bruce (1980). The PIMS Program. Cambridge, MA: The Strategic Planning Institute. Wolfe, J. (1997). The Effectiveness of Business Games in Strategic Management Course Work. Simulation & Gaming, 28, (4), 360-376. House, W. C. & Taylor, L. A. (1991). Critical Success Ratios: A Comparison of Two Business Simulations in a Multi-Year Environment. Developments in Business Simulation & Experiential Exercises, Association for Business Simulation and Experiential Learning, 137. Wolfe, J. (1987). The Effects of Game Complexity on the Acquisition of Business Policy Knowledge. Decision Sciences, 9, (1), 143-155. Keys, B. (1976). A Review of Learning Research in Business Gaming. Developments in Business Simulation and Experiential Learning, C. E. Neilsen (ed.), Association for Business Simulation and Experiential Learning, 76-83. Wolfe, J. (1985). The Teaching Effectiveness of Games in Collegiate Business Courses: A 1973-1983 Update. Simulation & Games, 17, (3), 251-288. Wolfe, J. & Roberts, C. R. (1986). The External Validity of a Business Management Game. Simulation & Games, 17, (3), 45-59. Miles, W.G., Biggs, W. D. & Schubert, J. N. (1986). Student Perceptions of Skill Acquisition Through Cases and a General Management Simulation. Simulation & Games, 10, (1), 75-86. Wolfe, J. & Roberts, C. R. (1993). A Further Study of the External Validity of Business Games: Five Year Peer Group Indicators. Simulation & Gaming, 24, (1), 21-34. Randel, Josephine, Morris, B.A., Wetzel, C. D. & Whitehill, B. V. (1992). The Effectiveness of Games for Educational Purposes: A Review of Recent Research. Simulation & Gaming, 23, (3), 261-276. FARIA, A. J. FARIA ODETTE SCHOOL OF BUSINESS Snow, C. C. (1976). A Comment on Business Policy Teaching Research. Academy of Management Review, 1, (2), 133- 135. UNIVERISTY OF WINDSOR WINDSOR, ONTARIO, CANADA, N9B 3P4 (519) 253-3000 Ext. 3101 Spect, L. B. & Sandlin, P. K. (1991). The Differential Effects of Experiential Learning Activities and Traditional Lecture Classes in Accounting. Simulation & Gaming, 22, (1), 196- 211. AD9@UWINDSOR.CA WELLINGTON, WILLIAM J. ODETTE SCHOOL OF BUSINESS Waggener, H. A. (1979). Simulation vs. Case vs. Text: An Analysis of Student Opinion. Developments in Business Simulation and Experiential Exercises. Bernard Keys (ed.), Association for Business Simulation and Experiential Learning, 113-118. UNIVERISTY OF WINDSOR WINDSOR, ONTARIO, CANADA, N9B 3P4 (519) 253-3000 Ext. 3151 R87@UWINDSOR.CA 336 Table of Contents Volume 31, 2004 Controlling the Complexity and Orenting Target Groups by a Modular, Server-Based Business Game System Learning Network Demonstration: Delivering Business Education in a Distance Learning Environment Economic Evolution, Human Capital Investment, and Adult Distributed Electronic Learning: A Literature Review Designing a Globalization Simulation to Teach Corporate Social Responsibility Developing and Teaching an Online / In-Class Hybrid: A Demonstration A Model for Evaluating Online Instruction An Evaluation of a Distributed Learning Course: A Students'-Eye Perspective Blended Learning Strategy Improved Business Writing Skills How to Receive and Process Attachemnts while Greatly Reducing the Risk of Viruses and Trojans Introducing Online Components to a Class: How to Increase teh Likelihood of Success Teaching Strategic Communications Online: Using Learning Outcomes to Develop a Case-Based Course Implementing Distance Approaches to Education: A Panel Discussion for ABSEL: Las Vegas, 2004 MANDI: Learning Management Through Field Sales Experience An International Capital budgeting Experiential Exercise A Primer To Combating Terrorism: Playing It Safe While On Overseas Assignment (An Experiential Exercise) Integrating The Business Curriculum With A Comprehensive Case Study: A Prototype The Case Brief: A Model For Case Analysis, Writing And Discussion Technology Infused Pedagogy And Delivery – A Sure Bet? A Proposal For Panel Discussion Absel Conference 2004 Research Strategy And The Bkl: Getting The Most From The Absel Archives Simple But Effective: Rediscovering The Class Discussion Needle And Thread: An Activity For Examining Various Management Behaviors A Customized Excel Data Analysis System For Use In Undergraduate Marketing Research Team Leader Selection - Does It Matter? The Power Of Perspective: Reframing Your Framing Skills For Innovative Instruction In Leadership And Influence Exercise: How Should Merit Raises Be Allocated? An Online Situation For Problem-Based Learning In A Junior-Level Management Course The Eden Alternative As A Roadway For Change: A Service Learning Quality Improvement Project Avoiding Catastrophe: The Role Of Individual Accountability In Team Effectiveness Omega Systems: A Change Management Exercise The Risks And Rewards Of Providing Students A Structured Cheating Opportunity Experimentation With Assessment Techniques: A Proposal For Panel Discussion Using A 2 - Page Case To Introduce Concepts Of Business Strategy Interactive Session The Integration Of Appreciative Inquiry And Experiential Learning For Peak Performance Appreciative Inquiry Case Story: New York City Leadership Challenge Individual Achievement Versus Team Performance: An Empirical Study With Business Games Some Strategists Don't Learn Or Can't Learn Computer Simulation: A Design Architectonic On The Value Of Bugs In Simulation Environments Online Sales Forecasting With The Multiple Regression Analysis Data Matrices Package Simulation Exercises And Problem Based Learning: Is There A Fit? A Study Of Business Game Stock Price Algorithms Assessing Individual Performance In A Total Enterprise Simulation Information Use In A Business Game Determining The Value Of A Firm Unsorting Algorithms For An Ordered List And Its Application To Business Simulations Teaching Public Finance Management Through Simulation Antecedents Of Game Performance Student Expectations Of Classroom Teaching Practices In Developing And Presenting Course Information In Hong Kong Implementation And Impacts Of The Balanced Scorecard: An Experiment With Business Games Impact: Shocking The Legacy Mindset Implementation Of The Eepad Framework Of Business Processes In An Accounting Information Systems Course Are Business Games Really Delivering What Students Are Led To Believe?? Reporting Lessons Learned: What Gets Reported; Who Gains Value Teacher Expectations Of Classroom Teaching Practices In Developing And Presenting Course Information In Hong Kong Student Reactions To The Use Of A Computer-Based Simulation As An Integrating Mechanism For A Mba Curriculum A Cognitive Investigation Of The Internal Validity Of A Management Strategy Simulation Game The Casino Challenge: Making Simulation Delivery A Safe Bet! Accounting For Company Reputation: Variations On The Gold Standard Foreign Currency Hedging: A Simulation The Influence Of Variables Easily Controlled By The Instructor/Administrator On Simulation Outcomes: In Particular, The Variable, Reflection. Absel Awareness Among Business School Faculty Validating Business Simulations: Does High Market Share Lead To High Profitability? Simulation Debriefing Procedures Coaching And Business Simulations: A Formula For Success? A Seminal Inventory Of Basic Research Using Business Simulation Games The Influence Of Scorecard Evaluation On Decisions And Outcomes