WHEN PROPHECY FAILS: A SMALL SAMPLE, PRELIMINARY STUDY Developments in Business Simulations and Experiential Learning, Volume 32, 2005 WHEN PROPHECY FAILS: A SMALL SAMPLE, PRELIMINARY STUDY Alan L. Patz University of Southern California alanpatz@mac.com ABSTRACT Recent studies using THE BUSINESS STRATEGY GAME have shown that the learning of and attention to strategy ratings led to superior and large performance differences between winning and losing teams. This result is observed whether or not the participating teams are reminded of the strategy rating importance at the beginning of the competition or of the relevant total enterprise (TE) simulation manual pages before both practice decisions and all subsequent real decisions. This study produces the same results but focuses on the academic achievements of the competing teams. The results indicate that higher GPA teams outperform lower GPA ones. INTRODUCTION Recent studies (Patz, 2002, 2003, 2004) using THE BUSINESS STRATEGY GAME (Thompson, A. A., & Stappenbeck, G. J., 1999, 2002) show that the learning of and attention to strategy ratings led to superior and large performance differences between winning, first place teams, and losing, last place ones. Other variables, such as price, do not matter. The ones that do—and form the basis of an eight-point strategy rating system—are broad or focused product line, quality, service, brand image, low cost, market share leadership, superior value, and global or focused coverage. This particular total enterprise (TE) simulation is concerned with manufacturing and marketing of athletic shoes—both branded and private label—using US dollars, Eurodollars, Japanese yen, and the Brazilian real in North America, Europe, Asia, and Latin America respectively. Participating teams also compete with celebrity endorsements, on the Internet, and with company owned retail stores In addition to the strategy rating each team is measured on five other dimensions. They are sales revenue, after tax earnings, return on equity, bond rating, an d company value However, it is the strategy rating system that is pivotal in all the above noted studies. In each case the winning teams had significantly higher strategy, demand, and overall performance scores. Except for unit manufacturing costs in the first study, the winning and losing teams have no significant pricing or unit manufacturing cost differences. But, the losing teams had significantly higher unit marketing costs. THE RESEARCH QUESTION The first study of this series (Patz, 2002) did not have the additional features of the next edition of THE BUSINESS STRATEGY GAME (Thompson, A. A., & Stappenbeck. G. J., 2002). This later edition—used in the second and third studies (Patz, 2003, 2004)—included internet marketing and online sales, Latin America as a new geographic region, an option to open a chain of company owned retail stores, revised initial plant capacities, and restrictions on plant capacity expansions when forecasted worldwide demand is 25 to 50% below worldwide production potential. As noted previously, the Brazilian real is the Latin American currency Nevertheless, the results were the same in all three studies. That is, using W for winning, first place teams, and L for last place, losing teams: 1. Price was not an important W and L distinction. 2. W firms experienced higher quantity demands than L firms. 3. W firms had lower unit marketing costs than L firms, and most important, 4. W firms strategy ratings were higher than L firms and the magnitude of the differences grew as the competition continued. The second study simply repeated the first results with the new edition of THE BUSINESS STRATEGY GAME, but the third one added one additional administrative procedure. That is, the students were reminded of the strategy rating importance and the relevant TE simulation manual pages before both practice decisions and all subsequent real decisions. In the first two studies, this importance was mentioned only at the beginning of the competition. Since the strategy rating prophecy failed to change the results in the third study, some other factor or factors had to be influencing these consistent results. The most obvious one, and the one examined in this study, is student grade point averages, GPAs. 240 Developments in Business Simulations and Experiential Learning, Volume 32, 2005 THE HYPOTHESES Therefore, there are two key hypotheses for this study. This first one: H1: W firms will achieve increasingly higher strategy ratings than L firms, is obvious. There is no reason to expect different behavior. However, with regard to GPAs and using all teams involved in the competition—not just the Wand L ones—the second one is: H2: Firms with high ending simulation scores will be composed of members with high course grade point averages. METHOD A TE simulation was conducted in 4 sections of an undergraduate, capstone strategic management course over a period of 2 semesters. Each section formed an independent industry, and a total of 169 students participated. All students were seniors majoring in the various fields of business administration. SIMULATION PROCEDURES After one class session devoted to the clarification of simulation rules, evaluation procedures, and decision- making mechanics, a two-year practice decision sequence was completed. Questions pertaining to the results of each session were answered and the evaluation procedure was restated. That is, students were reminded that the cumulative scores at the end of the simulation were the figures of merit. They were reminded also of the strategy rating importance and the relevant TE simulation manual pages before both practice decisions and all subsequent real decisions. The importance placed on ending cumulative scores rather than current period results emphasizes long- rather than short-term strategies. Moreover, attention was directed to three specific conditions. First, the actual ending period of the simulation would remain unknown. (Each period is a year in THE BUSINESS STRATEGY GAME, and the length of the semester allowed for a maximum of ten periods of play.) Second, all teams were expected to end their management tenure with a going concern, not a firm stripped of long term potential in order to gain short-term ranking enhancements. Third, 20% of the semester grade for the course depended on ending cumulative score rankings. Decisions were due at specific times, processed by the simulation model, and the results were available to participating teams within two days. This allowed five days before the next set of decisions, required on a weekly basis. SIMULATION SCORING In all trials of this simulation, the importance of each dimension in the overall percentage performance ratings is as follows: sales revenue, 5; after tax earnings, 15; return of equity, 20; bond rating, 20; company value, 20; and strategy rating, 20. The sum, of course, is 100%; and, as a result, each team received a current period and game-to-date score between 0 and 100. Furthermore, the participants were privy to the algorithm that determines cumulative scores in the simulation. These scores depended upon how each team’s cumulative results compared with the leading team’s results on each of the above noted six dimensions and their percentage weights. For example, if the cumulative sales of the leading team are 100, and the second place team’s cumulative sales are 80, then the second place team’s score on that dimension is (80/100)(5) or 4 where 5 is the above percentage weight assigned to sales revenue. Each team received a weekly (one year) summary of their year and game-to-date results, and prepared their next decisions based upon these statistics and a vast amount of other data provided by the TE participant’s program. GPA AND SIMULATION SCORE DETERMINATIONS Every student in this capstone course receives a score between 0 and 100 on each of nine graded exercises—seven individual and two group ones. The individual ones are five quizzes @ 6%, participation @ 10%, and a final exam @ 30% for a total of 70%. The group ones are a case research presentation @ 10% and the simulations @ 20% for a total of 30%. Then, multiplying each graded exercise score by its percentage weight yields semester scores between 0 and 100. Then, in order to not commingle semester scores and simulation scores, the GPA for each student is calculated without the simulation score. This permits a GPA score between 0 and 80 for each student, and the corresponding simulation performance score is the one assigned to the student’s firm by the scoring algorithm of the simulation. Therefore, each student in the sample has a GPA score and a final, game-to-date, simulation score. Last, for analysis purposes, each of the four sections is divided into halves. The upper half has the highest game-to- date simulation scores, and the lower half has the lowest ones. If hypothesis H2 is correct the upper half in each section should have the highest GPAs, and the same situation should hold for all four sections combined. RESULTS Six years of actual decisions were completed, and the key findings of this study are presented in Tables 1 and 2 and Figures 1 and 2. For example, the two-factor repeated measure analysis of variance shown in Table 1 indicates that on a 0 to 100 strategy rating scale, the average result for winners (W) over the six years, 89.1, was significantly higher than the 25.5 average for losers (L), F = 76.14, p < 241 Developments in Business Simulations and Experiential Learning, Volume 32, 2005 Table 1 Strategy Rating Analysis of Variance Summary Source SS df MS F p Between Ss 44579 23 Years 18243 5 3649 2.49 .0696 Ss w. Groups 26336 18 1463 Within Ss 77941 24 Strategy 48578 1 48578 78.13 <.0001 Performance x Years 17879 5 3576 5.60 .0028 Year x Ss w. Groups 11484 18 638 Figure 1 Strategy Scores 0 20 40 60 80 100 120 140 160 11 12 13 14 15 16 Years Winners Losers 242 Developments in Business Simulations and Experiential Learning, Volume 32, 2005 Table 2 GPA Analysis of Variance Summary Source SS df MS F p Industry 1 2065 1 2065 3.9724 .0505 Industry 2 99 1 99 .3059 .582 Industry 3 10570 1 10570 173.64 <.0001 Industry 4 3508 1 3508 7.9871 .0058 All Four Industries 588 1 588 4.2364 .0620 Figure 2 Class & Simulation Standings 0 10 20 30 40 50 60 70 80 Upper Half Lower Half P e rf o rm a n ce GPA Score 243 Developments in Business Simulations and Experiential Learning, Volume 32, 2005 244 .0001. This was true for each of the six years, F = 2.49, p = .069; and the performance by years interaction, F = 5.60, p = .0028, indicates that the strategy rating differences grew over the six-year competition exercise This confirms hypothesis H1. The high performing W teams consistently have the highest strategy ratings— a result that has held across four consecutive studies. Similarly, Table 2 and Figure 2 display the results for hypothesis H2. The results for the two-by-two factor (GPAs and Simulation Scores) analysis of variance is shown for each of the four industries and the combined industries. Only Industry 2 has no GPA differences between upper and lower half simulation performance scores. In all other cases, including the four industries combined, higher performance teams did have significantly higher GPA scores. Graphically, these results are shown in Figure 2 In short, even in a preliminary study using small samples, there is fairly strong support for hypothesis H2. In this study, firms with high ending simulation scores are composed of members with higher grade point averages. DISCUSSION This small sample, preliminary study was motivated by the remarkable consistency of results in three previous ones. When the participating TE competitors are instructed regarding the importance of strategy ratings, the results remain the same. Winning teams simple dominate the strategy dimension. So, when looking for other determinants of TE simulation success, course GPAs are the most obvious. But there are problems with this choice. First, do capstone course GPAs correlate with overall student GPAs? Also, typically in all four studies, the class GPAs form a rather tight curve. Does this make a difference? Moreover, past research in this area—using different simulations—needs to be considered if it is available. The question here is: Are there any reliable themes , whether or not GPAs are a main concern? Some of these themes could be the: 1. Use of more practice decisions 2. Use of a series of classes devoted entirely to the competition before beginning practice sessions. 3. Determination of interest (motivation) in the course itself. 4. An assessment of whether or not participant decision styles affect TE simulation outcomes. All of these are directions for future research, including the use of the new online simulations, e.g., (Thompson, A. A., & Stappenbeck, G. J., 2005) REFERENCES Patz, A. L. (2002). “Strategy Learning in a Total Enterprise Simulation.” Developments in Business Simulation and Experiential Exercises, Volume Twenty-Nine, 143-148. Patz, A. L. (2003). “Revisiting Strategy Learning in a Total Enterprise Simulation.” Developments in Simulation and Experiential Exercises, Volume 30, 213-219. Patz, A. L. (2004). “Some Strategists Don’t Learn or Can’t Learn. Developments in Business Simulation and Experiential Exercises, Volume 31, 160-165. Thompson, A. A., & Stappenbeck, G. J. (1999). The Business Strategy Game: A Global Industry Simulation (6th ed.). New York: McGraw-Hill Irwin. Thompson, A. A., & Stappenbeck, G. J. (2002). The Business Strategy Game: A Global Industry Simulation (7th ed.). New York: McGraw-Hill Irwin. Thompson, A. A., & Stappenbeck, G. J. (2005). The Business Strategy Game: Competing In A Global Marketplace. (Online 8th ed.). New York: McGraw- Hill Irwin. Table of Contents Volume 32, 2005 LEARNER BEHAVIOR IN THE ONLINE CLASSROOM EXPERIENCE THE EFFECTIVENESS OF A SIMULATION EXERCISE FOR INTEGRATING PROBLEM-BASED LEARNING IN MANAGEMENT EDUCATION DEMONSTRATION OF FOUR WEB-BASED SIMULATIONS THRESHOLD COMPETITOR: A MANAGEMENT SIMULATION ENTREPRENEUR: A NEW VENTURE SIMULATION MERLIN: A MARKETING SIMULATION MICROMATIX: A STRAGETIC MANAGEMENT SIMULATION LEARNING STYLES INFLUENCES ON SATISFACTION AND PERCEIVED LEARNING: ANALYSIS OF AN ONLINE BUSINESS GAME INTERNATIONAL INTERNSHIPS: DESIGN AND EXPERIENCES SIM MAP: TURNING ACTION-BASED LEARNING INTO SIMULATED CONSULTING PROJECTS NOTEL HEALTH SERVICES: A ROLE-PLAYING SIMULATION TEACHING EXPERIENTIALLY WITH THE MADELINE HUNTER METHOD: AN APPLICATION IN A MARKETING RESEARCH COURSE SIMULATING CUSTOMER LIFETIME VALUE: IMPLICATIONS FOR GAME DESIGN AND STUDENT PERFORMANCE VIRTUAL PROGRESS: SIMULATING ECONOMIC DEVELOPMENT ONLINE STRATEGIC MANAGEMENT: AN EVALUATION OF THE USE OF THREE LEARNING METHODS IN CHINA ADOPTION OF DISCUSSION-BASED TEACHING AND ASSESSMENT IN TEACHING STRATEGIC MANAGEMENT IN CHINA STUDENTS' VIEW ON THE USE OF CASE METHOD IN CHINA CHINESE STUDENTS' PERCEPTIONS OF BUSINESS GAMING EXPERIENCECSR - A CORPORATE SOCIAL RESPONSIBILITY SIMULATION CREATING DYNAMIC INTERACTION IN A VIRTUAL WORLD: ADD VALUE TO ONLINE CLASSROOMS THROUGH LIVE ELEARNING AND COLLABORATION: A DEMONSTRATION CAPABILITIES OF EXPERIMENTAL BUSINESS GAMING A COMPARISON BETWEEN SOLUTIONS AND DECISIONS IN A BUSINESS GAME VALIDATING BUSINESS SIMULATIONS: DOES HIGH PRODUCT QUALITY LEAD TO HIGH PROFITABILITY? ALIGNING ART AND EPISTEMOLOGY: ILLUSTRATIONS TO DISTINGUISH DISCOVERY FROM KNOWLEDGE BUILDING TUTORIALS USING WINK STUDENTS AS LAB RATS: THE ETHICS OF CONDUCTING NON-PEDAGOGICAL RESEARCH IN THE CONTEXT OF CLASSROOM SIMULATIONS AND EXPERIENTIAL LEARNING THE EFFECT ON GAME PERFORMANCE OF DIFFERENT MEASURES AND UNITS OF ANALYSIS IN QUANTITATIVE ANALYSIS ANALYZING AND THINKING WHILE PLAYING A SIMULATION COMPUTER BUSINESS SIMULATION DESIGN: THE ROCK POOL METHOD EXPANDING THE ROLE OF E-ROOMS IN DISTANCE LEARNING APPLICATIONS TO MANAGEMENT EDUCATION APPLICATION OF TRADITIONAL AND ONLINE JOURNALING AS PEDAGOGY AND MEANS FOR ASSESSING LEARNING IN AN ENTREPRENEURIAL SEMINAR DEVELOPING MANAGERIAL EFFECTIVENESS: ASSESSING AND COMPARING THE IMPACT OF DEVELOPMENT PROGRAMMES USING A MANAGEMENT SIMULATION OR A MANAGEMENT GAME INTERNATIONAL MANAGEMENT GAME Œ AN INTEGRATED TOOL FOR TEACHING STRATEGIC MANAGEMENT INTERNATIONALLY STUDENT EXPECTATIONS OF SIMULATIONS DISTANCE EDUCATION DELIVERY OF AN INTENSIVE SIMULATION BASED COURSE TEACHING SERVICE LEARNING USING A BUSINESS GAME ROLE-PLAY SIMULATION EDUCATIONAL PERSPECTIVE OF COLLABORATIVE VIRTUAL COMMUNICATION AND MULTI-USER VIRTUAL ENVIRONMENTS FOR BUSINESS SIMULATIONS SIMULATION PERFORMANCE & PREDICTOR VARIABLES: ARE WE LOOKING IN THE WRONG PLACES TO MEASURE THE RIGHT LEARNING? VIDEO CASE: JET-A-WAY INC. Œ FOCUSING ON DIVERSITY AND ENTREPRENEURIAL LEADERSHIP ACTIVE LEARNING: WHAT IS IT AND WHY SHOULD I USE IT? FACILITATING THROUGH COLLABORATIVE REFLECTIONS TO ACCOMMODATE DIVERSE LEARNING STYLES FOR LONG-TERM RETENTION ONLINE CUMULATIVE SIMULATION TEAM PERFORMANCE PACKAGE WHEN PROPHECY FAILS: A SMALL SAMPLE, PRELIMINARY STUDY USING EXPERIENTIAL LEARNING TO INTEGRATE THE BUSINESS CURRICULUM FORECASTING STOCK VALUE EMPLOYING PROGRESSIVE PRACTICES AND PRINCIPLES TO FACILITATE SEMINAR ROOM LEADERSHIP AMONG LEARNERS: SHARED POWER AND COLLECTIVE ACCOUNTABILITY INDIVIDUAL ACHIEVEMENT DOES NOT GUARANTEE TEAM PERFORMANCE: AN EVIDENCE OF ORGANIZATIONAL LEARNING WITH BUSINESS GAMES DECISION MAKING IN BUSINESS SIMULTION DESIGN ZUG UM ZUG 2015: COLLECTIVE BARGAINING AS A TWO-LEVEL GAME A NEW METHOD FOR MODELING INNOVATION AND R&D IN BUSINESS SIMULATIONS: ILLUSTRATED WITH A SIMULATION OF A NEW PRODUCT DEVELOPMENT PORTFOLIO DEVELOPING A MICRO SIMULATION EFFECT OF MARKET SHARE AND PRODUCTION EXPERIENCE ON COMPANY PROFITABILITY RE-DESIGNING A CURRICULUM THAT VALUES A WORK-INTEGRATED APPROACH TO STUDENT LEARNING HOW SIMULATIONS AND EXPERIENTIAL LEARNING FIT AS WE COMPLY WITH LEGISLATIVE AND AACSB ASSESSMENT GUIDELINES: HOW TO DEVELOP ACADEMICALLY SOUND COURSES THAT ALSO MEET STAKEHOLDER NEEDS EVALUATING SERVICE LEARNING: REFLECTION AND ASSESSMENT FROM THE STUDENT POINT OF VIEW SIMPLIFYING AND ENHANCING FINANCIAL ANALYSIS IN CASES AND SIMULATIONS OVERCOMING THE BUSINESS GAME COMPLEXITY PARADOX EXPLORING THE PREFERENCE IN LEARNING APPROACH AMONG THE HONG KONG UNIVERSITY STUDENTS: CASE STUDY, PROBLEM-BASED OR TRADITIONAL TEXTBOOK QUESTION AN EXERCISE FOR EXPLORING THE RELATIONSHIP BETWEEN JUNGIAN PSYCHOLOGICAL TYPES AND POLITICAL STYLE IN THE WORKPLACE EVALUATING THE DIRECTION OF RESEARCH IN ONLINE EDUCATION: ARE WE GOING ANYWHERE? AIS RAIL SYSTEM: A COMPUTER-BASED JOB-ORDER COST SIMULATION BLOOM BEYOND BLOOM: USING THE REVISED TAXONOMY TO DEVELOP EXPERIENTIAL LEARNING STRATEGIES DELIVERING A TECHNOLOGY-BASED CASE IN A TECHNOLOGICAL WAY: THE SMARTCART CASE USING THE INTERNET TO ENHANCE COURSE PRESENTATION: A HELP OR HINDRANCE TO STUDENT LEARNING TEACHING PRACTICES: A CLUSTER ANALYSIS OF STUDENTS IN HONG KONG TEACHING PRACTICES: A CLUSTER ANALYSIS OF TEACHING STAFF IN HONG KONG EVALUATING A SIMULATION WITH A STRATEGIC EXPLORATION TOOL A SYMBOLIC MODEL OF THE SIMULTANEOUS ACHIEVEMENT OF CONCRETE BENEFITS AND LEARNING BY PARTICIPATING GROUPS IN EXPERIENTIAL ACTIVITIES: ‚THE SPHERE OF EXPERIENTIAL LEARNING