IMPORTANCE RATINGS AND OPERATIONS DATA AS PREDICTORS OF BUSINESS GAME PERFORMANCE Developments in Business Simulation & Experiential Exercises, Volume 10, 1983 IMPORTANCE RATINGS AND OPERATIONS DATA AS PREDICTORS OF BUSINESS GAME PERFORMANCE W. C. House, University of Arkansas H. S. Napier, University of Arkansas INTRODUCTION Efforts to predict future business game performance in terms of such overall measures as rate of return on assets and stock market price hive not met with resounding success. However, the effect of including intermediate indicators, such as total sales volume, net income/sales, sales/R&D expenses, etc. for a number of teams in a multiple regression equation, to predict future return on assets or stock market prices for individual teams, has not been fully explored. A related question is whether student or manager rankings of the importance of selected performance indicators (e.g., total sales, sales/advertising, net income/gales, gales/R&D expense) in one year could be used to predict rate of return or stock market price in future periods. This study is designed to investigate the perceived importance, as well as the actual value of certain return measures used to forecast organizational performance. Six teams of undergraduate students (usually three to four members each) in a business policy class were used in an experiment to determine if performance ratings and previous period performance results can serve as valid predictors of future financial performance in a business game. Developed by Keys and Leftwich (1977) The Executive Simulation is a moderately complex game with two products and about a dozen decision variables, including selling price, advertising and research/development outlays, production units, number of salesmen and distribution centers, debt and dividend levels. Two trial decisions and eight quarters, simulating two years of play, were conducted. Income statements, balance sheets, and selected performance measures were calculated at the end of each quarter. During the first year of game play, group members were asked to complete a survey form which contained fourteen commonly used marketing, production, and financial performance measures, of which eight were selected for further analysis. Groups 1 and 2 filled out the survey forms during period 2 of play, groups 3 and 4 completed the forms during period 3, and group! 5 and 6 filled in their forms during period 4. Key performance measures were also recorded for the various groups during corresponding periods of the second year of play (i.e., period 6 for groups 1 and 2, period 7 for groups 3 and 4, and period 8 for groups 5 and 6). IMPORTANCE RATINGS AND FORECASTED PERFORMANCE MEASURES Table I contains, the importance ratings assigned by each of the six teams to eight a elected performance measures during year one. Five was the high eat possible rating, and one was the lowest. Average or Mean Ratings for all six groups for each selected performance measure as we as ranges were calculated to indicate the relative importance overall of each performance measure as well as the dispersion of rankings for the various groups. Net income/sales has the highest average rating followed by total unit sales. TABLE I IMPORTANCE RATINGS FOR SELECTED PERFORMANCE MEASURES BY BUSINESS GAME GROUPS (FIRST YEAR) Performance Measures I II III IV V VI __ X Range 1. Total Sales (Units) 4.33 3.00 4.33 5.00 3.00 3.67 3.89 2.00 2. Total Sales/ Salesmen 3.33 2.75 4.33 4.00 2.00 3.67 3.35 2.33 3. Net Income/Sales 4.67 3.00 4.33 3.67 4.25 4.00 3.99 1.67 4. Net Income/Assets 4.33 2.50 3.67 3.50 3.25 3.67 3.49 1.83 5. Current Assets/ Current Liabilities 4.67 2.50 3.00 3.50 2.50 3.67 3.31 2.17 6. Stock Market Price 3.33 4.00 2.67 3.00 3.50 4.33 3.47 1.60 7. Sales Revenue/ Advertising Expense 2.67 3.50 3.67 3.50 3.75 3.00 3.35 1.08 8. Sales Revenue/ R&D Expenses 2.67 3.50 3.33 2.50 4.00 3.00 3.17 1.50 Net income/assets and stock market price are third and fourth in importance on an average basis. Total sales/salesmen and sales revenue/advertising expense $ are ranked fifth and sixth in importance while current assets/current liabilities and sales revenue/R&D expense, on the average were rated seventh and eighth. Sales Revenue/Advertising Expense $ and Sales Revenue/R&D expense had the smallest ranges, indicating less disagreement on the importance of these measures among the six groups. Stock Market Price and Net Income/Sales had the next highest ranges followed by Net Income/Assets and Total Sale! (Units). The two measures with the largest ranges, indicating the greatest amount of disagreement on importance, were current assets/current liabilities and total sales As can be seen from Table I, total sales and net/income/sales, on the average were ranked highly more often than net income/assets or stock market price. Sales Revenue/Advertising, Sales Revenue/R&D Expense, Current Assets/Current Liabilities, and Net Income/Assets were ranked lower more often than the other four measures. Table II contains the values of the dependent and independent variables based on importance ratings which were used with a multiple correlation program to determine the degree of association between selected intermediate measures and the two chosen financial measures of return (i.e., return on assets and stock price). When values for variables X(3) and X(4), all for year one, are correlated with X(6) for year two, a multiple correlation coefficient of .96 is achieved. Thus, a high correlation with year two return on assets occurs when net income/assets (year one), and stock price (year one) are included in the multiple regression equation. When the appropriate first year values of X3 and X4 are substituted in the regression equation, (i.e., YRI - 93.42 - 10.39X3 - 15.72X4) predicted return on asset values can be calculated for the second year as shown in Table II In five of six cases, the forecasted return on asset figures are reasonably close to the actual results. Thus, the two variable multiple regression equation provides an efficient and effective forecast of future financial performance without the need to include a large number of variables. OPERATIONS DATA AND FORECASTED PERFORMANCE MEASURES Table III contains a listing of the actual results for eight selected performance measures for the six teams. 50 Developments in Business Simulation & Experiential Exercises, Volume 10, 1983 TABLE II VALUES OF VARIABLE INCLUDED IN MULTIPLE CORRELATION ANALYSIS TO PREDICT FINANCIAL MEASURES OF RETURN Team Total Sales (X1) Sales/ Salesmen (X2) Net Income/ Assets (X3) Stock Price (X4) Sales/ R&D (X5) Net Income/ Assets (Actual II) (X6) Stock Price (Actual II) (X7) 1 4.33 3.33 4.33 3.33 2.67 -6.34 38.47 2 3.00 2.75 2.50 4.00 3.75 2.99 50.44 3 4.33 4.33 3.67 2.67 3.33 14.81 93.69 4 5.00 4.00 3.50 3.00 2.54 -8.12 23.66 5 3.00 2.00 3.25 3.50 4.00 5.20 45.83 6 3.67 3.67 3.67 4.33 3.00 -10.84 35.52 Multiple Regression Equation: YRII = 93.42 -10.39X3 – 15.72X4 RMULT - .96 Calculation of ROA for individual teams: R1 = 93.42 – 10.39(4.33) – 15.72(3.33) = -3.92 R2 = 93.42 – 10.39(2.50) – 15.72(4.00) = 4.56 R3 = 93.42 – 10.39(3.67) – 15.72(2.67) = 13.32 R4 = 93.42 – 10.39(3.50) – 15.72(3.00) = 9.89 R5 = 93.42 – 10.39(3.25) – 15.72(3.50) = 4.63 R6 = 93.42 – 10.39(3.67) – 15.72(4.33) = -12.78 As Table III indicates, two thirds of the teams increased total sales in units, net income/sales, current ratio, and stock market price. Only one third of the teams monitored increased sales revenue/advertising dollar and only one-sixth of the teams increased sales/salesman and sales revenue/R&D outlays. COMPARISON OF FORECASTED ROA USING TWO DATA SETS Table IV contains first year values for selected performance measures (X1 to X6) which resulted for individual teams and the second year values for net income/assets and stock price for year two (X7 and X8). The regression equation is also shown. When values X3, X5, and are included in a multiple correlation equation to determine the association with period two financial measures, the first year net income/sales, sales revenue/R&D outlays, and stock market prices can be used to predict year two return on Assets with a high degree of accuracy for this sample. year two are reasonably close to the year two figures for return on assets. In the case of team four, the existence of negative figures for X2 and X3 make it impractical to calculate the return on assets for that team. As Table V indicates, in three of six cases (i.e., teams two, five, and six), the forecasted values for return on assets based on importance ratings produce a smaller forecasting error than when selected actual performance measures (year one) are used. In two of six cases, the forecasted values based on selected actual performance measures produce a smaller forecasting error (i.e., for teams one and three) than that for forecasted returns computed using importance ratings. It was not possible to calculate a valid forecast for team four using selected performance measures, so no meaningful comparison is possible. Overall, the average forecasting error using the actual performance measure was 1.92 or considerably less than the 4.34 average forecasting error for the importance ranking-based forecast. But for five of the six companies, the year two forecasted values for return on assets using either selected performance measures or importance rankings provides a reasonably accurate estimate of the return on assets four quarters later. COMPARISON OF FORECASTED STOCK PRICES USING TWO DATA SETS Table VI contains importance rankings for total sales (X1), total sales/salesmen (X2), net income/assets (x3), stock price (X4), and sales/R&D (X5) for period one and actual net income/assets (X6) and stock price (x7) for period two. A multiple correlation analysis relating importance ratings for X1 to X5 with stock market price for period two produces a multiple correlation coefficient of only .50. The only two variables included in the multiple regression equation are net income/assets, and stock market price for period one. Only the forecasted values for teams one, two, and five are reasonably close to the period two values. TABLE II ACTUAL PERFORMANCE RESULTS FOR SIX TEAMS USING SELECTED PERFORMANCE MEASURES Performance Indicator Group I Year Group II Year Group III Year Group IV Year Group V Year Group IV Year I II I II I II I II I II I II Total Sales Units 6372 6090 3980 6960 6086 8736 131 1816 7486 9436 5705 3140 Sales/Salesmen 797 435 398 387 405 317 35 182 299 236 501 262 Net Income/Sales 11.67 -6.78 0.02 2.04 5.56 11.84 -14.56 - 10.33 9.47 8.46 6.72 -15.54 Net Income/Assets 8.51 -6.34 0.99 2.99 5.81 14.81 -16.51 -8.12 1.68 5.20 5.99 -10.84 Current Assets/ Current Liabilities 2.86 2.26 1.19 0.93 1.35 2.05 0.55 0.72 3.70 29.77 1.47 1.38 Stock Market Price 61.32 38.47 44.48 50.44 69.77 93.69 32.43 23.66 43.44 45.83 63.71 35.52 Sales Revenue/ Advertising Expense 3.07 3.67 9.04 4.72 10.70 7.31 0.23 2.54 77.27 5.01 10.61 2.40 Sales REvenu/ R&D Expense 12.07 5.20 18.08 5.73 8.28 7.31 2.50 3.00 11.17 28.10 14.14 6.00 Table VII contains actual values for net income/sales, net income/assets, stock price, and sales/R&D for period one and return on assets and stock price for period two. In contrast with the analysis of importance ratings, the multiple correlation analysis of actual performance measures with period two stock price, including net income/sales, sales R&D $, and stock price for period one, results in a multiple correlation coefficient of .98. Estimated stock price values for teams two, three, and five are very close to actual figures, while the estimated values for teams one and six differ from actual values by moderate amounts. It is not possible to calculate an estimated value for team four because of the negative values assumed by some of the independent variables in period one. VALUES OF PERFORMANCE MEASURES INCLUDED IN MULT F Team Total Sales - 1 (X1) Net Income/ Sales - 1 (X2) Net Income/ Assets - 1 (X3) 1 6372 11.76 8.51 2 3980 0.02 0.99 3 6086 5.56 5.81 4 131 -14.56 -16.51 5 7486 9.47 1.68 6 5705 6.72 5.99 Multiple Regression Equation: YRI = 82.50 – 3.38X5 – 2.02X2 - .43X4 RMULTIPLE - .98 Calculation of ROA for individual teams: R1 = 82.50 – 3.38(12.07) – 2.02(11.76) - R2 = 82.50 – 3.38(18.08) – 2.02(0.02) - R3 = 82.50 – 3.38(8.23) – 2.02(5.56) - .4 R4 = cannot not be calculated accurately R5 = 82.50 – 3.38(11.17) – 2.02(9.47) - R6 = 82.50 – 3.38(14.14) – 2.02(6.72) - . Table VIII contains a comparison of forecasting errors for estimated stock prices based on importance TABLE IV IPLE REGRESSION ANALYSIS TO PREDICT PERFORMANCE MEASURES FOR UTURE PERIOD Stock Price - 1 (X4) Sales/ R&D - 1 (X5) Sales/ Advertising – 1 (X6) Net Income/ Assets - 2 (X7) Stock Price (X8) 61.32 12.07 12.07 -6.34 38.47 44.48 18.08 9.04 2.99 50.44 67.77 8.28 10.70 14.81 93.69 32.43 3388.00 0.23 -8.12 23.66 43.44 11.17 577.27 5.20 45.83 68.71 14.14 -10.61 -10.84 35.52 .43(61.76) = -8.43 .43(44.48) = 2.22 3(67.67) = 13.32 from data available .43(43.44) = 6.94 43(68.71) = -8.41 51 Developments in Business Simulation & Experiential Exercises, Volume 10, 1983 TABLE V A COMPARISON OF ESTIMTED VALUES FOR RETURN ON ASSETS USING IMPORTANCE RATINGS WITH ESTIMATED VALUES BASED ON PERFORMANCE MEASURES Team Actual Return on Assets II Two Variable Forecast Based on Ratings Forecasting Error Forecast Based on Performance Measures Forecasting Error 1 -6.34 -3.92 2.42 -8.43 2.09 2 2.99 4.56 1.57 2.22 2.34 3 14.81 13.22 1.49 14.31 0.99 4 -8.12 9.89 18.02 N/A -- 5 5.20 4.63 0.57 6.94 1.74 6 -10.84 -12.78 1.94 8.41 2.43 Total 26.01 9.59 Avg. TABLE VI VALUES OF TEAM IMPORTANCE RATINGS FOR SELECTED FACTORS TO BE CORRELATED WITH PERIOD II PERFORMANCE Team X1 X2 X3 X4 X5 X6 X7 1 4.33 3.33 4.33 3.33 2.67 -6.34 38.47 2 3.00 2.75 2.50 4.00 3.75 2.99 50.44 3 4.33 4.33 3.67 2.67 3.33 14.81 93.69 4 5.00 4.00 3.50 3.00 2.54 -8.12 23.66 5 3.00 2.00 3.25 3.50 4.00 5.20 45.83 6 3.67 3.67 3.67 4.33 3.00 -10.84 35.52 Multiple regression equation for stock prices: YSPII = 153.89 – 20.78 X4 – 9.70 X3 RMUL = .50 < R.05 = .81 Calculation of stock prices for individual teams: Y1 = 153.89 – 20.78 (3.33) – 9.70 (4.33) = 153.89 – 69.20 – 42.00 = 42.69 Y2 = 153.89 – 20.78 (4.00) – 9.70 (2.50) = 153.89 – 83.12 – 24.25 = 46.50 Y3 = 153.89 – 20.78 (2.67) – 9.70 (3.67) = 153.89 – 55.48 – 35.60 = 62.80 Y4 = 153.89 – 20.78 (3.00) – 9.70 (3.50) = 153.89 – 63.34 – 33.95 = 57.60 Y5 = 153.89 – 20.78 (3.50) – 9.70 (3.25) = 153.89 – 72.73 – 31.53 = 49.63 6 = 153.89 – 20.78 (4.33) – 9.70 (3.67) = 153.89 – 89.98 – 35.60 = 28.30 Y TABLE VII VALUES OF ACTUAL PERIOD I PERFORMANCE MEASURES TO BE CORRELATED WITH PERIOD II MEASURES X3 X4 X5 X6 X7 X8 Team Net Income/ Sales Return on Assets – 1 Stock Price -1 Sales/ R&D Outlays Return on Assets II Stock Price II 1 11.76 7.51 61.32 12.07 -6.34 38.47 2 0.02 0.99 44.48 18.08 2.99 50.44 3 5.56 5.81 62.77 8.23 14.81 93.69 4 -14.56 -16.51 32.43 N/A -8.12 23.66 5 9.47 1.68 43.44 11.17 5.20 45.83 6 6.72 5.99 68.71 14.14 -10.84 35.52 Multiple Regression Equation SPII = 178.09 – 5.43 X3 - 7.26 X6 + .81 X4 RMUL = .98 Calculation of estimated return on assets: SP1 = 178.09 – 5.43 (11.76) – 7.26 (12.07) + .81 (8.51) = 33.49 SP2 = 178.09 – 5.43 ( 0.02) – 7.26 (18.08) + .81 (0.99) = 47.52 SP3 = 178.09 – 5.43 ( 5.56) – 7.26 ( 8.23) + .81 (5.81) = 92.86 SP4 = cannot be calculated SP5 = 178.09 – 5.43 ( 9.47) – 7.26 (11.17) + .81 (1.68) = 46.94 SP6 = 178.09 – 5.43 ( 6.72) – 7.26 (14.14) + .81 (5.99) = 41.91 ratings and actual performance measures with actual stock prices. Only in the case of team one is the forecasting error for values based on importance ratings less than the forecasting error for values based on performance measures. The forecasting errors for teams two, three, five, and six are less for values based on actual performance measures than for values based on importance ratings. The average forecasting error for performance measure values is 3.25, considerably less than the average forecasting error of 13.99 for the values based on importance ratings. SUMMARY AND CONCLUSIONS Of the eight performance measures considered, net income/sales and total unit sales are ranked higher than net income/assets and stock market price by a majority of the teams. Sales Revenue/Advertising dollars and Sales Revenue/R&D Outlays, current assets/current liabilities, and total sales/salesmen were ranked lower than other measures by a majority of teams. The only importance ratings that correlate highly with period two net income/assets are net income/assets and stock market price for period one. Net income/Sales, Sales Revenue/R&D, and Stock Market Price for period one seem to be the most significant actual performance measures in predicting return on assets for period two. The only variable common to both forecasting approaches (i.e., importance ratings and performance measures) is stock market price. The average forecasting error is 4.34 for importance ratings and 1.92 for performance measures, so the performance measure based forecasts are clearly more accurate in most cases.. TABLE VIII A COMPARISON OF FORECASTED VALUES FOR STOCK MARKET PRICE BASED ON IMPORTANCE RATING WITH FORECASTED VALUES ABSED ON ACTUAL PERFORMANCE MEASURES Team Actual Stock Price II Forecasted Values form Importance Rating Forecast Error 1 Forecasted Values from Performance Measures Forecast Error 2 1 38.47 42.68 4.22 33.49 4.98 2 50.44 46.52 3.92 47.52 2.92 3 93.69 62.82 30.87 92.86 0.83 4 23.66 57.60 33.94 N/A -- 5 45.83 49.63 3.80 46.93 1.10 6 35.52 28.31 7.21 41.94 6.42 Total 83.96 16.25 Average 13.99 3.25 Net income/assets-I and stock market price-I are the most useful variables in predicting stock market price for period two from importance measures, but the forecasted values are not very accurate. Net income/sales-1, Sales Revenue/R&D- 1, and Stock Price-1 are the most important variables in predicting stock price for period two from actual performance measures. The average forecasting error for forecasts based on importance ranking is 13.96 compared to 3.25 for forecasts based on performance measures. Therefore, performance measure based forecasts of stock market price are much more accurate than forecasts based on importance measures. In addition, return on assets can be forecast more accurately than stock price based on the variables considered. In terms of importance rankings, it is possible to identify two to four variables which student teams believe to be more important than other variables on a selected list. In the case of both importance rankings and actual performance, six to eight variables can be reduced to two or three significant factors for purposes of predicting return on assets and stock market price in future periods. Return on assets can be predicted with a reasonable degree of accuracy using either importance ratings or performance measures, but only actual performance measures result in an accurate forecast of stock market price. Generally, performance measure based forecasts are more accurate than importance rating based forecasts for both measures. While the results of this study indicate some fruitful directions for further research, a note of caution should be interjected. The examination period included only eight quarters of play and six teams in one class. The effects of the interaction of the decisions of six teams and a changing economic index were not explicitly considered. Only a few factors and importance measures were evaluated. A larger number of influencing factors, more teams, and longer time periods are undoubtedly needed to validate the results. The business game results should also be compared with similar results for actual companies in consumer products and related industries. Despite these limitations, the exploratory results suggest it may be possible to identify a few key factors which are likely to have the greatest effect on selected performance measures. 52 Table of Contents Volume 10, 1983 Is the Computerized Business Simulation Relevant? Business Professionals Play a Student Game A Methodology For Assessing the Internal Validity of Business Simulations A Longitudinal Study of the External Validity of a Business Management Game Policy Analysis and Decision: A corporate Relocation Simulation Exercise Effective Listening: An Exercise in Managerial Communication The Symbol Exercise: An Initial Group Activity Concept Based Simulations Institutional Users of Experiential Learning Packages: A Preliminary View from Publishers' Adoption Lists How to Use Business Games in the Business Policy Course: The Students' Perspective Professors' Ratings of Business Policy Learning Methods Moot Trial: An Exercise in Trial Procedure and Evidence The Advertising Agency Game: An Experiential learning Exercise The Use of Videotaped Cases in Teaching Information Acquisition and Decision-Making Skills Experiencing Information Processing Strategies as a Means to Explore Decision Making Importance Ratings and Operations Data as Predictors of Business Game Performance Determinants of Performance in Computer Simulations Predicting Business Game Performance form Perceptions of Manager Information and actions Business Consulting: A Practicum for Undergraduate Internship as a Contingency Based Experiential Learning Program for More Effective Organizational Socialization: A Conceptual Framework MANSYM III Decision Support System Demonstration Learning the Concept of market Value Through Simulation Conflict Management for Economic Developers Development of Data Analysis Units Designed to Enhance Reasoning and Knowledge Transfer in the College Level Course BOSS: A Behavioral-Quantitative, Computer-Supported Game Entrepreneurial Potential: An Experiential Exercise in Self Analysis and Group Assessment Development of Strategists: Simulated Cases BANKRUPT: A Deceptively Simple Business Strategy Game Simulating Market and Firm Level Demand - A Robust Demand System COMPSIM A Computer Center Management Simulation Hiving Model: Assessing Management Skill Awareness The Johari Window, A Reconceptualization Role-Playing Based on Video-Tape Scenarios: An Application of Modeling to Building Supervisory Skills How to Internationalize Your Curriculum A Computerized Model of Human Behavior in a Total-Firm Management Simulation the Worksheet Approach for Simulation Game Strategy Analysis Teaching Competitive Bidding Using a DSS Generator Do We Learn from Experience? The Use of Theory Power for Increased Research Momentum in Business Simulation and Experiential Exercises Research Report on Programmatic Research on Perceived Learning Barriers with Simulation and Experiential learning An Empirical Examination of Conflict - and Nonconflict - Oriented Problem-Solving Technologies Management Curriculum: 1982 Humor as a Management Tool: Use in Formal Game Presentations Student Behavioral Change Through Teacher behavioral Change