PREDICTING BUSINESS GAME PERFORMANCE FROM PERCEPTIONS OF MANAGER INPORMTION AND ACTIONS Developments in Business Simulation & Experiential Exercises, Volume 10, 1983 PREDICTING BUSINESS GAME PERFORMANCE FROM PERCEPTIONS OF MANAGER INFORMATION AND ACTIONS W. C. House, University of Arkansas H. S. Napier, University of Arkansas INTRODUCTION A number of efforts have been made to predict business game performance in terms of such accepted measures as market share, stock market price, and return on investment using key indicators, multiple correlation, analysis of past influencing factors (such as advertising levels, pricing levels, and number of salesmen), Delphi group opinion studies in stages, etc. These efforts have met with mixed success depending upon the complexity of the economic environment, the experience and ability of the game participants, and the nature of the forecasting techniques involved. A novel approach to forecasting game performance which shows some promise as a forecasting tool involves taking surveys of group participants in the early periods of game play with a view to determining their attitudes toward the importance of selected aspects of management information and manager actions. The survey results are then used to predict some aspect of profit, financial, or market performance in future periods. Six teams of undergraduate students in a business policy class were used as game and survey participants. Generally, each team had three to four members. The Keys and Leftwich Executive Simulation, a moderately complex game with two products and about a dozen decision variables, was used as a vehicle to simulate the business environment. Two trial decisions and eight quarters of play covering two simulated years were made. Income statements, balance sheets, and selected performance measures were calculated for each team at the end of each quarter after decisions were made by participating teams. During the early periods of play, group members were asked to complete a survey form indicating their rating of the importance of selected top manager information and action items. In order to make the process ~re manageable, groups 1 and 2 filled out the survey form during period 2 of game play, groups 3 and 4 during period 3, and groups 5 and 6 during period 4. Key performance measures including stock market price and return on investment were recorded for each groups to correspond to the appropriate survey period (I.E., 2, 3, and 4 for groups 1-2, 3-4, and 5-6) the same performance measures were also recorded for corresponding periods of the second year of play (I.E., period 6 for groups 1-2, period 7 for groups 3-4, and period 8 for groups 5-6). For each group, the individual ratings were averaged to obtain a group rating of importance for each item by sunning the individual scores on the importance scale from 0 to 3 and dividing by the number of team members. An overall rating was also obtained by averaging the group ratings. SUMMARY OF STUDY RESULTS Table I contains, for each of the six groups, average importance ratings for 15 management information items. A mean or overall average was also calculated for each information item. By summing and averaging the group rankings, the information items receiving the highest ratings were financial resources, new technology, market potential, sales forecast, competition situation, and economic data. A majority of these six items, all selected for further analysis, were concerned with the internal environment. Conversely, moat of those items receiving the lowest importance ratings (I.e., population trends, community attitudes, company background, government activities, future possibilities, and operations capacity) were oriented toward the external environment. Actual return on total assets for each team is shown at the bottom of the table. TABLE II IMPORTANCE OF SELECTED MANAGER INFORMATION ITEMS FOR STUDENT TEAMS DURING FIRST YEAR OF BUSINESS GAME PLAY Groups (Averages) Manager Information 1 2 3 4 5 6 Overall Future possibilities 2.67 2.25 1.33 2.50 1.75 2.00 2.08 Industry Averages 2.33 1.25 2.00 3.00 1.75 2.67 2.17 Competitor Situation 2.67 2.75 2.33 2.00 1.75 2.33 2.30 Operations Capacity 3.00 1.00 2.67 2.00 1.50 2.33 2.08 Community Attitudes 2.00 1.25 1.67 2.50 1.25 2.67 1.89 Manager Experience 2.33 2.50 2.67 2.00 1.275 2.33 2.26 Economic Data 2.67 2.25 2.33 2.50 1.75 2.33 2.30 Market Potential 2.67 2.50 2.00 2.50 1.75 3.00 2.40 Personnel Resources 2.67 2.00 2.33 1.50 1.50 2.67 2.11 Company Background 2.67 1.50 1.33 2.00 1.50 2.00 1.83 Financial Resources 3.00 2.00 3.00 3.00 2.00 2.33 2.56 Government Activities 2.67 2.50 2.00 2.00 2.00 2.00 2.20 Sales Forecast 2.67 2.50 2.33 2.50 2.25 2.33 2.43 New Technology 3.00 2.50 2.67 1.50 2.50 2.67 2.47 Populations Trends 2.67 1.50 1.33 2.00 1.50 1.67 1.78 Return on Total Assets -6.34 2.99 14.81 -8.17 5.70 -10.84 Table II displays importance ratings for 15 selected management actions made by the six student teams. An overall average or mean was calculated for each individual items. The six actions receiving the highest average ratings included identify problems, define objectives, evaluate performance, consider alternatives, find solutions, and coordinate actions. Items receiving the lowest importance ratings were employ creativity, maintain standards, seek commitment, discover capabilities, anticipate obstacles, and schedule activity. Generally, the more specific management actions received the highest ratings. Return on total assets is also shown for each team at the bottom of Table II. Generally, it was hypothesized that if the importance rating of these items were known, then the performance in future periods could be predicted. To simplify the analysis, it was decided to confine the analysis to the items rated highest by the groups, in other words to use only the five or six items with the highest overall ratings. It was also decided to limit the performance measures to one of the more common ones, in this case, return on investment. The six items in Table I were 57 Developments in Business Simulation & Experiential Exercises, Volume 10, 1983 TABLE II IMPORTANCE RANKING OF SELECTED MANAGER ACTIONS FOR STUDENT TEAMS DURING FIRST YEAR OF BUSINESS GAME PLAY Groups (Averages) Manager Action 1 2 3 4 5 6 Overall Employ Creativity 2.67 1.75 1.67 1.50 1.50 1.00 -1.68 Indicate Priorities 3.00 2.50 2.33 2.00 2.00 2.33 2.36 Identify Problems 3.00 2.50 2.67 3.00 2.25 2.67 2.68 Clarify Mission 2.67 1.75 2.00 2.50 2.50 2.67 2.34 Schedule Activity 2.67 1.75 2.67 2.00 2.00 2.33 2.24 Discover Capabilities 2.67 1.75 2.00 2.00 1.75 2.00 2.03 Find Solutions 3.00 2.00 3.00 2.50 2.24 2.33 2.53 Maintain Standards 3.00 1.50 1.67 1.50 2.00 1.67 1.89 Anticipate Obstacles 3.00 2.00 2.00 1.50 1.75 2.00 2.04 Coordinate Actions 3.00 2.75 3.00 2.00 2.25 1.67 2.44 Define Objectives 3.00 2.50 2.33 3.00 2.00 3.00 2.60 Seek Commitment 3.00 1.50 1.33 1.50 1.50 2.00 1.80 Evaluate Performance 3.00 2.25 2.67 3.00 2.25 2.33 2.58 Allocate Resources 2.67 1.75 2.00 3.00 2.00 2.00 2.34 Consider Alternatives 3.00 2.00 2.33 2.50 2.50 2.67 2.50 Return on Total Assets -6.34 2.99 14.81 -8.17 5.20 -10.84 each individually correlated with the criterion variable, and then subjected to a stepwise multiple regression procedure to estimate the predictive capability of a combination of variable considered at one time. The results of both of these operations as regard manager information are shown in Table III. TABLE III CORRELATIONS OF SELECTED MANAGEMENT INFORMATION ITEMS WITH LATER PERFORMANCE Variable No. Item R Single R Multiple (Cumulative) 3 Competitor Situation ± .07 .93 (2) 7 Economic Data ± .45 .99 (4) 8 Market Potential ± .81 .81 (1) 11 Financial Resources ± .08 -- 13 Sales Forecast ± .42 .95 (3) 14 New Technology ± .22 -- 16 Return on Total Assets 1.00 1.00 As Table III shows, only market potential is highly correlated with return on total assets when the single correlation test is run. Economic data and sales forecast show only a moderate correlation with return on total assets and new technology, financial resources, and competitor situation demonstrate very low correlation with the criterion variable at a significance level of .05. However, when the various variables are tested in a multiple correlation analysis competitor situation plus market potential yields a .93 correlation coefficient. Adding in sales forecast raises the coefficient to .95 and the addition of a fourth factor, economic data, increases the correlation coefficient to .99. Thus, for the sample study, if the importance rating of four items is known (I.e., market potential, competitor situation, sales forecast, and economic data), a very accurate prediction can be made as to what a given team’s return on total performance will be. The multiple regression formula is as follows: ROTA = 87.33 - 25.34 MPR + 19.24 CSR - 41.90 SFR + 13.30 EDR where ROTA is the second period’s return on total assets, MPR is market potential rating, CSR is competitor situation rating, SFR is sales forecast rating, and EDR is economic data rating. When the individual company ratings for market share, competitor situation, sales forecast, and economic data for the past period are substituted into the multiple regression formula for management information, return on total assets can be predicted for the second period for each team as follows: ROTA1 = 87.33 – 25.34(2.67) + 19.24(2.34) – 41.90(2.67) + 13.30(2.67) = -12.44 estimated vs. -6.34 actual ROTA2 = 87.33 – 25.34(2.50) + 19.24(2.75) – 41.90(2.50) + 13.30(2.25) = 2.07 estimated vs. 2.99 actual ROTA3 = 87.33 – 25.34(2.50) + 19.24(2.33) – 41.90(2.33) + 13.30(2.33) = 2.08 estimated vs. 2.99 actual ROTA4 = 87.33 – 25.34(2.50) + 19.24(2.00) – 41.90(2.50) + 13.30(2.50) = -9.04 estimated vs. -8.12 actual ROTA5 = 87.33 – 25.34(1.75) + 19.24(1.75) – 41.90(2.25) + 13.30(1.75) = 5.65 estimated vs. 5.20 actual ROTA6 = 87.33 – 25.34(3.00) + 19.24(2.33) – 41.90(2.33) + 13.30(2.33) = -10.50 estimated vs. -10.84 actual In only one case (I.e., team 1) is the estimated return on total assets significantly different froth the actual figure. In 5 of 6 instances, the estimated return on total assets is within 1.00 return points of the actual figures, and in 3 of 6 cases the estimated return is within 0.50 or less of the actual return figure. Thus, for the sample data the four selected information item ratings provide a very accurate basis for predicting future ROTA for individual teams. Table IV shows single and multiple correlations of selected management actions with later profit performance. Only one of the selected items, define objectives, is highly correlated to return on total assets when a single correlation test is used. Identify problems and consider alternatives are moderately correlated with the criterion variable while find solutions and evaluate performance exhibit very low correlation with this variable. However, when identify problems is added to define objectives in a multiple correlation analysis, the correlation coefficient lumps from .83 to .92, ad the addition of evaluate performance caused the correlation coefficient to increase from .92 to .99. TABLE III CORRELATIONS OF SELECTED MANAGEMENT INFORMATION ITEMS WITH LATER PERFORMANCE Variable No. Item R Single R Multiple (Cumulative) 3 Identify Problem ± .53 .92 (2) 7 Find Solution ± .16 -- 11 Define Objectives ± .83 .83 (1) 13 Evaluate performance ± .27 .99 (3) 15 Consider Alternatives ± .55 -- 16 Total Return on Assets 1.00 1.00 For management actions, ratings of only three items are needed to predict future return on assets with a high degree of accuracy (I.E., define objectives, identify problems, and evaluate performance). The multiple 58 Developments in Business Simulation & Experiential Exercises, Volume 10, 1983 correlation formula for management actions can be defined as follows: ROTA = -26.27 – 67.66 DO + 131.80 ID – 57.69 EP Where DO = define objectives, ID = identify problems, And EP = evaluate performance When individual company ratings for define objectives, identify problems, and evaluate performance are substituted into the multiple regression formula for management actions, return on total assets can be predicted for the second period for each team as follows: ROTA1 = -26.27 – 67.66(3.00) + 131.80(3.00) – 57.69(3.00) = -6.92 estimated vs. -6.34 actual ROTA2 = -26.27 – 67.66(2.50) + 131.80(2.50) – 57.69(2.25) = 4.28 estimated vs. 2.99 actual ROTA3 = -26.27 – 67.66(2.33) + 131.80(2.67) – 57.69(2.67) = 13.96 estimated vs. 14.81 actual ROTA4 = -26.27 – 67.66(3.00) + 131.80(3.00) – 57.69(3.00) = -6.92 estimated vs. -8.12 actual ROTA5 = -26.27 – 67.66(2.00) + 131.80(2.25) – 57.69(2.25) 5.16 estimated vs. 5.20 actual ROTA6 = -26.27 – 67.66(3.00) + 131.80(2.67) – 57.69(2.33) -11.76 estimated vs. -10.84 actual In four of six cases, the estimated return is within 1.00 return points of the actual figure and in the other two cases the difference is less than 2.00 return points. Generally speaking, the formula for management actions provides a ~re accurate forecast of ROTA than management information items for only two of the six teams. CONCLUSION This exercise, involving students playing a moderately complex business game, has demonstrated it is possible to predict with a high degree of accuracy the future performance of a single indicator, return on total assets, for individual teams from surveys of importance ratings of information items and actions conducted in earlier periods. This holds true for this experiment despite the fact that future results are influenced by team decisions on 10-12 key variables, the interaction of team decisions with those of other teams, and the effect of an overall economic index. Priorities assigned to individual information items and management actions apparently give an indication of how effectively specific teams will respond to environmental and industry changes that have an important impact on present and future decision outcomes. Although importance ratings for specific individual information items do not correlate highly with return on total assets (with the exception of one information item and one management action), when three or four selected information items or management actions are combined in a multiple regression format, a high degree of correlation can be achieved and a valid prediction formula developed. Prom a list of 15 or 16 management information items and actions, 3 or 4 elements can be identified which appear to have a vital effect on future results. If the importance ratings of information items and actions can be collected for a representative group of managers of real life companies, it should be possible to compare the student team results with those for the manager groups. These comparisons should indicate similarities and differences in element importance ratings for companies and groups achieving higher and lower return on asset performance. It may even be possible to determine which information items or elements are of greatest importance to long run profit performance and which should receive more attention in the academic and business world. In all fairness, the results of this study must be tempered by the fact that the sample size is small and the examination period is limited. Only six teams of three to four members were included, and the business game cited was played for only eight quarters, after two trial decisions. The effects of the interaction of the decisions of the six teams and a changing economic index were not explicitly considered. Nevertheless, the initial study results are sufficiently promising to suggest consideration of correlations which may exist between selected information items or management actions and other marketing, production, profitability or financial position performance measure. 59 Table of Contents Volume 10, 1983 Is the Computerized Business Simulation Relevant? Business Professionals Play a Student Game A Methodology For Assessing the Internal Validity of Business Simulations A Longitudinal Study of the External Validity of a Business Management Game Policy Analysis and Decision: A corporate Relocation Simulation Exercise Effective Listening: An Exercise in Managerial Communication The Symbol Exercise: An Initial Group Activity Concept Based Simulations Institutional Users of Experiential Learning Packages: A Preliminary View from Publishers' Adoption Lists How to Use Business Games in the Business Policy Course: The Students' Perspective Professors' Ratings of Business Policy Learning Methods Moot Trial: An Exercise in Trial Procedure and Evidence The Advertising Agency Game: An Experiential learning Exercise The Use of Videotaped Cases in Teaching Information Acquisition and Decision-Making Skills Experiencing Information Processing Strategies as a Means to Explore Decision Making Importance Ratings and Operations Data as Predictors of Business Game Performance Determinants of Performance in Computer Simulations Predicting Business Game Performance form Perceptions of Manager Information and actions Business Consulting: A Practicum for Undergraduate Internship as a Contingency Based Experiential Learning Program for More Effective Organizational Socialization: A Conceptual Framework MANSYM III Decision Support System Demonstration Learning the Concept of market Value Through Simulation Conflict Management for Economic Developers Development of Data Analysis Units Designed to Enhance Reasoning and Knowledge Transfer in the College Level Course BOSS: A Behavioral-Quantitative, Computer-Supported Game Entrepreneurial Potential: An Experiential Exercise in Self Analysis and Group Assessment Development of Strategists: Simulated Cases BANKRUPT: A Deceptively Simple Business Strategy Game Simulating Market and Firm Level Demand - A Robust Demand System COMPSIM A Computer Center Management Simulation Hiving Model: Assessing Management Skill Awareness The Johari Window, A Reconceptualization Role-Playing Based on Video-Tape Scenarios: An Application of Modeling to Building Supervisory Skills How to Internationalize Your Curriculum A Computerized Model of Human Behavior in a Total-Firm Management Simulation the Worksheet Approach for Simulation Game Strategy Analysis Teaching Competitive Bidding Using a DSS Generator Do We Learn from Experience? The Use of Theory Power for Increased Research Momentum in Business Simulation and Experiential Exercises Research Report on Programmatic Research on Perceived Learning Barriers with Simulation and Experiential learning An Empirical Examination of Conflict - and Nonconflict - Oriented Problem-Solving Technologies Management Curriculum: 1982 Humor as a Management Tool: Use in Formal Game Presentations Student Behavioral Change Through Teacher behavioral Change