EVALUATION OF PERFORMANCE IN BUSINESS GAMES: FINANCIAL AND NON FINANCIAL APPROACHES Developments in Business Simulation and Experiential Learning, Volume 28, 2001 EVALUATION OF PERFORMANCE IN BUSINESS GAMES: FINANCIAL AND NON FINANCIAL APPROACHES Antonio Carlos Aidar Sauaia, University of São Paulo – Brazil asauaia@usp.br ABSTRACT The purpose of this paper is to propose that the system for evaluating Business Game performance be expanded. The traditional sectorial multiples, indicators of financial accounting, reflect past performance and to them must be added operational measures, vectors of future performance. INTRODUCTION Prior to Business Games it is usual for participants to formulate these questions: a) How many rounds are we going to play? b) How are we going to be rated? c) Are we supposed to plan for the continuation of the simulated company? Such questions have prompted this study. According to Kaplan and Norton (1992), any system of evaluation has a strong influence on the behavior of the managers. In Business Games, students are aware that to achieve success they must conceive strategies, implement them and properly control the results. Mature and experienced executives already understand that measures of performance directly influence the behavior of all collaborators. "That which is not measured is not managed" (Kaplan and Norton, 1997:21). Furthermore, they have perceived that return on investment and profit per share, traditional measures of financial accounting may be quite restricted when results beyond these such as continued improvement and innovation are sought. These measures may have been acceptable in the industrial age, but are outdated when new competencies and qualifications are taken into consideration. Scholars and researchers of Business Games (Keys and Biggs, 1990:71) also believe that an important part of learning is the result of a profound reflection on the part of the participants. It occurs when conceptual models that represent the simulated entrepreneurial activities are compared with the results of their experimentation. When the cause and effect relationships are drawn up. The overall performance achieved in each round of the game is shown by the sales and production reports and also by the financial statements. Real company managers as well as academic researchers have been making efforts to improve and overcome the shortcomings of the traditional performance evaluation. While some systems highlight financial measures others recommend operational ones. Included are the level of satisfaction of internal and external clients, length of process cycles and incidence of deficiencies. "The financial results will take form by themselves" (Kaplan and Norton, 1992:71). Managers and students should not be required to choose between financial or operational evaluations. This approach will never provide a clear and comprehensive picture of the business performance in its critical domains. A more complete picture will become available when assessment incorporates financial as well as operational measures. STAKEHOLDERS VS. STOCKHOLDERS For Friedman (University of Chicago) and followers, profit is the main objective. "Companies are oriented towards a maximization of the stockholders profits by means of resources cautiously managed in accordance with the current law"(SILBIGER, 1995:53). Evaluation of a company however is not restricted to the numbers. Each assessor faces a personal judgment problem taking into account past experience and the objectives of the assessment. For instance: the accounting value of a company is obtained by adding the purchase cost of all assets to calculate the Net Worth; however the economic value is established according to the potential of future performance. Included are such considerations as the business risk, financial risk and estimated growth rate. Other factors may affect company value: the long term economic outlook, the amount and frequency of dividend payments, market price of the share, asset characteristics, the nature of the business, history of the business and the intangible assets, to name a few. When attempting to identify the important performance measures, an answer is not easily found because it depends upon how the evaluation is being directed. MODELS OF FINANCIAL EVALUATION The figures shown in financial statements are of limited significance. The true picture may be obtained analyzing the performance ratios of the results of a company in comparison with others in the same economic sector. While small margins and large volumes are characteristic of some sectors, others operate with large margins and very limited volumes. The use of financial ratios makes possible comparison between companies in the same sector and between current and past performance. SECTORIAL MULTIPLES Dozens of ratios may be obtained from a company's financial statements. Such performance indicators generally relate to the past. A review of the important financial ratios permits an overall evaluation of the company in addition to identifying problem areas (Wheleen and Hunger, 1986:30-1). Most ratios come from a basic set of financial indicators – Sectorial Multiples – which may be grouped into 4 categories Liquidity ratios – measure the organization's ability to meet its financial commitments. 210 Developments in Business Simulation and Experiential Learning, Volume 28, 2001 Profitability ratios - measure the organization's degree of success, with respect to achieving the expected profit. Activity ratios – measures the efficient use of the organization's resources. Leverage ratios – measure the contribution of financing by the owners in comparison to that by the creditors. In spite of the severe limitations, magazines such as Forbes, Business Week, The Economist, Fortune and " Exame: Maiores e Melhores" (Brazil), have developed analyses based upon traditional financial ratios in order to rank companies in accordance with Sectorial Multiples, rewarding those with outstanding performances in each sector. DISCOUNTED CASH FLOW Only a short time ago, the value of a company used to be calculated by evaluating it's operational assets – an existing business, a factory, a production line or its position in a given market. The traditional method then was the Discounted Cash Flow – DCF still in use. However, the version of the DCF that has been accepted as a standard in the last twenty years – and uses the Weighted Average Cost of Capital –WACC as a discount factor is considered obsolete (Luehrman, 1999:3). Notwithstanding, Business Schools and textbooks continue to propose the WACC simply for the sake of tradition and not because of better results. Furthermore, alternative methods such the Adjusted Present Value have been devised to assess operations or assets in the state in which they are found. The APV method may be useful for managerial analysis of an asset, and also to identify the sources of value. All methods of Discounted Cash Flow operate with the forecast of future cash flow, discounted to a present value by rates that reflect the associated risks. In this way each financial maneuver is analyzed separately, accruing the value of each to the business as a whole.(Luehrman, 1997:3). OPTION PRICING The decision to make Research and Development investments may imply two different aspects: how much will be invested or what kind of expense will be made. Whatever the final decision, it is the opportunities that are being assessed. When such an investment is undertaken, immediate operational cash flow is not generated, but with some luck, there will be an opportunity to invest again in the future. The same thing may be observed with various types of expenses in Marketing. When an investment is made to create a new brand or to strengthen an existing one, a certain result is produced, however not immediately. Companies with new technologies, ideas for product development and positions to be defended in fast growing markets, hold valuable opportunities. For some, opportunities are the most valuable thing they have. A frequent approach to formally assess opportunities is to wait until they mature, until the decision to invest cannot be further delayed. In financial terms, an opportunity is similar to an option. "If Research and Development proves that the idea is sound, we will invest". An option is a right, not a commitment to exercise a purchase or a sale at a fixed price, today or up to a given date in the future. (Luehrman, 1997:136-8). So an option to buy in the stock market may insure the holder the right to buy the share at $10 anytime during a year. If the share is worth $11, there will be a clear valorization of the option. If the value is $9, the option still is valorized because the expiration of the option ends only after a year, and if the price of the share should go up in the next months, it still may surpass $10, before the year is over. The opportunities that arise in organizations have the same characteristics as options. "If R&D demonstrates that the investment is sound", is the analogous to "if the price of the stock goes up in the next months". The potential investment to be made corresponds to the exercise of purchasing an option. The uncertainty about the future value of the operational assets is portrayed by means of the variance of the returns on the assets: that is to say it is similar to the variance of the returns on the purchase options. The recommended tactic for this analysis is based upon a mapping that compares the real project to a simple option (LUEHRMAN, 1997:141). NON FINANCIAL MODELS To measure a company's performance, sales, profits, productivity of the assets and all the other usual financial indicators may be brought into focus. Part of the manager's work is to keep the stock price high, follow the Price Earnings ratio and monitor the Balance Sheet. It is to be expected that investors and analysts as well as the participants of the Business Games do likewise. But when the long-term health of a company is to be assessed, the traditional financial indicators do not suffice. The non- financial assessments are receiving increased attention. (LOW, 1999:3). Consider the difference between the accounting value of the share and its market value. The worth of a business cannot simply be described by traditional financial data. When considering the typical assets of a company, it may be noted that there is an increasingly larger portion of intangible assets in relation to the tangible ones. The past financial performance is no longer a good indicator of the future. A recent study by the Ernst & Young Center of Entrepreneurial Innovation reports that in equity analyses the greater the use of non-financial information, the more precise are the forecasts of future profits. For the majority of the companies studied, 35% of the items used in business evaluations by institutional investors relate to non-financial information. Among them are: Ability of the company to carry out the previously established strategy. This measure, which by itself is the most important one, indicates if the managers succeed in taking difficult decisions and in exploiting opportunities quickly; if the company reached the objectives in the time set forth and within the allocated budget. Administrative credibility. If the company carries out what it states, it scores well in this item. Quality of strategy. The view that the administrators have of the future must optimize the creation of value in 211 Developments in Business Simulation and Experiential Learning, Volume 28, 2001 an economy which is highly volatile, globalized and interconnected. Innovation ability. Some companies are innovators, while others are followers. This defines their agility in adapting to technological and market changes. Capability of attracting and retaining talented people. Turnover of collaborators may be larger or smaller than that of competitors. New people may be better qualified and more experienced than those that leave. Market position. Portfolio managers and analysts measure the velocity in generating sales, profits and cash flow based upon products and markets identified during the last three years as well as the strength of the brand. Managerial experience. This refers to the abilities and experience with which the management team serves the organization and does not refer to their tenure on the job. An index of performance may be created for a senior manager in a similar situation. Executive remuneration. Alignment of remuneration policies with strategic interests is observed as well as the number of executives whose remuneration is linked to the creation of value. Quality of important procedures. This verifies how well a company carries out its strategies and if there are plans or procedures to facilitate adjustment to variations in market conditions. Leadership in research. This measure discloses how well management understands the relation between the creation of knowledge and its practical utilization. Analysts measure the R&D budget as a percentage of sales, profits and cash flow. Companies must understand the importance of non- financial measures. If they are incapable of managing and communicating such information, investors and financiers will be obliged to grope for their own conclusions. In practice, the financial market is already making its own evaluation by adopting measures of practical interest, increasingly turning towards non-financial measures. Alert managers will strive to achieve for their company superior results in all these areas Consultants, professional evaluation concerns and Business Schools are studying and changing their evaluation approaches (LUEHRMAN, 1997:135). -As a routine, companies will use more than one formal evaluation method, which is not a mere redundancy, but a way to develop analyses for specific requirements. -The analysis of the future cash flow will continue as the most formal basis for evaluations. WACC will be adopted when required by the analysis of Adjusted Present Value or the like. -Many companies will assess opportunities inherent to R&D and Marketing activities using tools derived from option pricing, simulation and decision trees. The purpose is to achieve more refined comparisons and to provide line managers with more formal analyses regarding their market positions. -Extend the capability of analysis in the company. People inside the company may learn evaluation faster than those on the outside can learn about the company business. -Processes of capital budgeting will be less rigid and more flexible. -The tendency for active participation of the financial executives in the formulation of strategy and development of the business may be accelerated. BSC – BALANCED SCORECARD The BSC - Balanced Scoreboard proposed by Kaplan and Norton (1992) has been tested in many real world companies and encompasses financial measures that show the result of actions already adopted. To these financial ones are added operational measures that quantify the customer's satisfaction, the internal procedures and the innovation activities and improvement of the organization – operational measures that represent vectors of future financial performance. The Balanced Scorecard permits administrators to view the business from four perspectives, providing answers to four basic questions (KAPLAN and NORTON, 1996:49-153): Perspective of the consumer -How does the consumer see us? Essential indexes recommended: market participation, consumer retention, consumer satisfaction and consumer profitability. Internal Perspective -What must be highlighted? Innovations; Operations; Service after sale. Perspective of Innovation and Learning -Can we continue to grow and create value? Employee satisfaction; Employee retention; Profitability per employee. Financial perspective - How do we view our shareholders? Return on Equity or Economic Value Added – EVA. Return: on sales, of assets. Increase/Characteristics of the results. Productivity for cost reduction. SCORING IN BUSINESS GAMES According to Keys and Wolfe (1990:316) learning and performance in Business Games may be classified in three areas: 1. Entry data such as choice of game and team organization; 2. Teaching and learning procedure; 3. Exit data produced in the Game. In Business Games the performance factors used are almost always based on financial measures such as profit, cash position, owners equity, dividends paid out and occasionally market share. The MICROMATIC simulator, for instance, has a standardized scoring routine that includes: profit after taxes, return on sales, return on assets, return on equity and profit per share. The administrator of the game can decide to exclude some indexes if he suspects that the participants can manipulate them: return on equity and profit per share (Washbush and Gosenpud, 1994:84; 2000:168). Undesired effects resulting from evaluation criteria may have their influence reduced by attributing weights to the indicators. 212 Developments in Business Simulation and Experiential Learning, Volume 28, 2001 But, in the software LAPTOP: Marketing Simulation, profit after tax is transformed into profit per share (EPS) which is the essential measure for the evaluation of performance during the period (Faria and Dickison 1994:36). Tangehdahl (2000:122) proposed that the share value be used as the only measure of performance in Business Games. He posed two questions: 1. Does share value properly measure the factors that indicate the participants' success in the simulation? 2. Can share value be calculated with precision in a business simulation? He considered a positive reply for the first question, when the value of the share directly reflects to what extent the students had practiced the important concepts of business simulation. Goosen (1994) had already discussed the second question and had shown that it was possible to calculate a share price with a model considering gains in the future. In the simulator MMG – The Multinational Management Game (KEYS, EDGE and WELL, 1992:49), scoring is based upon seven indices of the scoreboard: 1. market share 2. return on sales 3. return on assets 4. inventory turnover 5. asset turnover 6. debt over total assets 7. return on equity According to the MMG participant's textbook, the seven key indices offer a comparative evaluation of the competing teams. The game's administrator may change the weighting of points in each game (ten points as a maximum and one point as a minimum). The total of points so acquired is accumulated year by year. The authors of MMG did not propose the market price of a share as one of the performance evaluation criteria. Nevertheless, it behaves in a plausible manner and attracts the participant's attention because of its evolution in accordance with the annual profit, the dividends paid and the increase of shares in circulation. Variations suggest an expectation of the market with regard to future results. In accordance with the Balanced Scorecard (KAPLAN and NORTON, 1992), the financial measures used to rate teams in the MMG are directed towards past results and not toward future ones. Furthermore, measures for performance evaluation illustrate actions of the participants during simulation, not those they may carry out in the future. In the real world, companies are more interested in their manager's capabilities with respect to future than in relation to the past. Using data from exit reports, trend- measuring capabilities are created on the Scoreboard (see Table 1). CONCLUSIONS As the industrial era is over and the information era is underway it seems appropriate to review what performance measures are adopted by simulated companies. One of the key factors for successful learning is the feedback proffered by business simulations to students. Many simulations only utilize the financial indices to assess performance. Traditional financial measures do not embrace the future dimension. The sole use of financial measures may be deceptive to the game administrator and to the students. As a consequence it is proposed that: 1) Business Games must use more than one formal method of evaluation, associating some of the existing financial methods for a better understanding of past events. 2) A non-financial approach must be included in the analysis of Business Game results including identification of main vectors for future financial results. 3) Data in the exit reports of Business Games should be analyzed and associated to produce, immediately, additional indices with measures of the tendencies. This paper does not conclude the subject of Performance Evaluation but resumes the dialog on an important topic related to the learning process in Business Games. REFERENCES (UPON REQUEST) 213 Developments in Business Simulation and Experiential Learning, Volume 28, 2001 Table 1 – BALANCED SCORECARD versus MMG SCOREBOARD (MMG: existing indices; proposed indices) Measures of Strategy Strategic Objectives Measures of Occurrence Measures of Tendencies Financial perspective: Return on equity Return; on sales; on assets; profit growth; profile of sales (product A and B); Productivity by cost reduction Return on total assets Return on equity Debt to total assets Return on sales Shareholder satisfaction (market price of share) Interest rate for company in relation to the reference interest that banks charge for top clients. (financial risk) Client's perspective: Market share; Consumer retention; Acquisition of consumers; Consumer satisfaction; Consumer profitability Market Share (in each market) Consumer profitability (for each market, for each product). Customer satisfaction (difference between supply and demand) Growth of market share International operations (number of markets; product mix); Number of salespeople per distribution center. Internal Perspective: Innovation Operations Service after sales Inventory turnover Asset turnover Productivity of work stations Learning and innovation perspective: Employee satisfaction Retention of employees Profitability per employee Employee satisfaction (Turnover of employees/absenteeism) Sales per employee Productivity per employee 214 Table of Contents Volume 28, 2001 Using A Web Based Tutorial Program To Enhance Student Learning Reflecting On Reflection: An Examination Of Reflective Learning And Assessing Outcomes Overcoming Obstacles To A Cross-Global Collaborative Classroom Panel Discussion On Building And Maintaining Trust In The Abselesque Classroom Cosmopolitan-Based Cross National Segmentation In Global Marketing Simulations Strategic Management And The Case Method: Survey And Evaluation In Hong Kong Designing Interactive Self-Learning Modules Using Macromedia Director Teaching Financial Markets Using The Wall Street Journal Core Course Outcomes Assessment And Program Continuous Improvement Using An Integrative Business Plan Œ An Empirical Evaluation Using A Mock Trial Activity For Interdisciplinary Learning Fidelity, Verifiability, And Validity Of Simulation: Constructs For Evaluation 2001 International Collegiate Business Strategy Competition: 37 Years Of Collegiate Competition Motivating Students: An Initial Attempt To Operationalize The Curiosity Gap Model E-Learning: The Next Wave Of Experiential Learning Helping New Game Adopters: Four Perspectives An Initial Validity Invesitigaion of A Test Assessing Total Enterprise Simulation Learning Administering A Marketing Simulation - Common And Varied Practices Among Instructors Issues In Case Method Instruction: Class Discussion Leadership Sex Composition, Cohesion, Consensus, Potency And Performance Of Simulation Teams Integrating Management Curriculum: Linking Courses With Frontpage Absel: The Way We Talk! Teaching Strategic Management: A Case Study In The Diffusion Of Innovation In Education A Demonstration Of Bussim Strategy A Total Enterprise Simulation Teaching Methods: Etic Or Emic Antecedents Of Work Team Performance In A Business Simulation: Personality And Group Interaction When Experiential Educators Reflect Upon Their Craft: A Qualitative Research Report Strategic Sensemaking In Organizations: Model Formulation And Operationalization Panel Discussion: A Model For Initiative Development: Discussion And Ideas Of Individual Spirit Harmonics Assessing The Efficacy Of Experiential Learning In A Multicultural Environment Are Business Schools Producing 21st Century Managers? Blue Horizon Cruises: An International Simulation Exercise To Illustrate Conflicting Economic And Cultural Models Different Perceptions Of The Same Thing: An Experiential Exercise Current Use Of Agribusiness Simulation Games: Survey Results Of University Agribusiness And Agricultural Economics Programs Team Building 101 For Accountants An Online Evaluation Of The Compete Online Decision Entry System (Codes) Total Enterprise Simulation Winners And Losers: A Preliminary Study Gilding The Lily, Enhancing The Pedagogical Experience, Or Fixing What Isn™t Broke: The Dilemma Of New Simulation Versions The Absel Endnote Database: The Perfect Tool For The Bernie Keys Library Decision-Making Exercises (A), (B), And (C): Examining The Role Of Conflict In Group Decision-Making Group Decision Simulation (A)-(F): Examining Procedural Justice In Group Decision-Making Processes Evaluation Of Performance In Business Games: Financial And Non Financial Approaches A Framework For Evaluating Simulations As Educational Tools Congruence II™: A Strategic Business Board Game The Tournament Concept: Extending The Idea Of Using Simulations As Instruments Of Assessment The Economic And Political Transition Of The Mexican State In The Threshold Of Twenty First Century: From The Entrepreneurial State To The State Of Entrepreneurs. Experiential Learning For Accountants: The Not For Profit Project An Investigation Of The Environmental Awareness Attained In A Simple Business Simulation Game A Daily Exercise In Theory Application: Is This Why I Had To Take Marketing, Management, Finance –? An Alternative View to Scoring Selected Simulation Games and Experiential Exercises