THE USE OF AN EXPERT SYSTEM TO DEVELOP STRATEGIC SCENARIOS Developments in Business Simulation & Experiential Exercises, Volume 15, 1988 106 THE USE OF AN EXPERT SYSTEM TO DEVELOP STRATEGIC SCENARIOS Marian Sackson, Pace University New York Andrew Varanelli, Jr., Pace University New York ABSTRACT The mission of the expert system model development was to provide an experimental environment to test its decision making capabilities as it simulated group decision makers. The expert system acquired knowledge dynamically as it reacted to a stochastically changing environment during three simulated years of competing In a business environment. The experiment intends to provide a preliminary understanding of the methods by which an expert system develops strategies. It also contributes to the growing interest by the business community in applicability of expert systems to business decision making. INTRODUCTION This paper will describe an experiment conducted to test an expert system model developed to simulate group decision makers in a strategy development and policy making environment. The experiment is limited in scope. It Is a first step to building a comprehensive expert system model. The intent of the model is to provide an experimental environment to test the use of expert systems as a tool to help decision makers in evaluating policy and strategy circumstances facing the organization. This paper will discuss expert systems and their relevance to the decision making aspects of strategy development and policy implementation. Then it will explore the issues of group decision making that are frequently encountered in the development of strategic alternatives. Finally the details of the experiment, the validation process, and the preliminary results will be examined. DECISION MAKING Decision making is as fundamental to human existence as it is complex. It involves many different kinds of knowledge and planning behaviors. Decision making requires cognitive activity and is psychological in nature. The background for the analysis of the decision process was based on the extensive study of human problem solving performed by Newell and Simon (1972) as well as the analysis of the decision process by Simon (1960), In Simon’s model (1960). decision making is a three phased process of 1) intelligence, the process of finding possible courses of action for making a decision; 2) design, the process of finding possible courses of action; and 3) choice, the process of choosing among courses of action. Humans operate as information processors. One inputs a set of symbolic structures from the task environment. The cognitive processor retrieves information from memory, evaluates and processes the information until the problem is solved. The psychology of the person determines the internal representation of the task environment. Learning enhances the cognitive process by making the information retrieval more efficient. Newell and Simon (1972) developed a computer program, Logical Theorist, to simulate the decision making behavior during problem solving in specific task environments. These early achievements have spawned further interest in the development of computer programs that became known as “expert systems”. EXPERT SYSTEMS Expert systems are computer programs that model the decisions of human experts, work in specific task domains, gain power by having large amounts of knowledge and use an automated inference process to draw conclusions from that knowledge. Many of the current expert systems capture the knowledge of more than one expert in the same task domain and provide timely advice when a human expert is unavailable. Rauch (1984) writes that the term “expert system” exists because this class of computer programs are normally thought to require human specialists for solution. The programs are also called “knowledge-based” because researchers have found that accumulating large amounts of knowledge, rather than sophisticated reasoning techniques, is responsible for the success of the computer programs. Research efforts in the area of expert systems have resulted in the development of computer programs that provide the user with a context for specifying different courses of action (Winston and Prendergast, 1984). The user is able to project a scenario beginning from that action and is able to estimate the likely consequences of that action. Waltz’s (1983) assessment of promising directions for future research in the applicability of expert systems included creating methods for developing new inference rules from problem solving experiences. Little (1986) in his summary of the research opportunities in the decision sciences and the management sciences, highlighted the importance of understanding decision making over time, understanding expert judgment and understanding group decision making among people in real organizations. In the research described herein, the authors will attempt to respond to the statements of Waltz (1983) and Little (1986) and develop and assess the ‘reactions” of an expert system to a group decision making process. Central to this interest will be (1) the effect on the expert system of a stochastic decision making environment and (2) the success of implementing new decision rules as experience in decision making is gained by the decision makers. Developments in Business Simulation & Experiential Exercises, Volume 15, 1988 107 PROPOSED RESEARCH EFFORT A management game was the environment chosen as an experimental setting to provide an adequate test of the expert system designed to develop and monitor strategic alternatives. This setting offered the opportunity to test the expert system model in a real stochastic decision making environment while at the same time having a level of control over the limits of the experimental task domain. The general method that the experiment has followed: * A set of expert system decision rules based on testimony of expert users was developed The relationships among the variables that affect the decisions came from the testimony of faculty members in a university management department. A truncated Delphi technique was used to collect expert testimony. * An initial knowledge base was built from four business years of historical data covering strategic decision making in a specific company. * The expert system was developed using the decision rules and knowledge base discussed above. The expert system shell used for this study, Expert Ease, is a microprocessor based program. Expert Ease develops inference rules by applying examples of the decision making process. Examples can be added periodically, thus, modifying the rule set and simulating experiential learning. * After the model was built, it was tested using a set of data from a recently completed three year game iteration. This test was implemented by taking the results of a recent game, replacing one of the teams that participated in the game with the expert system and rerunning the game period-by-period starting with the initial historical data and continuing for 7 quarters. The experiment used ‘real world decisions” previously made by six of the participating student teams and the new decisions developed by the expert model, replacing the seventh team. * The results obtained when the game was played by all student teams were compared with the results obtained when one student team was replaced by the expert system. * The expert system mode’ was modified based on the experimental performance. The same criteria used by the game administrators to evaluate the student teams’ performance was used to evaluate the expert system performance. This evaluation included measures such as percentage market share, product prices, productivity, price. earnings ratio, price-dividend ratio, stock price, and profit on sales. If the expert system model’s performance was inferior to the competition based on the benchmark criteria, the relationships between the variables that formed the expert system model’s decision rules were adjusted. The game was then replayed with the modified expert system model. Again, the experimental performance was evaluated. Adjustments to the decision rules were made as necessary. * Finally, the expert system was used in a life game iteration where the system competed against student teams. The game competition covered twelve actual competitive cycles comprising three business years. GAME DESCRIPTION The task environment used for the strategic decision making experiment was the strategy planning and decision making component of a competitive business game applied for a sequence of time periods. At Pace University, the Business Strategy and Policy Game (BUSPOG) by Eldredge and Bates (1980) has been used for a number of years as a management decision making laboratory in the Business Policy course. The BUSPOG is a management exercise involving three levels of managerial activities: functional (marketing, production, personnel and finance), coordinative and organizational. The BUSPOG deals with a hypothetical refrigeration industry made up of as many as seven companies which compete within three markets. The management team of each company makes a set of operating decisions for each quarter of the year. The particular decisions include selling price, product research and development, scheduled production work week, raw materials ordered, profit sharing, and dividends. See Figure 1 at the end of the text for the actual list of the decisions the teams make each quarter. The decisions are processed through the game computer program. The results of the competition for each quarter are reported to each company in the form of computer printouts. These results depend not only on a management team’s internal decisions; but also on the companies external environment which includes the customers, competitors, and the economic situation. The BUSPOG computer program incorporates a number of hypothetical relationships. These relationships represent a conceptualized view of how such an industry and its environment might behave. In the real world, the forms of some of these relationships are known with a degree of certainty while those of others are only vaguely known. This is also the situation in the game. The game’s economic environment is determined by seven variables: GNP, personal consumption, expenditures for durable goods, number of household formations, bond interest rate, loan interest rate, and raw material cost. The stochastic factors are described in Figure 2. The University’s Management Department faculty, acting as the knowledge source, were given a set of 30 possible operating variables and relationships that influence the decisions made while “playing” the game. They rated the importance of the variables. They listed the factors affecting these variables in game competition. Faculty responses were evaluated and recirculated using a truncated Delphi technique. From this process, the model developers evolved a set of specific decision rules which were incorporated into the initial expert system model. After the expert system model was fine tuned using several controlled experimental runs, it became one of the group decision makers competing with student groups in an actual industry. The student groups were not told they were competing against an expert system. Developments in Business Simulation & Experiential Exercises, Volume 15, 1988 108 THE EXPERT SYSTEM MODEL According to Christensen and Andrews (1978), strategy is the pattern of decisions in a company that I) shapes and reveals its objectives, purposes, or goals; 2) produces the principal policies and plans for achieving these goals; and 3) defines the business the company intends to be in and the kind of economic and human organization it wants to be. The management game provided the means for studying decision making in the broad strategic sense within a controlled experiment. The research design planned to allow the development team to study the strategies. More importantly, the effectiveness of the decision can be tested in a simulated environment within a realistic time period. Inferences can be drawn regarding the longitudinal impact of strategic scenarios in a collapsed time frame. In designing the expert system model the research team evaluated the decision rules and data available in order to set up an expert model which could best respond to the demands management faces in the game’s competitive environment. Experience has indicated the three basic strategies work best in the game’s simulated market place. These strategies are: 1. High Volume - Low Price: This approach seeks overall cost leadership using high volume sales coupled with low cost production. In turn, this low cost will enable the company to set a lower selling price resulting in raising market share. Margins are usually small; but if price leadership can be maintained performance is usually satisfactory. 2. Product Differentiation: This approach will entail the production of a differentiated product that will culminate in the attainment of a steady market share while achieving a high profit margin. The firm benefits by reducing the impact of price competition and establishing a recognized perception of the product’s characteristics and quality. 3. Reactionary: This approach is followed by the firm that is no longer capable of assuming a leadership position in the market place. The firm cannot maintain a highly competitive stance and reacts to the behavior of the other companies in the competitive environment. This strategy is usually the result of a lack of financial resources. With the above as a background, the expert system development started. The model was to be constructed in three parts. These are: 1. The Analysis Section. Its function is to analyze the current operating quarter’s results and compare these with historical data. Based on this analysis, through the use of specific decision rules, the model will decide to continue with the current strategy or switch to a different strategic scenario. 2. Strategy Development and Implementation. This section of the expert system comprises three distinct substructures: a) high volume - low price procedures, b) product differentiation procedures and c) reactionary procedures. The analysis section will route the process through the appropriate decision making algorithm. The selected set of procedures in each substructure is a specific knowledge base containing the relationships that will implement the model’s chosen strategy. 3. The Reporting Segment. This model section will report the developed decision list formatted for the input to the management game. Figure 3 shows a schematic diagram of the expert system described above. RESEARCH QUESTIONS The research effort, will attempt to describe the group decision making process over time through an operationally defined model. The model will attempt to simulate the behavior of group decision making while engaged in management gaming. The research questions studied are: 1. How will the expert system react to the stochastically changing environment of the management game? 2. How and to what extent will the expert system modify the decision rules as a result of acquiring “knowledge” through changing examples of decisions made in the dynamic, competitive game environment? 3. As the expert system simulates the game over time, will it in the long run develop the game algorithm or its decision making model? 4. How will the expert system perform relative to student actors in the same competitive arena? Obviously, no single study can fully examine all these components. This research attempts to gain some insight into the strategic decision process and present an operational basis for further research. EXPERIMENTAL PERFORMANCE The initial experiment eliminated the strategy selection decision. It used a knowledge base that followed the reactionary strategy. The researchers decided to minimize the complexity of validating the expert system by employing one knowledge base at a time. An industry, composed of seven companies, was chosen for the validation phase for the following reasons: 1. The competitive behavior and performance of the seven companies within the industry had been somewhat similar. There was a “best” performing company according to the specific criteria measurements; but that company’s performance was not outstanding relative to the other companies. 2. The specific company chosen to be replaced by the expert system had performed as an average competitor in the original game iteration. 3. The validation process can be more easily Developments in Business Simulation & Experiential Exercises, Volume 15, 1988 109 controlled using a fairly homogeneous environment. As changes are made to the variable relationships, the changes in the behavior of the expert system in the competitive environment become more apparent. 4. The researchers decided, as a means of establishing a control group, to re-use the operating decisions made by the student teams for the original “live” game iteration. After seven quarters of game playing, substituting the expert system for an existing company and rerunning a previously completed game iteration, the expert system’s performance was evaluated. The expert system, using the historical data, priced the products above the competition in the two commercial markets and lower in the industrial market that was undeveloped by the competition. Initially the system’s productivity was low in comparison with the competition; therefore, the expert system reacted by increasing the process research and development budget and the production workforce while maintaining a stable plant and equipment investment. The expert system gradually lost Its share of the two commercial markets. Its product price remained high in comparison to the competition despite using comparable advertising budgets and slightly increased direct sales expense budgets. The financial reports showed that the expert system maintained a steady but slow growth in net income. The stock price was lower than the competition, owing in part to low profits and in part to low dividend payments. The total evaluation of the first use of the expert system revealed that the expert system reacted too slowly to changes in the competitive arena. Also, the pricing model needed modifications. The dividend policy development rules had to be changed to reflect some consideration of industry norms. Figure 4 illustrates a sample of the expert system’s operating decisions in comparison to those of the students. As a result of the first test of the expert system, the researchers made adjustments within the relationships of variables in the product pricing and dividend payment inference system. The same seven quarters were rerun starting with the original historical knowledge base. As in the first experimental validation test, the other six competing companies made the same decisions they had made when the game was run “live” with all student teams The decisions of the expert system for each quarter became part of the knowledge base and was used by the expert system for the next quarter decisions. The expert system’s second seven quarter performance results were compared with the first experimental performance based on the selected benchmark criteria specified previously. This time the expert system’s product pricing was comparable to the competition’s product pricing. The expert system was able to maintain a larger market share than the first experiment. The expert system did not concentrate on the industrial market because its other product prices were competitive in the commercial markets. Nonetheless, the expert system’s market share gradually decreased during subsequent quarters. The explanation for this phenomenon seemed to center around a shortage of raw materials which reduced the expert system’s production capacity. The system also hired and fired production workers in quick order. It seemed to react too quickly to changes in the forecasted product demand, the finished products and raw material inventory. These production problems caused influences in plant operation resulting in higher product costs. The model therefore had to increase prices. Demand suffered. The financial reports showed again the expert system maintained a steady but slow growth in net profit. The stock price was still lower than the competition, but closer to that of the other companies than the first experiment. The second set of performance measures are illustrated in Figure 5. As a result of the second test of the expert system, adjustments were made in the production modeling section. First, the raw material ordering sub-system was modified to better calculate required inventory levels. Second, the production workers staffing model was changed to eliminate wide swings in worker levels. The hiring and firing of production workers was better controlled. A third test run was completed. The results revealed the expert system’s performance was more competitive and generally better than some of the competing companies. It maintained a larger market share, better dividend payments, and better productivity than previous test runs. The expert system seemed to be sufficiently fined tuned to compete in a live competitive environment. Figure 6 compares the expert system’s performance to the team it replaced in test 3. CONCLUSIONS At this time the experiment is not yet completed. By the time this paper is presented additional experimental results will be available. The expert system model has completed several fine stages of tuning. Many modifications to game impact variables and to decision rules have been made to improve performance. During the fall of 1987 the expert system will have completed the process of performing in a live competitive setting along with the student actors. During the live run student teams are unaware that they are competing with an expert system. At the end of the game iteration, the student teams will be asked to complete a questionnaire. They will be asked: 1. Did your team consider the competition when making operating decisions? 2. Did you establish a strategy before starting the game? 3. If, you had an initial strategy, did you stay with the strategy or change to another strategy? 4. Did you attempt to understand or consider the strategy of competing companies? 5. Did you think there was anything unusual about the performance of any of the competing companies? If so, identify the company by name or names. Developments in Business Simulation & Experiential Exercises, Volume 15, 1988 110 The questionnaire will also be administered to regular student industries. A comparison between the two groups will be used to determine whether students in the special industry sensed a different type of competition. The researchers believe a satisfactory competitive performance by the expert system will result in the students assumption that the expert system was another live student team. REFERENCES Christensen C.R. and Kenneth Andrews et al, Business Policy Text and Irwin, Inc. Homewood, Ill. 1978. Eldredge, D. and N. Bates, The Business Strategy Policy Win. C. Brown Publ., Dubuque, Iowa, 1980, 2nd ed. Little, John, “Research Opportunities in the Decision and Management Sciences”, Management Science, Vol. 32, No. 1, 1986. Newell, Allen and Herbert Simon, Human Problem Solving, Prentice-Hall Inc. Englewood Cliffs, N.J., 1972. Rauch, H. “Probability Concepts for an Expert System Used for Data Fusion”, The Al Magazine, Vol. 14, Fall, 1984. Simon, Herbert, The New Science of Management Decision, Harper and Brothers, New York, N.Y., 1960. Waltz, David, “Artificial Intelligence: An Assessment of the State-of-the-Art and Recommendation for Future Directions”, AI Magazine, Vol. 13, Fall 1983. Winston, Patrick and Karen Prendergast, The Al Business, The MIT Press, Cambridge, Mass, 1984. FIGURE 1 OPERATING DECISIONS MADE BY BUSPOG TEAM PARTICIPANTS The simulated competition of BUSPOG requires that the management team for each company make a set of operating decisions once for each quarter of a year. The particular decisions involved include these: Marketing 1. Selling price for each of the three markets. 2. Advertising budget for each of the three markets. 3. Salespersons hired or discharged in each of the three markets. 4. Product research and development budget for the company. Production 1. scheduled production work week for the company. 2. Change in the production labor force for the company. 3. Allocation of finished product to the three markets. 4. Process research and development budget for the company. 5. Raw materials ordered for the company. 6. Plant investment budget for the company. FIGURE 1 CONTINUED Personnel 1. Sales salaries for the company. 2. Sales training budget for the company. 3. Production wage rate for the company. 4. Production training budget for the company. 5. Profit sharing for the company. Finance 1. Bonds sold or redeemed for the company. 2. Bank loan requested for the company. 3. Dividends paid by the company. 4. Stock issues by the company. 5. Long term savings account deposit or withdrawal for the company. FIGURE 2 STOCHASTIC FACTORS 1. Manufacturing productivity. 2. Raw material required per unit of finished product. 3. The number of salespersons who quit if their salary and/or profit sharing is less than the industry average. 4. Finished product demand. 5. Values appearing in the industry estimates report. Developments in Business Simulation & Experiential Exercises, Volume 15, 1988 111 Developments in Business Simulation & Experiential Exercises, Volume 15, 1988 112 Table of Contents Volume 15, 1988 The Role of Experiential Knowledge and Human Information Processing in Decision Making A Semantic Differential Instrument to Evaluate Experiential Teaching Methods A Comparison of Two Approaches to Management Skill-Building in an Organizational Behavior Course: A Replication Integrating Simulations: A Model for Business Policy Success Capstone Renaissance = Simulation + Interaction + DSS A Hybrid Method of Executing a Management Simulation: Combining the Best of Mainframes and Microcomputers Providing an Experiential Dimension to Cost/Managerial Accounting Courses Utilization of Computerized Tax Research Services in the Tax Research Curriculum Using and Expert System Based Decision Aid in Accounting Information Systems Event-Extended Entity-Relationship Diagrams for Understanding Simulation Model Structure and Function Multiple Objectives in the Development of the Gordon Macro Game A Comparative Study of Strategic Performance Factors in Actual and Simulated Business Environments An Empirical Investigation of Integrated Spatial-Proximity MCDM-Behavioral Problem Solving Technology Group Decision models Computer Simulation of Human Interaction The Role of Experiential Learning and Simulation in Teaching Management Skills Expert Systems - The New Business Simulation Tool Integrating Prolog into and Undergraduate Logistics Course Simulating Material Requirements Planning on Lotus 1-2-3 Innovation in Management Education: The Impact of the AACSB Experiential Learning in the International Environment Educational Testing with the Microcomputer A Simulation of Investment Analysis, Portfolio Management and Reporting Using Lotus 1-2-3 The Use of an Expert System to Develop Strategic Scenarios Two Exercises for Teaching about Motivation Sex Roles and the Good Manager A Form and Process for Nonconfidential Peer Evaluations Simulation and the Recalcitrant Student Employee Rights-Student Rights: A Classroom Exercise Computer Simulated Competition: An Alternative to Team Play Management Simulation The Relationship of Locus of Control and Vividness of Imagination Measures to Simulation Performance Formal Planning, Simulation Team Performance, and Satisfaction: A Replication Experimental Analysis of Magnitude and Source of Students' Inequitable Classroom Perceptions in Three Reward Conditions Strategy Design, Process and Implementation in a Stable/Complex Environment: An Exploratory Study Matching a Strategy Simulation to the Business Policy Literature: A Black Box Approach to Simulation Development An Evolutionary Classroom Experiential and Computer Simulation Model of a Corporate Strategic Planning System Collective Bargaining in the City of Elson: A Public Sector Experience Should Students Play Games in Labor Relations? Applying Cognitive Educational Objectives to Business Management Cases Grading as a Teaching and Feedback Mechanism: Involving Students in the Grading Process Teaching Controversies: A New Approach to Computer-Assisted Instruction and Simulation Packages Simulating Demand in and Independent-Across-Firm Management Game Advertising Response in the Gold and Pray Algorithm: A Critical Assessment A Model for Pricing Decisions in First Period Marketing Simulation Games Jog Your Right Brain: An Exercise for the Classroom and for Research Six Thinking Hats: An Exercise to Combat Confusion and Develop Thinking Skills Communicating in Context: A Simulation for Learning Business Communication A Simulated Consulting Service for the Compete Marketing Simulation Game Action Exams in the Consumer Behavior Class Using Focus Groups to Teach Problem Definition in Basic Marketing Research The Use of Journals in Management Simulations: A Literature Review and an ABSEL Response An Initial Step Towards Developing and Using an Expert System with a Business Simulation Self-Managed Learning: An Experiential Course Design Using the QWL Paradigm A Review of Current Developments in Experiential Learning Bring the Real World into the Classroom Assessing Student Performance on a Business Simulation Experience Minimizing Startup Anxiety: Case Studies of Simulation Experiences A Tale of Two Shepards Or Using Simulation in a Class Without Walls Enhancing Business Simulations Through the Utilization of Experiential Activities Involving Local Community Executives Simulation with Integrated Spreadsheets: The Design and Development of a Conversational Marketing Concepts Decision Game Introducing INMART: An International Marketing Simulation Using a Computer-Based Business Plan Assistant in Conjunction with a Marketing Simulation Game Overview of the ABSEL Guide to Experiential Learning and Simulation Learning