COMPARISON OF PROBLEM-SOLVING TECHNOLOGIES: A FREE SIMULATION APPROACH Developments in Business Simulation & Experiential Exercises, Volume 9, 1982 236 COMPARISON OF PROBLEM-SOLVING TECHNOLOGIES: A FREE SIMULATION APPROACH Michael N. Chanin, Baruch College Harris J. Shapiro, Baruch College ABSTRACT The purpose of this paper is to present results of a comparison of three problem-solving technologies utilized in the context of a business game. First this paper discusses some methological issues of existing business games and existing problem-solving technologies. Second, the paper introduces the methodology, design and hypotheses of the present experiment. Third, the results of an empirical investigation into comparative effectiveness of High Structure-Dialectical Problem-Solving Technology (DPST), Medium Structure- Devil’s Advocate Problem Solving Technology (DAPST) and Low Structure-Control Group (LSPST) are analyzed. Finally, the need for further theoretical and empirical research is emphasized. INTRODUCTION In the last 10-15 years Business Gaming has been successfully established in the curriculum of American Business schools as a valuable and important teaching tool. Recently, together with a continuous development of general business games, more specialized games are being developed in areas of production, marketing, multinationals, etc. Computer Simulated Business games have become increasingly popular for the following reasons. First, they allow participants to take part in upper level managerial work and thus learn by a “hands on approach”. Second, the business game simulation lets the students make numerous decisions, changes in strategy and evaluate their outcomes without having to endure the actual consequences of such decisions in the real world. Third, almost instantaneous feedback permits simulation of several years of operations in a matter of several days, weeks or months thus increasing the level of participants’ motivation. Fourth, the business game requires the students to use integrative and systematic decision-making in dealing with a wide variety of organizational problems. Fifth, the business game coordinates presentation and application of diverse information and various managerial techniques to a wide variety of organizational areas. Finally, the business game provides unusual opportunities to learn through interaction, exchange of knowledge and communication with teammates. Nevertheless, at the present time there are still considerable deficiencies in business games and their methodology of teaching. First, there is a tendency to overemphasize the operational (quantitative) elements of the game without adequately addressing the qualitative aspects of it. As a result of this the business game often degenerates into a simplified accounting exercise. Second, only a few games provide an explicit formal organizational structure that allows the delineation and allocation of specific executive functions among members of a particular team. This usually leads to duplication of efforts, confusion and excessive and time-consuming arguments in the problem-solving process. Third, the performance in most business games is primarily based on the team’s achievement (ROI, Rank, etc.) and no individual input can be formally evaluated. The absence of personal responsibilities and corresponding rewards creates opportunities for some students to be passive and not to carry out their share of work. Finally, our examination of existing business games indicates that little if any attention is given to organization of decision-making processes through application of problem-solving technologies. As a result of this the problem-solving in majority of teams tends to be unorganized, spontaneous, inconsistent, counterproductive and inhibits the learning process and development of problem-solving skills in many students. Without having the necessary skills in group decision-making and given that students have different levels of knowledge and ability the problem-solving process very often degenerates into a situation where it is completely dominated by one or two power-oriented students leaving out other students from the process. The predominance of compromising and avoiding modes of behavior among students limits the potentials for learning and understanding because specific problems are analyzed only from a very narrow perspective of a few dominant students in the team. There is evidence suggesting that effectiveness of business games as a teaching tool can be considerably improved through development and application of specific problem- solving technologies. A substantial body of literature and research has been generated in the last 20 years on such problem-solving technologies as Brainstorming, Nominal Group Technique and Delphi (2,3,7,12). However, the research in this area is still in developmental stage and reflects basically the “consensus” or “equilibrium” school of organizational theory which views conflict an inherently undesirable, destructive and unacceptable. Only recently have a few problem- solving technologies such as Devil’s Advocate PST, Dialectical Inquiry PST and Dialectical PST made an explicit use of conflict in their constructs (4,8,13,15). These researchers not only accept conflict as inevitable, necessary and even desirable under specific organizational and environmental conditions but also assume that moderate levels of conflict can facilitate individual performance and ability to adjust to uncertainties of the changing environment. DAPST and DIPST are two problem-solving technologies most actively studied for financial prediction. Questioning the validity of the Dialectical Inquiry methodology as applied to the field studies, Cosier and his associates undertook a number of controlled laboratory studies, which proved DIPST to be inferior or at best as effective as the DAPST, and the Planning Expert Approach (6,5,16,17). In these studies the subjects were asked to make financial predictions, i.e., about price to earnings ratios based upon given cue values (information) about current ratios, inventory turnover and debt-equity ratios. The examination of the above mentioned studies shows Developments in Business Simulation & Experiential Exercises, Volume 9, 1982 237 not only the existence of methodological problems attributed to the limitations of the theoretical foundation of DIPST-the Hegelian dialectical methods but also the misconceptions about and misinterpretation of dialectical concepts. In addition there exist considerable shortcomings in operationalization and experimental design of studies using DAPST and DIPST. However, the major problem is that conditions of controlled laboratory settings are not suited for comparative studies on conflict-oriented problem-solving technologies. METHODOLOGY Procedure Taking into account the shortcomings of the previous research the present study has utilized an experimental technique known in the literature on laboratory experimentation as a free simulation technique Guetzkow (10). This technique departs radically from standard laboratory techniques by exposing participants to a considerably larger number of real-world-like events simultaneously. In addition, there exists a dialectical element in the free simulation technique where events that take place during the experiment are shaped, at least in part, by the changing behavior of the subjects themselves. Recognizing the value of the free simulation technique, Fromkin and Streufert stated that “one might say that experimenters who use this research method are creating field research in the laboratory” (9,p.423). Subjects and Design The subject pool was composed of 200 senior undergraduate students enrolled in the business policy course at Baruch College of the City University of New York. A total of six sections, three day and three evening, were utilized in the present study. The six sections of the business policy course were assigned as follows: two sections to DPST treatment, two sections to DAPST treatment, and two sections to control treatment. A total of 51 teams (27 “day student”) firms and 24 “evening student” firms) were randomly organized into six industries. Allocation (registration) of students into specific teams (firms) was also performed randomly. Description of the Simulation The present experiment is based on the following four major elements: (1) The Executive Game, developed by Henshaw and Jackson (11); strategic and operational planning; (3) formal organizational structure, and (4) utilization of DPST and DAPST. In the Executive Game up to nine firms are competing in the manufacture and sale of a single medium-technology product. The major objectives in this game are to achieve highest profits, return on investment, and the most important--the highest rank in the industry based on ROI. The game offers a dynamic business case, whose outcome is determined by the internal functioning of the top executives, external interaction of the competing firms, and prevailing economic conditions affecting the industry market potential. Although the computer program is essentially deterministic, the game itself involves a high degree of uncertainty, which stems not only from imperfect long range predictions of economic factors, but also from unpredictable and very often erratic behavior of competing firms. All teams were required to develop strategic and medium range plans and submit three annual and one final report Because of the nature of the product manufactured by all firms, the strategic planning time horizon was assumed to be three years, medium-range one year, and short- range (operational) one quarter. The experiment simulated 12 quarterly decisions which were made weekly with almost immediate performance feedback (computer printouts). All teams were allowed to revise the strategic plan and medium- range plans at the end of the first and second years. To facilitate the implementation of the firm’s long-range policies, a special organizational structure was developed in order to create a departmental (functional) type of organization in the experimental groups and an organic type of organization in the control groups. Four subjects were assigned to each team (firm) composed of a president and three vice presidents, with specifically designated executive functions. Teams were also allowed to change the organizational structure, which in the context of this simulation meant the selection of a new president and/or reassignment of executive functions. A concept of “vested interest” was incorporated into the experiment. This concept implies that problem-solvers participating in group decision-making have both high individual stakes in the outcome of these decisions and corresponding individual rewards and the ability to affect the results of the group decisions by exhibiting a high degree of persuasion and influence on the other teammates. Thus, the application of this concept permits the experiment to closely approximate the actual situational climate in the real business world. Therefore, the “vested interest” intensifies and strengthens the commitment of participants and also enhances the sense of realism, leading to an increase in the depth of the exposure of the subjects to the simulated decision- making process (1,18). Examination of the existing problem-solving technologies allows us to distinguish four common variables: task structure, procedure (protocol), conflict-handling methods and degree of control by decision-makers over the problem- solving process. The DPST can be viewed as a high structure problem-solving technology where task structure is high, procedure is highly specified, conflict-handling method is prescribed and degree of control is low. DAPST has a moderate task structure and the procedure only moderately specified. The conflict- handling method is generally outlined and degree of control is also moderate. The low structure technology utilized by the control group possesses low task structure, unspecified procedure, and conflict- handling method. However control over problem-solving is high. DPST Step 1 - Development of Individual (Conflicting) Plans The individual strategic or operational decisions are prepared by the participants independently under conflicting sets of assumptions. Each firm (group) consists of a president and three vice presidents. The vice presidents prepare three different conflicting strategic or operational plans by conjoining the same “data base” (accumulated information about their own and their competitor’s performance from the computer printouts) with different assumptions about the environment and different understanding of the business game. Step 2 Process of the Structured Debate The structured debate is conducted in the classroom. It is undertaken in two stages: a)first, each vice president presents his/her decision with the corresponding set of assumptions and policies for approximately three to four minutes (maximum total twelve minutes); b) second, after the presentation of the three plans is completed, a general discussion of each plan is undertaken. Pros and cons of each plan and corresponding assumptions are examined for approximately three to four minutes (maximum total twelve minutes). Developments in Business Simulation & Experiential Exercises, Volume 9, 1982 238 Step 3 - Synthesis-Development of a Final Group Plan In the last stage the participants agree on a final mutually acceptable set of assumptions and develop a strategic or operational plan (forecast). After the completion of the structured debate all members of the organization should agree on a joint set of assumptions and make a joint decision (maximum six minutes) ott the eight decision variables. This set of accepted assumptions is employed to develop the final strategic or operational decision (maximum 45 minutes). It should be noted that sometimes an individual’s plan will be accepted with only slight modification, but as a rule the final (joint) strategic or operational decision will differ from individual ones. The operationalization of DAPST, similar to Mason’s interpretation (1969), involves a four-step problem- solving process: (1) development of strategic and operational plans (forecasts); (2) plan presentation at the management briefing session; (3) management critique of the plan; and (4) development of a final plan. The control teams utilizing a low structure problem- solving technology (LSPST) were allowed to choose any set and sequence of activities, and also to have the opportunity (freedom) to develop a problem-solving procedure according to their needs and desires. The problem-solving technologies are the independent variables and dependent variables are the objective variables: Rot, rank, profit, sales, etc. Hypotheses Strategic planning with its dynamic, ill-structured and ill- defined type of problems necessitates a structured approach as an uncertainty reducing method. Recently Mintzberg et al. (14) and Nitroff et al. (15) have argued for a structured approach to unstructured decision-problems. Dialectical problem-solving technology (DPST) can be viewed as a member of a larger class of highly structured problem- solving technologies. It is presumed that the application of DPST will lead to the development and more successful implementation of strategic and operational plans in terms of objective performance. Hypothesis 1 Experimental groups employing DPST will tend to outperform DAPST and control groups in ranking, ROI and profit measures. Hypothesis 2 The application of DPST increases awareness and understanding of economic and planning problems; therefore subjects utilizing DPST will tend to outperform the DAPST and control groups in scoring on business game tests (indirect measure of performance). RESULTS We have already emphasized the need for a structured approach to unstructured strategic problems through a high or medium structure problem-solving technology. At the present time, “organizing” of the problem-solving process becomes very important because of the consistently increasing uncertainty in the organizational environment and inherent conflicting nature of different alternatives of a strategic plan. Hypothesis 1 stated that DPST groups will outperform DAPST and control groups on a number of objective measure of performance. In order to identify which of the three problem-solving technologies is superior, we trichotomized all teams into high, medium and low level of performance based upon the industry rank. The allocation of teams into high, medium and low levels of performance is interesting from the point of view of relative standing, but cannot be tested statistically. Therefore, in order to establish the advantage of a specific problem- solving technology, a number of t-tests were conducted for such variables as sales in units, profit, cost per unit, ROI, and absolute ranking. (See Table 1) Examination of results in Table 1 indicates that we may characterize the relationship between DPST and DAPST as well as between DAPST and DSPT as inconsistent in terms of statistical significance. However, in terms of actual performance the DPST group have done considerably better than other groups. The DPST groups together sold 635 thousand units, as compared to 599 thousand units sold by the DAPST groups (t1.69, p(.O5). The DPST group’s cost per unit ($27.64) was also lower than cost per unit in DAPST groups ($28.8O)(t=2.O, p<.01). The profit of DAPST groups ($808 thousand) was almost double of the control groups ($438 thousand) with t=2.1O at p.05. DPST groups earned $1. million in profit, as compared to $438 thousand earned by the control groups (t-2.1O, p<.05). Similarly the average ROI for a DPST was 14.36 percent, as compared to 12.2 percent for a control team (t=1.71, p .05). In Hypothesis 2 we stated that the very nature of DPST increases awareness and understanding of the problem through the structured debate and critical evaluation of various alternatives. Therefore it was expected that subjects in DPST groups will tend to outperform the DA- PST and control groups in scoring on the business game tests. Table 2 contains information on means, standard deviations, t-values and significance level for DPST, DAPST and control groups. The analysis of Table 2 shows that no significant differences were found between DPST and DAPST groups. However, both DPST and DAPST were significantly different (p<.05), one tail test) from LSPST. A comparison of problem-solving technologies on the basis of the different industries showed only one statistically significant difference between Industry 5 (DPST) and Industry 6 (control). Thus, Hypothesis 2 is supported only partially. These results not only support our previous findings that DPST is superior to LSPST, but also indicates that DAPST may have advantage over LSPST. Summarizing our findings for performance hypotheses, we may state the following: 1. There exists a stable and significant difference between DPST--a high structure conflict oriented technology and low structure technology (control) in terms of performance. 2. No stable sig- Developments in Business Simulation & Experiential Exercises, Volume 9, 1982 239 nificant relationships were found between DPST and DAP- ST on one hand and DAPST and LSPST on the other hand. However, DPST tends to outperform DAPST and DAPST tends to do better than LSPST on many parameters. DISCUSSION AND CONCLUSIONS This study shows the existence of a very strong and statistically significant difference between a high structure problem-solving technology (DPST) and a low- structure problem solving technology (LSPST). Even though the differences between DPST and DAPST on one hand DAPST and LSPST on the other were not consistently significant, there is some advantage of DPST over DAPST and DAPST over LSPST. Incidentally, post-research interviews with the students elicited the fact that DPST and DAPST students derived greater satisfaction than the LSPST students. We may identify two reasons for the lack of consistent superiority of DPST. First, the “Executive Game” may not be sufficiently complex to provide the necessary environment for strategic planning. Second, it is also possible that in the perception of the student subjects the level of conflict in DPST and DAPST is not adequately differentiated. In LSPST, because of the lack of externally generated structure there is a tendency by the informally emergent leaders to impose an internally generated structure. This may explain the inconsistencies between DAPST and LSPST. The ultimate test of any method, including dialectical, is practice and real life. We have to agree on this point with (6) who stated that before advice is offered to practitioners, the dialectical inquiry method should be further investigated. In order to establish the “base line” benefits of DPST the future research should refine the dialectical method and problem- solving technologies, and utilize a more diversified and comprehensive decision-making environment Business Game, NYU Management Game). It may be useful to test the conflict-oriented problem-solving technologies (DPST, DAPST) against non-conflict problem-solving technologies (Nominal Group Technique, Delphi). Finally, future research should account for levels and types of conflict behavior involved in problem-solving processes. Also, more control should be exercised in future studies over impact of such variables as personality traits, beliefs, motives, and needs of problem- solvers, as well as task roles, group norms and roles, leadership styles and so forth. REFERENCES 1. Baldwin, J.D. Influences detrimental to simulation gaming. American Behavioral Scientist, 12, 1969, pp. 14-21. 2. Bouchard, T.J., Jr. and M. Haire. Size performance and potential in brainstorming groups. Journal of Applied Psychology, 54,1, February 1969, pp. 1-29. 3. Carr, D.F., T.B. Green and R.E. Hughes. A comparison of interacting, nominal, pooled and brainstorming groups in the barrier identification phase of the problem-solving process. In L.R. Trueblood (Ed.), Proceedings, Southwest Division, Academy of Management, New Orleans, 1977, pp. 200-204. 4. Chanin, N.N. An empirical study of problem-solving technologies and conflict-handling behavior modes. Unpublished doctoral dissertation, The City University of New York, 1980. 5. Cosier, R.A. The effects of three potential aids for making strategic decisions on prediction accuracy. Organizational Behavior and Human Performance, 22, 2, 1978, pp. 295-306. 6. Cosier, R. Inquiry method, goal difficulty, and context effects on performance. Decision Sciences, 11, 1980, pp. 1-16. 7. Delbecq, A.L., A.H. Van de Ven, and D.H. Gustafson. Group Techniques for Program Planning: A Guide to Nominal Group and Delphi Processes. Glenview, Ill.: Scott, Foresman and Company, 1975. 8. Emshoff, J.R. and I.I. Mitroff. Improving the effectiveness of corporate planning. Business Horizons, October 1978, pp. 49-60. 9. Fromkin, M.L. and S. Streufert. Laboratory experimentation, In Marvin D. Dunnette (Ed.). Handbook of Industrial and Organizational Psychology. Chicago: Rand McNally, 1976. 10. Guetzkow, M. Ed. Simulation in Social Science: Engle- wood Cliffs, N.J.: Prentice Hall, 1962. 11 Henshaw, R.C. & J.R. Jackson. The Executive Game. Homewood, Ill.: Richard D. Irwin, Inc., 1978. 12. Herbert, T.T. and E.B. Yost. A comparison of decision quality under nominal and interacting consensus group format: the case of the structured problem. Decision Sciences, 10, 3, July 1979, pp. 358-370. 13. Mason, R.O. Dialectics in decision-making. A study in the use of counter-planning and structured debate in management information systems. Internal Working Paper No. 87. Soc. Sci. Project, Space Sci. Lab., Univ. of California, Berkeley, California, June 1968. 14. Mintzberg, H., D. Raisinghani and A. Theonet. The structure of “unstructured” decision process. Admin. Science Quarterly, 21, June 1976, pp. 246-275. 15. Mitroff, V.P., B. & R.H. Kilmann. The applic. of behav. and philos. tech, to strategic planning: a case stud of a large Fed. agency. Man. Sci., 24, 1, 1977, pp. 44-58-5 16. Schwenk, C.R. The effects of two strategy formulation aids: a lab, study involving a business case decision. Unpub. doc. disser. Indiana University, 1980. 17. Schwenk, C. & R.A. Cosier. Effects of the expert, Devil’s Advocate and dialec. inquiry meth. on predic. perf. Organiz. Behav. and Human Perfor., 26, 1980, pp. 409-424, 18. Stagner, R. & Rosen, H. Psy. of Union-Manage. Relations. Belmont, C.A. Wadsworth, 1969. Table of Contents Volume 9, 1982 The Value of Conjoint Analysis in Enhancing Experiential Learning The Effect of Participation in a Collective Bargaining Simulation on the Expectations and Attitudes of Union and Management Representatives Conflict Resolution in Experiential Learning The Use of Simulation to Test Theories of Bargaining in a Business Context The Nominal Group Technique: A Vehicle for Improving Case Method Courses A Framework for Developing a Business Policy Case Produce Your Own Video Cases for Classroom Use: A Demonstration An Experiential Effect from Charismatic Encounters Nine Topic Oriented Mini Simulations: Descriptions, Purposes, and Observations A Hospital Simulator (HOSPSIM) A Report of the Model and Results Expected from Field Testing An Evaluation of SLIM (A System Laboratory for Information Management) An Experiential Exercise Introducing Students to the Role Ambiguity Faced by Salespersons The Advertising Research Project as an Innovative Experiential Learning Technique The Small Group Research Project: An Experiential Learning Approach for Undergraduate Marketing Research Students The Relationship of Cognitive Style Maps to the Preference for Experiential Learning of Undergraduate Students The Learning Style Inventory Debate Revisited: An Empirical Assessment fo the Construct Validity Issue Related to Experiential Learning Theory Conducting a Classroom to Facilitate Career Goal Attainment Corporation Executives' Ratings of Policy Learning Techniques The Generation and Application of Evaluation Criteria for Management Policy and Strategy Simulation Games Experiential Opportunities with Microcomputers Leading Students to Learning: The Teacher's Obligation An Experiment in Teaching Principles Courses: The Mini Debate Formula Developing Creative Thinking Through Experiential Learning Heuristic and Systematic Evaluation of Policy: Exercise in Decision Making A Case Study Approach for the Litigation Decision: Employing Decision Analysis to Determine When a Business Should Settle or Go to Court Learning Negotiation Skills Through Simulation Union vs. Management: A Simulation of Collective Bargaining in Action Inside the Black Box: An Analysis of Underlying Demand Functions in Contemporary Business Simulations A Review of Channel Exercises and the Description of a New Alternative SIMCON I: A computer Based Simulation Model for Evaluating Physical Distribution Strategies Involving Order Consolidation Toward Competency-Based Management Education: The Interpersonal/Communications cluster The Value of Pre-Teaching in Role Playing Some Effects of Positive Personal Reinforcement upon Socializing Students in an Experiential Learning Course Who Gains and Who Does Not from Experiential learning Toward the Ultimate Experiential Exercise, the Student View Simulating Professional Writing Experiences in the Classroom The Johari Window as a Measure of Personal Development Windows Into Management: A Participative Aid to Learning Get Your Faculty Involved A Pedagogical Approach to Business Gaming for the Commuter Student Super Service for Computer Game Administrators Expand the Role of Simulation with Creative Scenarios The Delivery, Administration, and Evaluation of an Executive Development Program Using a Total Enterprise Business Game An Application of Experiential Learning in International Trade and Foreign Direct Investment International Management: Building Bridges Analysis of a Business Simulation Exercise: Organizational Survival and Success The FALRIS Organizational Scavenger Hunt Trainee V. Trainee Subordinates' Evaluation of Experiential Learning Longitudinal Analysis of an Innovative Teaching Intervention Student Perceptions of Effective Teacher Behaviors Revisited Realism and Learning: The Evolution of a Management Game A Merger and Acquisition Simulation A Stock Market Investor Simulation Experiential Learning in Consumer Behavior: Perceptual and Attitude Change Exercises The Recycling Industry Problems of Women in Management: A Role Playing Exercise for a Course in Contemporary Organizational Problems Experimental Three Weekend Course: Empirical Results A Process for the Analysis and Development of Course Content and Instructional Methodology for Large Class Sections Problems Associated with the Assessment of Experiential learning Using the Multiple Choice Test Combining Lecture and Simulation Teaching Methodologies Qualitative Determinants of Team Performance in a Simulation Game The Effects of Different Team Sizes on Business Game Performance Comparison of Problem-Solving Technologies: A Free Simulation Approach Consistency in Business Games A computer Simulation of Personnel Selection Decisions Giving Praise Exercise Job Enrichment A Look at the Spoken and Written Word in Organizations A Pattern of Group Communication A New Generation in Business Simulation Adapting Mainframe Business Simulations to Min Computers File Access is the Essential Prerequisite to Time-Flexible and Interactive Computer Simulation A Microcomputer Simulation for Teaching Retail Location Strategy Blocks & Chips: A computer-Assisted, Geno-Typical Entrepreneurial Game Development of a Self-Paced Course in Business Statistics The Involvement of Student Bodies in the Teaching of Advanced Technical Concepts Systems Learning Sequence: An Experiential Course Module for Management Information Systems An Experiential Approach to Developing Managerial Competencies Communication Research, Inc. An Experiential Learning Activity Developed as a Practicum for a Course in Organizational Communication