DO WE LEARN FROM EXPERIENCE? Developments in Business Simulation & Experiential Exercises, Volume 10, 1983 139 DO WE LEARN FROM EXPERIENCE? James W. Gentry, Oklahoma State University Alvin C. Burns, Louisiana State University Members of ABSEL are dedicated to the proposition that students can learn from experience. After all, “Experiential Learning” is part of the organization’s name. Even the relatively uneducated adhere to the notion of “trial and error” learning. The following quote, attributed to Confucius, has been cited I Hear, and I Forget I See and I Remember I Do and I Understand previously by ABSEL participants to express their conviction that “experiential learning’ is effective. Others have cited Sophocles’ quote from 400 B.C., “One must learn by doing the thing, for though you think you know it-you have no certainty, until you try. Or, one could quote George Santayana, “The great difficulty of education is to get experience out of ideas.” While I concur in general, the intent of this paper is to suggest that experience can lead to learning under the right conditions. Studies which found that experience did not lead to learning will be cited and discussed. The findings form this stream of research are more consistent with George Bernard Shaw’s view: “We learn from experience that we never learn from experience than they are with the quotes given earlier. One point to be made is that experiential learning should be “trial and error learning,’ with emphasis on the error. HOW DO WE LEARN? To the extent that we learn from trial and error,’ the learning is essentially inductive in nature. We experience certain situations and we generalize rules to ex plain what happened. Or as Kelly [17] described the process, The person who merely stands agog at each emerging event may experience a series of interesting surprises, but if he makes no attempt to discover the recurrent themes, his experience does not amount to much. It is when man begins to see the orderliness in a sequence of events that he begins to experience them.…From the standpoint of the psychology of personal. constructs, it is the learning which constitutes experience. Most learning occurs through outcome feedback--an action is taken and we observe the outcome. In many cases, we judge the quality of the decision by the favorableness of the outcome. Phrases such as “we reward productivity, not effort’ and “bottom-line management’ indicate emphasis on outcomes. Decision theorists (one example is Emery and Tuggle 1976) have frequently pointed out that outcomes frequently depend on factors outside the control of decision makers, and that we should evaluate the decision process rather than the outcome. In the long run, a good decision process should result in more profit; but this may not be true in the short run. This emphasis on process rather than outcome feedback has found its way into our approaches to teaching. When we grade cases, we are objective in that we say that the and empirical support that precedes it. In grading final recommendation is not as critical as the logical simulation games, we do not weight the entire game grade on the game results, but rather place quite a bit of emphasis on the students’ discussion of their game strategies and their justifications for the specific decisions. On the other hand, it is simpler to observe their game-end profit or their recommended case solution than it is to divine the process used to bring about these outcomes. Further, far less effort is required to critique the outcomes than to critique the process. Even if we as instructors provide process feedback, students may concentrate on the outcome feedback (grade) and ignore the process feedback (written comments). Most students have cane to expect consistency between the two types of feedback, and they are not easily placated when distinctions between the two are made. For example, in one play of a channels game, the manufacturers were the only members of the channel to end in black, although it was largely their responsibility that the other firms were inventory rich and demand poor. Even when the long run implications of this situation were pointed out, the manufacturers were upset that their performance grade was lower than some of the other firms. Students are not alone in finding the distinction between good/bad decisions and good/bad outcomes to be counter intuitive. Most people do. After all, we have a lifetime of experience in learning from outcomes. Outcomes are visible, available and often unambiguous; the process, however, often must be inferred on the part of the instructor. In many business contexts, process feedback is almost impossible. For example, salespersons perform cost of their duties outside the home firm’s environment; consequently, the sales manager must evaluate outcomes (sales totals) rather than process. Those process variables which are available (such as sales calls) often do not provide much insight into the sales process. So what if outcome feedback is so prominent we do learn from experience, don’t we? Of course we do, but not always so well as indicated by the next section of the paper. Most of our formal education process deals with deductive learning, as opposed to inductive learning. As instructors, we teach very abstract concepts and we attempt to make then context independent. In much of the education process in business, we teach optimal rules of decision making, and we further teach students to recognize problems as belonging to a class of problems having a given structure. This concentration on the decision process is something in which business educators can take pride; we are on the right track. ‘It is a truism that when presented with a problem, professionals view the problem within the structures they have been trained to see’ (6, 1981, p. 73]. However, it must be admitted that students leave without a systematic view of the problem area as their training has generally been within a narrowly defined content area. Consequently, it is common to see problems being forced into a structure which is not appropriate. Deductive learning, as typically observed in Developments in Business Simulation & Experiential Exercises, Volume 10, 1983 140 our business program, is not totally satisfactory. So the question remains, what’s wrong with inductive learning-- learning from experience?” HOW WELL DO WE LEARN FROM EXPERIENCE? There is evidence that training and experience have little impact on the quality of decisions made. Goldberg [10] found no difference between experienced clinical psychologists, student interns, and “off- the-street subjects in their ability to evaluate Bender-Gestalt protocols. Trumbo, Adams, Milner, and Schnipper [22] reported that experienced grain inspectors were no more accurate than less experienced inspectors in grading wheat. Goldberg [11], after reviewing over 20 such investigations, concluded that the amount of professional training and experience of the judge is not related to his judgmental accuracy. Brehmer [2] summarized his own program of research, as well as the studies of many others in the area, as showing that people do not learn optional strategies from experience even if they are given massive amounts of practice. After reviewing the cognitive biases found in student subjects, Slovic 1201 stated that there is no evidence to indicate that the biases are not present also in the decision making of business executives. As alluded to in the previous section, learning from experience is largely based on Outcome feedback. Because of the way feedback occurs and the methods that humans use to test rules via experience, positive reinforcement can occur even for incorrect rules [24]. More specifically, representation of outcomes in memory is thought to be often of categorical form--successes and failures, rather than absolute levels of the criterion [8]. Further research [15; 21; 23] indicates that people tend to focus on positive outcomes. In studies measuring subjects’ ability to judge the contingency between variables x and v from information in a 2x2 table such as Figure 1, people were found FIGURE 1 2x2 TABLE FORMAT x _ x y positive hit false positive _ y false negative negative hit to judge the strengths of the relationship by the frequency of positive hits while generally ignoring the other three cells. The implication of this finding is that people do not use all available outcome feedback even when it is presented systematically. Einhorn and Hogarth [6] conclude that one must pay attention to nonoccurences of the event as well as occurrences in order to develop a correct decision process. Without the search for disconfirming evidence, the development of decision processes will be based on the more visible and more memorable successes. Conditions do exist where decision makers have the chance to make many decisions in very similar circumstances; for instance, weather forecasting and race handicapping meet these conditions. In such situations, outcome feedback is quickly received and decision rules can be corrected relatively easily. In many, if not most, situations we do not have the opportunity to view the problem structure systematically. In fact, as Brehmer [21 points out, often we do things which will preclude getting the information we need to evaluate our judgment. For example, Einhorn [4] discusses the process of finding grant opportunities. Regardless of the validity of the judgement of value of the research application, it is clear that those who get grants are going to produce more and probably also better research than those who do not get grants. Thus the judgment process is confirmed. Action is always selective, meaning that we select certain proposals to be funded, at the same time we are selecting proposals that will not be funded. Therefore, the only outcomes which we can ob serve are the positive hits and the false positives. A recent study by Christensen-Szalanski. and Beach [3] provides insight into the specific nature of the experience needed to foster learning. Research investigating subjects’ ability to process information has consistently found that they do not process information optimally (i.e., they do not follow the dictates of Bayes Theorem). The Bayesian Model prescribes the optimal means of integrating sample information (in the form of likelihoods) with what has been learned from experience (in the form of prior probabilities also refers to as base rates). Research has found that subjects ignore the base rates, and make their posterior judgments based on the more vivid sample information. In a disease- diagnosis paradigm, Christensen-Szalanski and Beach [3] found that subjects ignore the base rates (only 7% of the population has contracted the disease) and concentrate on the sample information (a positive test result; with the test’s being correct 80% of the time). If one integrates properly the base rate information with the sample information, one should give a probability of about .23 that the individual has the disease. for ever, must subjects give estimates of about .80. In a second manipulation in the study, the subjects were shown the same slides indicating whether the patient had the disease (7 diseased out of 100 slides), but they were also shown the test results and were able to note that the majority of the patients with positive test results did not have the disease. The subjects who were shown the interaction between the base rates and the sample information were able to provide probability estimates much closer to the optimal level of .23. Thus, consistent with the conclusions of Einhorn and Hogarth 15], subjects were able to process the new information properly only after seeing evidence disconfirming the strong relationship between a positive test result and having the disease. However, most decision making tasks do not meet the conditions of seeing quick feedback or being able to have the decision task repeated systematically. Thus, most decision makers may not have the opportunity to search for disconfirming evidence even should they so want. Underlying the observed failure of people to learn from experience is their desire to develop simple, invariant decision rules in a world that is complex and stochastic. People desire rules of the nature “if A, then B.” Such rules are often easily supported by the observed evidence; in some cases, we find that even horrendously bad heuristics are supported by the occurrence of rare events which happen to be consistent with the individuals’ view of the relationship between the variables. Thus, failure to acknowledge the probabilistic nature of the world can lead to the development of poor decision rules. One problem area involves people’s failure to understand the nature of regression effects. Developments in Business Simulation & Experiential Exercises, Volume 10, 1983 141 Many of us are familiar with the concept of “regression to the mean,’ the statistical phenomenon that finds the observations of an extreme value to be followed by an observation much closer to the mean in the vast majority of cases. Yet we fail to apply this knowledge in real world settings. Kahneman and Tversky [16] discuss an example of a flight instructor developing the rule that you should punish poor performance but not reward good performance. That heuristic was developed after the instructor had observed that student pilots did better the day after a very poor performance and a good chewing out. On the other hand, the flight instructor had observed that after praising the student pilots profusely for exceptionally good performances, they would invariably do worse the next day. Thus the failure to recognize the existence of regression effects led to the development of a very indefensible decision rule. Translation of the example above to a more common business situation is straightforward: a manager, faced with extremely poor short run results, takes a particular action and observes that things get better. Thus a causal relationship is developed in that manager’s mind, even though there is a good possibility that things would have gotten better had s/he done nothing. Frau a societal view, this improper learning from experience may be more disgruntling in the case of medical decision making. Given the results of many studies on clinical inference, the ability of medical personnel to diagnose who is and who is not in need of treatment is questionable.1 Thus, doctors often will find that new therapies are successful, as the patient gets better. To the extent that what is happening is a regression effect, placebos would have the same effect as the new therapies. It is understandable how practices such as “bleeding” patients were once standard procedures; doctors learned (inappropriately) fret experience that the approach worked. IMPLICATIONS FOR EXPERIENTIAL LEARNING The point of the discussion to this point is not that we do not learn from experience, but rather that we frequently do not learn well from experience. Much of our learning is based on outcome feedback, and few trials are involved. Thus we may mistakenly believe that we have developed a good decision rule, because it worked well in a couple of isolated Instances. The purpose of experiential learning is to give students practice in a simulated decision task. Few experiential exercises, simulation games, or cases offer the student repeated trials under similar circumstances. Thus, the search for disconfirming evidence is frequently not feasible. To the extent possible, we should attempt to alter that situation. Students are not likely to consent to repeat the same basic task under similar conditions a number of times, so individual learning from outcome feedback is not likely to be systematic. One alternative is that recommended by Greenhalgh and Meslin [12] which required students to rate the favorability of various outcomes, Conjoint analysis is then used to help the students learn which criteria are more important to them. While the approach does not relate outcomes with decisions directly, it does provide the student with a far better perspective as to the variety of possible outcomes. Even when using a systematic approach to the problem structure, the debriefing session for the particular exercise is very critical. 1 An employee of the Oklahoma Health Planning Commission told one of the authors that most (80%) illnesses will go away without any treatment. Emphasis on the debriefing session is a commonly stated guideline, but our reasons for supporting this emphasis have a somewhat different tone than usually found. Previous ABSEL sessions on debriefing [1; 14) have stressed the need for the creation of a positive atmosphere. This recommendation of a positive atmosphere for debriefing may stem from belief such as that expressed by Greenhalgh and Neslin (12), “Learning from experience.., is time- consuming and usually involves only negative feedback which is non-directive.” While it is agreed that a warm atmosphere is crucial so that no student has his/her ego grossly deflated, more emphasis should be placed on the negative outcomes and on making the negative feedback more directive. We are dealing with trial and error learning, and we should put more emphasis on the error. To be sure, examples of good decision processes should be presented and stressed in experiential exercise de briefings, simulation game summaries, and case discussions. In addition, examples of poor decision processes should be stressed, especially when it is giving students the opportunity to discover the inadequacy of improper decision rules through the presentation of disconfirming evidence. REFERENCES [1] Altman, Steve (1978), “A Kiss Before Debriefing,’ Proceedings, ABSEL, Denver, 5-6. [2] Brehmer, Berndt (1980), “In One Word: Not Fran Experience, Acta Psychologica, 45, 223-241. [3] Christensen-Szalanski, Jay J. J. and Beach, Lee Roy (1982), “Experience and the Base-Rate Fallacy,” Organizational Behavior and Human Performance, 29 (April), 270-278. [4] Einhorn, Hillel J. (1982), “Learning from Experience and Suboptimal Rules in Decision Making,” in Daniel Kahneman, Paul Slovic, and Amos Tversky, Judgment Under Uncertainty: Heuristics and Biases, New York: Cambridge University Press, 268-286. [5] Einhorn, Hillel J. and Hogarth, Robin M. (1978), “Confidence in Judgment: Persistence of the Illusion of Validity,” Psychological Review, 85, 395-4 16. [6] Einhorn, Hillel J. and Hogarth, Robin M. (1981 ), “Behavioral Decision Theory: Processes of Judgment and Choice, Annual Review of Psychology, 32, 53- 88. [7] Emery, Douglas R. and Tuggle, Francis D. (1976), “On the Evaluation of Decisions,” MSU Business Topics, 24 (No. 2 Spring), 42-48. - [8] Estes, W. K. (1976), ‘The Cognitive Side of Probability Learning,” Psychological Review, 83, 37-64. [9] Gaeth, Gary J. and Shanteau, James (1981 ), “Training Expert Decision Makers to Ignore Irrelevant Information: A Comparison of Lecture and Interactive Training Procedures,” Psychology Report #811, Kansas State University. [10] Goldberg, L. R. (1959), “The Effectiveness of Clinicians’ Judgments: The Diagnosis of Organic Developments in Business Simulation & Experiential Exercises, Volume 10, 1983 142 Brain Damage from the Bender-Gestalt Test,” Journal of Consulting Psychology, 23, 25-3 3. [11] Goldberg, L. R. (1967), “Research on the Clinical Judgement Process,’ Proceedings, N.I.P.P. Section on Clinical Psychology, Nijmegen, Netherlands. [12] Greenhalgh, Leonard and Scott A. Neslin (1981), “The Value of Conjoint Analysis in Enhancing Experiential Learning,” Journal of Experiential Learning and Simulation, 3 (#3-4, December), 173- 179. [13] Hogarth, Robin M. (1982), On the Surprise and Delight of Inconsistent Responses,’ in Robin M. Hogarth (ed.), Question Framing and Response Consistency, San Francisco: Jossey-Bass, Inc., 9 1- 103. [14] Hunsaker, Phillip C. (1978), Debriefing: The Key to Effective Experiential Learning, Proceedings, ABSEL, Denver, 5-6. [15] Jenkins, H. M. and Ward, W. C. (1965), “Judgment of Contingency Between Responses and Outcomes,” Psychological Monographs: General and Applied, 79, Whole No. 594. [16] Kahneman, Daniel and Tversky, Amos (1973), “On the Psychology of Prediction,” Psychological Review, 80, 237-251. [17] Kelly, George A. (1955), The Psychology of Personal Constructs, Vol. 1, New York: N. W. Norton and Co. [18] Shanteau, James (1979), “Information Integration Theory Applied to Training of Experts,” Proceedings, AIDS, New Orleans. [19] Shanteau, James and Gaeth, Gary J. (1979), “Training Expert Decision Makers to Ignore Irrelevant Information,” Proceedings, Psychonomic Society, Phoenix. [20] Slavic, Paul (1972), “Fran Shakespeare to Simon: Speculations--and Some Evidence--About Man’s Ability to Process Information,” ORI Research Monograph, 12 (No. 2). [21] Smedslund, J. (1963), ‘The Concept of Correlation in Adults, Scandinavian Journal of Psychology, 4, 165- 173. [22] Trumbo, D., Adams, C., Milner, M., and Schnipper, L. (1962), “Reliability and Accuracy in the Inspection of Hard Red Winter Wheat, Cereal Science day, 7, 62-71. [23] Ward, W. C. and Jenkins, H. M. (965), “The Display of Information and the Judgment of Contingency,” Canadian Journal of Psychology, 19, 231-241. [24] Wason, P. C. (1960), “On the Failure to Eliminate Hypotheses in a Conceptual Task,” Quarterly Journal of Experimental Psychology, 12, Table of Contents Volume 10, 1983 Is the Computerized Business Simulation Relevant? Business Professionals Play a Student Game A Methodology For Assessing the Internal Validity of Business Simulations A Longitudinal Study of the External Validity of a Business Management Game Policy Analysis and Decision: A corporate Relocation Simulation Exercise Effective Listening: An Exercise in Managerial Communication The Symbol Exercise: An Initial Group Activity Concept Based Simulations Institutional Users of Experiential Learning Packages: A Preliminary View from Publishers' Adoption Lists How to Use Business Games in the Business Policy Course: The Students' Perspective Professors' Ratings of Business Policy Learning Methods Moot Trial: An Exercise in Trial Procedure and Evidence The Advertising Agency Game: An Experiential learning Exercise The Use of Videotaped Cases in Teaching Information Acquisition and Decision-Making Skills Experiencing Information Processing Strategies as a Means to Explore Decision Making Importance Ratings and Operations Data as Predictors of Business Game Performance Determinants of Performance in Computer Simulations Predicting Business Game Performance form Perceptions of Manager Information and actions Business Consulting: A Practicum for Undergraduate Internship as a Contingency Based Experiential Learning Program for More Effective Organizational Socialization: A Conceptual Framework MANSYM III Decision Support System Demonstration Learning the Concept of market Value Through Simulation Conflict Management for Economic Developers Development of Data Analysis Units Designed to Enhance Reasoning and Knowledge Transfer in the College Level Course BOSS: A Behavioral-Quantitative, Computer-Supported Game Entrepreneurial Potential: An Experiential Exercise in Self Analysis and Group Assessment Development of Strategists: Simulated Cases BANKRUPT: A Deceptively Simple Business Strategy Game Simulating Market and Firm Level Demand - A Robust Demand System COMPSIM A Computer Center Management Simulation Hiving Model: Assessing Management Skill Awareness The Johari Window, A Reconceptualization Role-Playing Based on Video-Tape Scenarios: An Application of Modeling to Building Supervisory Skills How to Internationalize Your Curriculum A Computerized Model of Human Behavior in a Total-Firm Management Simulation the Worksheet Approach for Simulation Game Strategy Analysis Teaching Competitive Bidding Using a DSS Generator Do We Learn from Experience? The Use of Theory Power for Increased Research Momentum in Business Simulation and Experiential Exercises Research Report on Programmatic Research on Perceived Learning Barriers with Simulation and Experiential learning An Empirical Examination of Conflict - and Nonconflict - Oriented Problem-Solving Technologies Management Curriculum: 1982 Humor as a Management Tool: Use in Formal Game Presentations Student Behavioral Change Through Teacher behavioral Change