A COMPUTER SIMULATION OF PERSONNEL SELECTION DECISIONS Developments in Business Simulation & Experiential Exercises, Volume 9, 1982 243 A COMPUTER SIMULATION OF PERSONNEL SELECTION DECISIONS Dallas T. DeFee, State University of New York at Binghamton ABSTRACT This paper describes a computer simulation of personnel selection decisions. The performance of an individual decision maker is compared to a Normative Decision Model (NDM) in a computerized simulation designed to reveal systematic patterns of bias (if they exist), the conditions under which these develop, and methods by which choices can be improved. The primary objective of the simulation is to generate a set of empirical generalizations regarding the functional relationship between patterns of choice behavior and decision parameters such as base rate, judgement accuracy, knowledge of results, and cost of errors. These generalizations and supporting protocol analysis may be grist for developing a process model of human choice behavior. A secondary objective is to explore ways in which choice behavior may be improved by exposing decision makers to graphical feedback regarding cost-benefit tradeoffs. INTRODUCTION Behavioral decision theory has become an area of vigorous research concerned both with the description and improvement of human judgement (Slovic, Fischhoff, and Lichtenstein, 1977). Improving the quality of human judgement requires an understanding of the underlying processes of human information processing, a knowledge of the organizational context in which decisions are made, and an appreciation of the substantive problems which elicit choice behavior. The need for decision makers to confront risky and difficult tradeoffs may well be met only by overconfidence in fallible judgements (Einhorn and Hogarth, 1978). The selection decision in personnel administration is one occasion for choices which are both socially visible and often difficult. A NORMATIVE DECISION MODEL The literatures on decision theory and human information processing identify several models presumed to be paramorphic representations (Hoffman, 1960) of human judgement. Anderson (1974) illustrates how relatively simple algebraic models can account for variety of perceptual and judgement phenomenon. Hammond (1977) has developed a social judgement theory, which as noted by Slovic et al, uses the Brunswik lens model as the framework far describing how environmental cues can be used to predict judges responses. Regression models in particular have become very popular in recent research (Dawes and Corrigan, 1974). These models have both power and parsimony as descriptions of how information is combined to yield overall evaluative judgements but have been criticized, however, on the grounds that they reveal little about the process of human judgement (Zeleny, 1976). For the most part, these are models of human judgement, i.e. information selection, filtration, and integration, rather than models of choice behavior per se (Slovic et al, 1977). A Normative Decision Model has been described in its most general form by Einhorn and Schacht as an “analytical framework for dealing with questions regarding the validity of human adjudgement” (1977, p. 126). Essentially the same model, however, has been used in the context of the personnel selection decision for some time (Casio, 1978; Sands, 1973). In general, the model deals with the following prototype decision situation: an administrator is faced with a set of either-or action choices. An evaluative judgement (x) has been made which represents the overall assessment of the person or object under consideration. Also, there is an implied cutoff on the judgement such that if the judgement exceeds a critical value then action A will be taken, else not. Thus x is assumed to be a continuous variable which serves as a predictor of some future outcome, y. which is also a continuous variable. The y variable is a criterion for the efficacy of the judgement x, a measure of subsequent performance in a personnel context or, more generally, an objective standard of comparison. In most situations, of course, the criterion variable does not exist at the time a decision is made; if it did, there would be no need for the judgement. Thus the original judgement must await validation until more information is revealed, or until the consequences of action can be discerned. In this formulation, the criterion variable is also assumed to yield a dichotomous outcome, a value of x greater than some cutoff represents a successful outcome, a value less than the cutoff a failure. This the overall model specifies four action-outcome combinations as illustrated in the Figure. Model Parameters The Normative Decision Model as outlined here has four structural parameters and two “contextual” parameters. The structural parameters are: (1) the number of discrete action choices which must be made, i.e., "sample size” or “opportunity set”, (2) judgement accuracy in the form of the correlation between predictor and criterion, (3) base rate or the naturally occurring rate of successful outcomes, and (4) selection ratio or the proportion of positive action choices. The two contextual parameters are needed to make the model a realistic representation of choice behavior: (1) costs of both types of error in an action choice, and (2) the pattern of prior outcomes in terms of success or failure. The specification of this model thus assumes unambiguous actions and outcomes, estimable cost functions and a predictor-criterion relationship as well as a situation involving on-going choice behavior. In addition, the model is made more tractable by assuming predictor (judgement) and criterion (performance) variables have a bivariate normal distribution. The Figure illustrates the model’s structural parameters. Model Characteristics Note that the Normative Decision Model exhibits the following characteristics: Explicit considerations of cost tradeoffs between two types of predictive error, i.e. the classic distinction between Type I and Type II errors. The task of the decision maker is to choose a cutoff score such that overall costs are minimized. Systematic departure from this optimum cutoff is a “risk perference”, “bias”, Developments in Business Simulation & Experiential Exercises, Volume 9, 1982 244 or a violation of the norm to maximize expected utility. Probabilistic outcomes. Any single choice may be optimal given a set of specific values, yet nonetheless produce unfavorable outcomes. Imperfect judgement will preclude perfect prediction. Similarly, good results may be due to chance rather than optimal choice. Single-stage choice. The decision maker has no opportunity to reconsider after new information is available, nor can choice be postponed. Stable preferences and parameters. The model assumes that the outcome preferences of the decision maker are not only measurable, but consistent across tine. The functional relationships among parameters are stationary. It should be clear that model given above is a model of individual choice rather than a model of individual judgement. The model operates after the individual decision maker has made an evaluative judgement (or a series of evaluative judgements) and is faced with the implicit dilemma of selection from among alternative actions. Further, the model reveals that two types of deviation from a rational norm are possible. First, the decision maker may consistently overweight the harm from a incorrect positive choice and thus have a “failure-avoiding” bias which is associated with a cutoff score lower than optimal. Alternately, the decision maker may overweight the importance of an incorrect negative choice and thus have a “success-seeking” bias. Of course these two types of bias could be thought of in “benefit” terms as well as “costs” terms and, indeed, the model operates exactly the same whether the problem is one of benefit maximization or costs minimization. As a psychological problem, however, there may be strong effects associated with a costs versus a benefit focus. THE COMPUTER SIMULATION Whether this model functions well as a descriptive model has not been tested directly. The evidence suggests not; individual decision makers will ignore base information (e.g. Lyon and Slovic, 1976; Tversky and Kahneman, 1974), avoid examination of the negative hit rate, (Einhorn and Hogarth, 1975), reveal both insufficient adjustment to new information and anchoring bias (Tversky and Kahneman, 1974). The model is far from being as comprehensive or realistic as a formal model could be (Rapoport and Burkheimer, 1971). Nonetheless, the given model is both representative of a variety of choice situations (Einhorn and Hogarth, 1978) and simple enough to be implemented as a flexible and easy to use computer simulation. The simulation presents the user with a prototype decision situation as described in the Normative Decision Model. It gives the user feedback regarding the outcomes of a series of hypothetical choices. The context of decision is currently that of a “personnel director” screening job applicants. The parameters of the Normative Decision Model can be varied systematically by the user while patterns of choice behavior are recorded. The interactive computer terminal allows both “self pacing” and automatic data collection and feedback. BENEFITS OF SIMULATION Individual differences among decision makers in such variables a statistical knowledge (Schoemaker, 1979), general aptitude/skill, tolerance for stress/ambiguity, motivation level, and familiarity with the area in which choices are made (Meister, 1976) will all affect patterns of choice. Their personal factors, however, may be dominated by task and situational variables (e.g. Payne, 1976), which can be systematically varied in the simulation. This allows the user to gain an immense amount of experience with possible choice outcomes. Prior outcomes from choice have a strong influence on current choice behavior. Data from Staw and Ross (1978) indicate that in resource allocation decisions prior failure is likely, on balance, to encourage a decision maker to rationally evaluate informational cues and objectively review potential outcomes; to behave, in short, with a heightened rationality characteristic of what Staw and Ross call prospective focusing. Under conditions of unambiguous negative feedback, both success-seeking and failure- avoiding bias will be reduced. Another area for significant learning with the simulation is the interaction of base rate, relative cost of Type I and Type II error and systematic bias in choice. The effects of probable gains or losses may be crucial in choice behavior if recent theorizing by Kahneman and Tversky (1979) is even partially correct. Kahneman and Tversky offer an alternative to expected utility theory which they call “prospect theory’, which presumably explains some of the inconsistencies that have been repeatedly observed between human behavior and utility theory. In prospect theory, a separate value is assigned to both gains and losses rather than a single utility to the outcomes of a choice. There are two utility functions; one for gains which is concave and one for losses which is convex and, in addition, steeper than the gain function. This non-symmetrical effect may produce clear-cut patterns of choice bias for situations with either very high or low base rates and large differences between the cost of Type I versus Type II errors. Individuals using the simulation can clearly see the effects of such a choice bias. Finally, one of the central features of the simulation is that it graphically shows how the explicit consideration of risk benefit tradeoffs is a prerequisite to sound choice. The evidence to date (Tversky 1972; Slovic 1975) suggests that decision makers will rely on procedures which are easy to justify, explain and defend, This implies that choices are often based on constructing good justification, i.e., an explanation which can be persuasively defended despite any poor or unexpected outcome. Slovic et al concluded that if this rationalization hypothesis has merit, then “research is vital for teaching us how to communicate risk-benefit and other valuable analytic concepts in ways that would enable such material to be woven into the fabric of convincing justifications” (1977, p. 30). The simulation addresses this issue in two ways; first, by determining if individuals routinely deviate from optimal choice by ignoring or overestimating the effect of low probability but highly salient outcomes, and second, by determining if the graphical depiction of risk-benefit tradeoffs influences the content of specific choices. Developments in Business Simulation & Experiential Exercises, Volume 9, 1982 245 REFERENCES [1] Anderson, N. H. Algebraic Models in Perception. In Handbook of Perception. (New York: Academic Press, 1974). [2] Casio, W. Applied Psychology in Personnel Management. (Virginia: Reston Publishing, 1978). [3] Dawes, R. M. and B. Corrigan, Linear Models in Decision Making. Psychological Bulletin, 1974, 81, pp, 95-106. [4] Einhorn, H. J. and R. M. Hogarth, Confidence in Judgement: Persistence of the Illusion of Validity. Psychological Review, 1978, 85, pp, 395, 416. [5] Einhorn, H. J. and S. Schacht, Decisions Based on Fallible Clinical Judgement. In N. F. Kaplan and S. Schwartz (Eds.) Human Judgement and Decision Processes in Applied Settings. (New York: Academic Press, 1977). [6] Hammond, K. R., J. Rohrbaugh, J. Mumpower, and L. Adelman, Social Judgement Theory: Applications in Policy Formation. In M. F. Kaplan and S. Schwartz (Eds.) Human Judgement and Decision Processes in Applied Settings. (New York: Academic Press, 1977). [7] Hoffman, P. J., The Paramorphic Representation of Clinical Judgement. Psychological Bulletin, 1960, 57, pp. 321-344. [8] Kahneman, D., and A. Tversky, Prospect Theory: An Analysis of Decisions Under Risk. Econometrica, 1979, 47, pp. 263-291. [9] Lyon, D. and P. Slovic, Dominance of Accuracy Information and Neglect of Base Rates in Probability Estimation. Acta Psychologica, 1976, 40, pp. 287-298. [10] Schoemaker, P. J., The Role of Statistical Knowledge in Gambling Decisions: Moment vs Risk Dimension Approaches. Organizational Behavior and Human Performance, 1979, 24, pp. 1-17. [11] Payne, J. W., Task Complexity and Contingent Processing in Decision Making: An Information Search and Protocol Analysis. Organizational Behavior and Human Performance, 1976, 16, pp. 366-387. [12] Rapoport, A., C. J. Burkheimer., Models for Deferred Decision Making. Journal of Mathematical Psychology, 1971, 9, 508-538. Developments in Business Simulation & Experiential Exercises, Volume 9, 1982 246 [13] Sands, W. A. A Method for Evaluating Alternative Recruiting-Selection Strategies: The CAPER Model. Journal, of Applied Psychology, 1973, 57, pp. 222- 227 [14] Slovic, P. Choice Between Equally-Valued Alternatives. Journal of Experimental Psychology, 1975, 1, pp. 280- 287. [15] Slavic, P., B. Fischhoff, and S. Lichtenstein. Cognitive Processes and Societal Risk Taking. In J. S. Carroll and J. W. Payen (Eds.) Cognition and Social Behavior. (Hillsdale, New Jersey: Erlbaum, 1976). [16] Slovic, P., B. Fischhoff, and S. Lichtenstein. Behavioral Decision Theory. Annual Review of Psychology, 1977, 28, pp. 1-39. [17] Staw, B. M., and J. Ross. Commitment to a Policy Decision: A Multi-Theoretical. Perspective. Administrative Science Quarterly, 1978, 23, pp. 40-64. [18] Tversky, A. Elimination by Aspects: A Theory of Choice. Psychological Review, 1972, 79, pp. 281-299. [19] Tversky, A., and D. Kahneman. Judgement Under Uncertainty: Heuristics and Biases. Science, 1974, 185, pp. 1124-1131. Table of Contents Volume 9, 1982 The Value of Conjoint Analysis in Enhancing Experiential Learning The Effect of Participation in a Collective Bargaining Simulation on the Expectations and Attitudes of Union and Management Representatives Conflict Resolution in Experiential Learning The Use of Simulation to Test Theories of Bargaining in a Business Context The Nominal Group Technique: A Vehicle for Improving Case Method Courses A Framework for Developing a Business Policy Case Produce Your Own Video Cases for Classroom Use: A Demonstration An Experiential Effect from Charismatic Encounters Nine Topic Oriented Mini Simulations: Descriptions, Purposes, and Observations A Hospital Simulator (HOSPSIM) A Report of the Model and Results Expected from Field Testing An Evaluation of SLIM (A System Laboratory for Information Management) An Experiential Exercise Introducing Students to the Role Ambiguity Faced by Salespersons The Advertising Research Project as an Innovative Experiential Learning Technique The Small Group Research Project: An Experiential Learning Approach for Undergraduate Marketing Research Students The Relationship of Cognitive Style Maps to the Preference for Experiential Learning of Undergraduate Students The Learning Style Inventory Debate Revisited: An Empirical Assessment fo the Construct Validity Issue Related to Experiential Learning Theory Conducting a Classroom to Facilitate Career Goal Attainment Corporation Executives' Ratings of Policy Learning Techniques The Generation and Application of Evaluation Criteria for Management Policy and Strategy Simulation Games Experiential Opportunities with Microcomputers Leading Students to Learning: The Teacher's Obligation An Experiment in Teaching Principles Courses: The Mini Debate Formula Developing Creative Thinking Through Experiential Learning Heuristic and Systematic Evaluation of Policy: Exercise in Decision Making A Case Study Approach for the Litigation Decision: Employing Decision Analysis to Determine When a Business Should Settle or Go to Court Learning Negotiation Skills Through Simulation Union vs. Management: A Simulation of Collective Bargaining in Action Inside the Black Box: An Analysis of Underlying Demand Functions in Contemporary Business Simulations A Review of Channel Exercises and the Description of a New Alternative SIMCON I: A computer Based Simulation Model for Evaluating Physical Distribution Strategies Involving Order Consolidation Toward Competency-Based Management Education: The Interpersonal/Communications cluster The Value of Pre-Teaching in Role Playing Some Effects of Positive Personal Reinforcement upon Socializing Students in an Experiential Learning Course Who Gains and Who Does Not from Experiential learning Toward the Ultimate Experiential Exercise, the Student View Simulating Professional Writing Experiences in the Classroom The Johari Window as a Measure of Personal Development Windows Into Management: A Participative Aid to Learning Get Your Faculty Involved A Pedagogical Approach to Business Gaming for the Commuter Student Super Service for Computer Game Administrators Expand the Role of Simulation with Creative Scenarios The Delivery, Administration, and Evaluation of an Executive Development Program Using a Total Enterprise Business Game An Application of Experiential Learning in International Trade and Foreign Direct Investment International Management: Building Bridges Analysis of a Business Simulation Exercise: Organizational Survival and Success The FALRIS Organizational Scavenger Hunt Trainee V. Trainee Subordinates' Evaluation of Experiential Learning Longitudinal Analysis of an Innovative Teaching Intervention Student Perceptions of Effective Teacher Behaviors Revisited Realism and Learning: The Evolution of a Management Game A Merger and Acquisition Simulation A Stock Market Investor Simulation Experiential Learning in Consumer Behavior: Perceptual and Attitude Change Exercises The Recycling Industry Problems of Women in Management: A Role Playing Exercise for a Course in Contemporary Organizational Problems Experimental Three Weekend Course: Empirical Results A Process for the Analysis and Development of Course Content and Instructional Methodology for Large Class Sections Problems Associated with the Assessment of Experiential learning Using the Multiple Choice Test Combining Lecture and Simulation Teaching Methodologies Qualitative Determinants of Team Performance in a Simulation Game The Effects of Different Team Sizes on Business Game Performance Comparison of Problem-Solving Technologies: A Free Simulation Approach Consistency in Business Games A computer Simulation of Personnel Selection Decisions Giving Praise Exercise Job Enrichment A Look at the Spoken and Written Word in Organizations A Pattern of Group Communication A New Generation in Business Simulation Adapting Mainframe Business Simulations to Min Computers File Access is the Essential Prerequisite to Time-Flexible and Interactive Computer Simulation A Microcomputer Simulation for Teaching Retail Location Strategy Blocks & Chips: A computer-Assisted, Geno-Typical Entrepreneurial Game Development of a Self-Paced Course in Business Statistics The Involvement of Student Bodies in the Teaching of Advanced Technical Concepts Systems Learning Sequence: An Experiential Course Module for Management Information Systems An Experiential Approach to Developing Managerial Competencies Communication Research, Inc. An Experiential Learning Activity Developed as a Practicum for a Course in Organizational Communication