DEVELOPING AND USING WEIGHTED APPLICATION BLANKS: AN EXPERIENTIAL EXERCISE Developments in Business Simulation & Experiential Learning, Volume 11, 1984 120 DEVELOPING AND USING WEIGHTED APPLICATION BLANKS: AN EXPERIENTIAL EXERCISE Donald B. Dawson, Northern Illinois University Albert S. King, Northern Illinois University Ralph F. Catalanello, Northern Illinois University ABSTRACT Development and use of effective weighted application blanks begins with an analysis of procedures required in the developmental process. An experiential exercise is presented which gives a “hands-on” experience in developing and using weighted application blanks. The exercise was developed to aid conceptual and operational understanding of procedures in weighted application blank development. The hypothetical data used were developed solely for the purposes of the exercise. Upon completion, the individual should have a keen insight into the development and use of weighted application blanks. INTRODUCTION Learning Objectives 1) To understand the logic and systematic procedures for developing weighted application blanks (WABS). 2) To give the individual a “hands-on” experience in developing WABS. 3) To develop an awareness of the possible uses, implications, and ramifications of using WABS. Materials Needed: 1) Pencil and calculator. General Instructions: 1) Read the overview prior to beginning the exercise. 2) Read the introductory material for each section as you come to that section. Then, follow specific instructions given for each section. NOTE: Each section builds on previous ones, hence it is important to understand each section before moving on. OVERVIEW Selection and placement of individuals is one of the most difficult problems facing an organization. Improper (i.e., discriminatory) selection and placement can have far-reaching repercussions. Selection and placement techniques have been developed and refined that give management a foundation for basing selection decisions. One of these selection techniques is the Weighted Application Blank (WAB) [3]1. A WAB ". . . provides one systematic method for determining which personal factors are important in specific occupations and how to use them in selection,” [3, p. 5]. The WAB 1 Much of the material presented has been adapted from Development and Use of Weighted Application Blanks, by George W. England, Rev. Ed., Industrial Relations Center, University of Minnesota, 1971. attempts to identify what personal factors reliably differentiate groups of potential desirable and undesirable employees. By identifying personal factors of job applicants, a rapid screening of the applicants is possible. The results, along with other information, can be used to enhance the selection and placement of individuals in organizations.2 There are numerous studies that statistically support the notion that WABs are successful in differentiating potential desirable and undesirable employees [2; 6]. One study found that even in relatively low-level jobs with modest hiring and training costs, a large savings was obtained by using a WAB [4]. The potential to an organization for proper use of a WAB is not however, without limits. One problem is that WABS tend to lose predictive efficiency over time. Another obstacle is that practitioners and researchers simply do not know how to design and develop WABS. An experiential exercise which presents a step-by-step method for development may foster continued emphasis on their use and promote renewed interest in successful selection and placement techniques. The procedure for developing a WAB is straightforward:3 1) Choose an appropriate criterion. 2) Identify criterion groups. 3) Select application blank items to be analyzed. 4) Specify item response categories. 5) Determine item weights. 6) Analyze independent samples for cross- validation. 7) Set WAB scores for selection. 2 The results obtained through the introduction of a WAB should not be the sole basis for making selection decisions. See, Pace, Larry A. and Lyle F. Schoenfeldt, “Legal Concerns in the Use of Weighted Applications,” Personnel Psychology, Vol. 30 (1977), pp. 159-16 6. 3 This is the basic procedural outline as indicated in England (1971), however, some steps have been modified for this exercise. Developments in Business Simulation & Experiential Learning, Volume 11, 1984 120 A DILEMMA A company that employs a large number of people is faced with a problem that has cost thousands of dollars over the past few years. Many employees hired in the past two or three years left the company after a very brief employment period. Management is even more concerned since the company is trying to place individuals in jobs which resulted in literally thousands of applicants for relatively few positions. A technique that will determine what individuals appear to be potentially desirable and, at the same time, help screen out applicants that appear to be undesirable is much needed. CHOOSING A CRITERION The whole WAB development process determines what personal factors predict a particular criterion. The systematic method for developing WABS begins with determination of a particular criterion that differentiates between potentially desirable and undesirable employees. It is critical to identify the most appropriate criterion that makes this differentiation. However, before a criterion can be used, it is necessary to evaluate its characteristics and to ascertain the criterion’s appropriateness. With the problem presented, management has determined that hiring individuals who will remain with the company for a number of years is crucial. Therefore, management believes that desirable employees will be those having long “job tenure” with the firm. Thus, the criterion chosen is job tenure. Instructions: With the following partial list of characteristics [2], circle the appropriate choice based on whether that characteristic should be necessary--(N), desirable--(D), or undesirable--(U) for the criterion of job tenure. Then, compare your answers with the discussion that follows. CRITERION Characteristics Job Tenure Reliable N D U Representative N D U Related to other criteria N D U Predictable N D U Measurable N D U Relevant N D U Uncontaminated and bias-free N D U Characteristic Discussion Reliable No matter what criterion is ultimately chosen, it has to be reliable. The reliability of a criterion is necessary for pre diction purposes. In this particular instance, job tenure appears, at least for the present time, to be a reliable criterion. Necessary Representative Any criterion chosen should be a representative measure of the problem. In this instance, management has defined the problem as being one of tenure. Therefore, the criterion of job tenure appears to represent the problem. Necessary Related to other Criteria In this instance, it is undesirable for job tenure to be related to other criteria. If it was related, it would be difficult to determine if it was actually being predicted. Undesirable Predictable If a criterion appears unpredictable, it may be fruitless to attempt to predict it. In this instance, job tenure appears to be predictable. Necessary Measurable If a criterion cannot be measured, it is difficult to justify it as being appropriate. Job tenure can be easily measured. Necessary Relevant This is the most important characteristic to assess. If the criterion is not related to the problem, results will have no meaning. Job tenure appears quite relevant to the problem. Necessary Uncontaminated and bias-free If a criterion is biased, results obtained will be distorted, and have little meaning. While it is extremely difficult to obtain a completely unbiased criterion, it is something to strive for in any study. To be completely accurate, an unbiased criterion is Necessary. If, in analyzing a chosen criterion, it is found that it may not be as “good” as desired, it is wise to search for one that could be possibly better. IDENTIFYING CRITERION GROUPS Choosing an appropriate criterion and identifying criterion groups are the most important steps in developing a WAB. If an inappropriate criterion is chosen and criterion groups are not identified properly, the whole intent of the study is not being met. An appropriate criterion coupled with properly identified groups leads to meaningful results. Once an appropriate criterion is chosen, two groups of current employees are formed: a High-Criterion Group (Group I)-- representing desirable employees, and a Low-Criterion Group (Group II)--representing undesirable employees. In order to make best use of the data, both groups should be as large as possible, with a minimum of 75 employees per group. These groups are subdivided into a Weighting Group and a Holdout Group. Each Weighting Group should include 2/3 of the larger group, with the remainder going into Holdout Groups. When all groups are formed, major questions need to be answered: 1) What is the distinction between short and long tenure?, and 2) How far back, chronologically, should we go to get data? Developments in Business Simulation & Experiential Learning, Volume 11, 1984 121 With regard to the first question, management has determined from the records that employees who worked at least one year tended to stay on for a long time. However, many employees left the company within 6 months. Therefore, long tenure will include employees who have worked for at least one year, and short tenure will include those who left within 6 months. With regard to the second question, the problem encountered has only been a phenomenon of the past three years. Therefore, records of all employees hired in the last three years will be examined for the necessary data. Employees with long tenure are desirable, and those with short tenure are considered undesirable. Such a clear distinction between desirable and undesirable employees is not always available. In many instances, a job analysis/performance appraisal study is needed to determine work behaviors and attributes of current employees. Once work behaviors and attributes are determined, personal factors that relate to those behaviors and attributes can be examined and ultimately measured. SELECTING APPLICATION BLANK ITEMS TO BE ANALYZED The type and number of items included in a WAB depend upon two considerations: the application blank itself and the legal ramifications of various items included. A WAB looks no different than an ordinary application form. A new application form is not needed because applicants do not know the application blank is weighted. “Legally,” any item can be used, either from the application form or from other personnel records. However, if any items lead to adverse impact upon a protected class (race, religion, or national origin), a Bona Fide Occupational Qualification (BFOQ) may have to be demonstrated in order to use that particular item [7, Section 14, Paragraph a]. The number of items in initial stages of the weighting process should be as large as possible, because many items may not differentiate between desirable and undesirable employees. Items that do not differentiate are not included in the final WAB; they have no predictive value. Items that differentiate must be analyzed to determine whether they bear an actual relationship with the purpose of study. For example, could an individual’s marital status influence his/her tenure on a particular job? Plausibly, marital status does influence job tenure because it might restrict an individual from moving from place to place. Thus, marital status might differentiate between desirable and undesirable employees. Instructions: For purposes of this exercise, only 6 items are included (to keep the exercise from becoming too cumbersome). Look at the 6 items below and reason if they might differentiate between desirable and undesirable employees. (1) Age; (2) Number of years previously employed; (3) Average tenure of former jobs; (4) Distance of residence from work place; (5) Marital status; and (6) Number of years of formal education. SPECIFYING ITEM RESPONSE CATEGORIES For each item used in the initial stages of development, data are divided into response categories. There are basically three methods: 1) Equal Frequency Classes--divides responses of combined (total) weighting groups into four or five classes with approximately an equal number per class. 2) Equal Interval Classes--divides the responses into equal intervals within a range of responses (e.g., for Age: 21-26, 27-32, etc.). 3) Maximum Weight Classes--trial and error approach to maximize differences of a particular response item. This method is least desirable because it may capitalize on chance differences within a particular item. Some items can easily be divided into response categories by a natural division. For example, marital status can easily be divided into the response categories of single, married, divorced (including legally separated), and widowed. For purposes of the exercise, all items are divided into response categories using Equal Frequency classes, or by a natural division. Instructions: 1) From data in Table 14 total responses in each category of "Age" for both High and Low Weighting Groups (use the worksheets provided). 2) Total both Columns 2 and 3. 3) Convert numbers in Response Categories into percentages; i.e., divide each Response Category in Column 2 by the total of Column 2. Place this number in Column 4. Do the same for Columns 3 and 5. 4) Subtract Column 5 from Column 4 for each Response Category. 5) Do this for remaining application blank items. DETERMINING ITEM WEIGHTS To determine whether desirable workers have different response patterns from undesirable workers, it is necessary to determine Net Weights and Assigned Weights. This is a straightforward process-- using the charts in Tables 2 and 3. Instructions: To determine Net Weights, use Charts A & B in Table 2. Take the difference in percentages (Column 6), and locate that same percentage in Chart A or Chart B. Place the corresponding Net Weight in Column 7. If percentages are negative, use the same process, only treat the differences in percentages and Net Weights as negatives also. To determine Assigned Weights, use Table 3. Locate the appropriate Net Weight in the table, and place the corresponding Assigned Weight in Column 8. 4 Tables and worksheets will be provided by authors upon request. Developments in Business Simulation & Experiential Learning, Volume 11, 1984 122 ASSIGNED WEIGHT VALUES Compare your Assigned Weight values with those presented in Table 4. The values you computed should be the same. Net Weights are converted to Assigned Weights because many Net Weights turn out to be negative, and thus, there is difficulty in totaling each applicant’s final score. Any item having the same Assigned Weight for each Response Category does not differentiate between desirable and undesirable, and should be discarded. Of the six initial items used in the analysis, only four (Age, Average Tenure, Distance of Residence, and Number of Years Previously Employed) appear to differentiate desirable and undesirable employees. These are the items to use in the final WAB. ANALYZING INDEPENDENT SAMPLES FOR CROSS- VALIDATION With these items that differentiate the two Weighting Groups, it is necessary to “reaffirm” them by cross- validating the data in the Holdout Groups. Each Holdout Group consists of another sample of employees independent of the Weighting Groups. If cross- validation shows that a differentiation exists between desirable and undesirable employees, greater confidence can be placed in items selected for use in the final WAB. The statistical analysis used in cross-validation must consider the dichotomy between desirable and undesirable employees. Therefore, the Point-Biserial Coefficient of Correlation should be calculated.5 The formula is: where: Mp = mean of values in the favored (desirable) category. Mq = mean of values in the remaining (undesirable) category. ót = standard deviation of the total sample. p = proportion of cases in the favored (desirable) category, (i.e., number of cases in desirable category/total number of cases. q = proportion of cases in the remaining (undesirable) category. Instructions: Using data in Table 5, calculate rpbi for each item selected in the final WAB. 5 For a good discussion of the Point-Biserial Coefficient of Correlation, see J. P. Guilford, Fundamental Statistics in Psychology and Education, (New York: McGraw- Hill, 1978), 6th ed. The correlation coefficient r pbi depends directly upon the differences between means Mp and Mq. If r pbi is significant at a specified level of confidence, a difference between desirable and undesirable employees exists for that particular item. Therefore, cross-validation supports use of that item in the final WAB. Instructions: From Table 6, determine whether r is significant at p .05 level of confidence. Then compare your results with those that follow. Note: If r pbi is calculated as a negative, treat it as a positive number. Item r pbi Significant at p .05 Age .6521 yes Average Tenure .5439 yes Distance of Residence .4234 yes Number of Years Previously Employed .5000 yes Do the items clearly differentiate between desirable and undesirable employees? What confidence can you hold in the results? SETTING WAB SCORES FOR SELECTION The practicality of a WAB is that selection decisions can be enhanced by determining which applicants appear to be more desirable employees. It is difficult in practice, however, to set rigid cutting scores for selection. Rigid cutting scores constantly need to be validated [7, Section 5, Paragraph B], and fluctuations in supply and demand of potential applicants might influence the number of potentially desirable employees. One alternative is to develop a range of scores in which more desirable applicants appear at the top of the range, and the more undesirable appear at the bottom. Those appearing more desirable can be evaluated further to assess other qualifications. EXTENSIONS AND APPLICATIONS A WAB can be used as a screening device to rapidly identify potentially desirable applicants. It may be used with other tests and measures as a predictor of possible “success” on the job. The WAB provides a quick and inexpensive technique to process many applicants and determine which appear to be more desirable. Those applicants scoring high can be evaluated on other qualifications to assess their potential to the company. M p - M q δ t x √pqrpbi = Mp - Mq δt x √pq = x √ = = rpbiAge: Mp - Mq δt x √pq = x √ = = rpbiAge: Mp - Mq δt x √pq = x √ = = rpbiAge: Mp - Mq δt x √pq = x √ = = rpbiAge: Average Tenure: Distance of Residence: Number of Years Previously Employed: Developments in Business Simulation & Experiential Learning, Volume 11, 1984 123 As a predictor, it is very important that the WAB be valid. With an unvalidated WAB, meaningful predictions cannot be made. Also, a WAB may improve predictions made by other selection techniques, and thereby further enhancing the personnel functions. Thus, increased power is obtained for differentiating between high and low human resource performance. CONCLUDING POINTS What is the basic procedure for developing a WAB? What are the most important steps in developing a WAB? What needs to be considered in selecting items to be used in a WAB? Can you think of any drawbacks that might limit the use of a WAB? In what ways can the WAB be used? REFERENCES [1] Blum, Milton L. and James C. Naylor, Industrial Psychology, Rev. Ed., (New York: Harper & Row, 1968), p. 182. [2] Dunnette, Marvin D. and James Maetzold, “Use of Weighted Application Blank in Hiring Seasonal Employees,” Journal of Applied Psychology, Vol. 39, No. 5, (May 1955), pp. 308- 310. [3] England, George W., Development and Use of Weighted Application Blanks, Rev. Ed., Industrial Relations Center, University of Minnesota, (1971). [4] Lee, Raymond and Jerome N. Booth, “A Utility Analysis of a Weighted Application Blank Designed to Predict Turnover for Clerical Employees,” Journal of Applied Psychology, Vol. 59, No. 4, (1974), pp. 516-518. [5] Roach, Darrell E., “Double Cross-Validation of a Weighted Application Blank Over Time," Journal of Applied Psychology, Vol. 55, No. 2, (1971), pp. 157-160. [6] Scott, Richard D. and Richard W. Johnson, “Use of Weighted Application Blank in Selecting Unskilled Employees,” Journal of Applied Psychology, Vol. 51, No. 5, (Oct. 1967), pp. 393-395. [7] Uniform Guidelines on Employee Selection Procedures, Equal Employment Opportunity Commission, 1978. Table of Contents Volume 11, 1984 Simulation Gaming as a Means of Researching Substantive Issues: Another Look A Further Test of the Group Formation and its Impacts in a Simulated Business Environment Impact of Economic Patterns on Student Performance in Computer Business Simulation Games Majority Fallacy Game with Independent Student Simulation and a Case Introducing the Marketing Channel Laboratory A Comparative Evaluation of a Marketing Game A Study of Comparative Effectiveness of Problem-Solving Technologies The Impact of Hierarchical and Egalitarian Organization Structure on Group Decision Making and Attitudes Risk-Free Decision Making The EX-STRA Export Strategy Game Computer Education for Management Students Developing a Computer Game/Job Simulation to Teach Functional Literacy Skills Experiencing Socialization First Hand: An Experiential Exercise in Organizational Socialization Networking Distributive Versus Integrative Approaches to Negotiation: Experiential learning Through a Negotiation Simulation Managerial Education and the Real World: Foudations for Designing Educational Tools Diagnosing Group Climate to Improve Supervisory Effectiveness Student background as a Factor in Simulation Outcomes: The Collective bargaining Example The Use of Pre-Plays in Management Education Experiencing the Process Debrief: A Workshop ABSEL Megatrend Roots MEGATRENDS for Business Simulation and Experiential Learning The Effects and Consequences of the Megatrends on Simulation Gaming: One View Opportunities for the Future: ABSEL's Role Experiential Learning-Based Discussion vs. Lecture Based Discussion: A Comparative Analysis in a Classroom Setting An Evaluation of the Minitab Package in Teaching Business Statistics Concepts A Path Analytic Study of the Effects of Alternative Pedagogies Developing and Using Weighted Application Blanks: An Experiential Exercise Building Airplanes Individual vs. Group Grade: An Exercise in Decision making A Marketing Plan Exercise: Development of Interteam Cooperation Using a Coordinated Experiential Approach Using Student Experience as the Basis for a Consumer Behavior Learning Exercise Student Evaluations of Instructors: What do Students Believe? A Description of the SOFTCAT Computer Assisted Teaching System Comparisons of Practitioners' and Professors' Perceptions of Business Policy Content and Learning Methods The Perceived Relationship Between Pedagogies and Attaining Course Objectives in the Business Policy Course The Use of Simulation in the Teaching of Business Policy A Research Study on Strategic Decisions in a Business Simulation Strategic Management Decision Making Researched Via Simulation Gaming Using Simulation to Investigate Factors in Competitive Bidding Combining Experiential Learning and management Assistance A Model for Teaching Management Skills Putting Experience Back into Experiential Learning: A Demonstration The Teaching and Behavioral Measurement of Managerial/Organizational Competencies: Developing Experiential Exercises and Simulations A Simulation Game Model for Conglomerates QCLAB - A Microcomputer Laboratory in Quality Control CTSS: A Commodity Trading Simulation System Problem Solving: An Exercise on Learning, Coaching, and Operant Conditioning A Demonstration of the Effects of Feedback as a Category of Reinforcement The Assessment of Feedback and Disclosure in Interpersonal Relations: An Experiential Exercise A Study to Determine Whether the Teaching of Basic Grammar Skills in Business Communication Classes Improves Students' Business Letter Writing Corporate Maladies Through the Eyes of the Memo Writer: A Seldom Used Experiential Tool Executive Bailout at Shake & Spear, Inc. The H.E./L&P Merger Intercultural Nonverbal Communications: An Experiential Exercise The Evolving Business Policies Course - Is Management Gaming the Logical Pedagogy? The Use of Decision Simulations in Management Training Programs: Current Perspectives Humanizing the Business of Medicine: The Use of Simulated Patients to Train medical Students Systematic Integration of Simulation Methods in a Graduate Management Curriculum Modeling Non-Price Factors in the Demand Functions of Computerized Business Using Spacial Relationships to Estimate Demand in Business Simulations Two Algorithms For Redistribution Of Stockouts In Computerized Business Simulations Leadership And Strategic Behavior A Comparison Of Two Business Strategy Simulations For Microcomputers Incorporating Decision Support Systems Into Management Simulation Games: A Model And Methodology Using Micro-Computers To Support The Analysis Of Complex Cases: It's As Easy As 1-2-3 Strategic Formulation Consistent With Pims: A Micro-Computer Application