THE ASSESSMENT CENTER AS A TEACHING/LEARNING DEVICE Developments in Business Simulation & Experiential Exercises, Volume 13, 1986 260 THE ASSESSMENT CENTER AS A TEACHING/LEARNING DEVICE R. Bruce McAfee, Old Dominion University Alex Havryluk, Old Dominion University ABSTRACT An Assessment Center is a process in which one group of’ participants evaluates another group on a selected set of’ dimensions. This paper describes the nature of these Assessment Centers and how one was implemented and conducted in a college classroom. Furthermore, it analyses obstacles in evaluating and conducting one. Finally, it compares the Assessment Center use of’ experiential learning exercises with the more traditional ways they are used. We conclude that Assessment Centers are potentially a rich source of new ideas for ABSEL members. INTRODUCTION Most university classes in Business Administration do not integrate or pull together a student’s skills, knowledge, and behavior. With the exception of’ the capstone course “Business Policy,” they focus on specialized cognitive areas. They do an excellent job of’ preparing students to be students - or possibly teachers. Students become authorities on communication, financial analysis, and/or control systems along the lines of a Monday morning quarterback. They get very little of the “hands-on” experience that challenges the knowledge, skill and emotional make-up of’ a manager. This is unfortunate because the university is better equipped than the workplace to provide dispassionate observation and supportive feedback necessary for a manager-in-training. It is the authors’ opinion that the Assessment Center, used by industry largely as a screening device, could be a very powerful classroom device to heighten “hands-on” learning and skills development. In this paper we will briefly describe the nature of an Assessment Center and explain how this method was implemented at Old Dominion University. Finally, we will examine the use of this technique as an experiential learning device. WHAT IS AN ASSESSMENT CENTER? The Assessment Center is a process in which one group of participants (assessors) systematically evaluates another group of participants (assessees) on a selected group of dimensions. These evaluations are ultimately fed back to the assessees. Normally, participants are evaluated on eight to twelve dimensions, the most frequent of which are: leadership, persuasiveness, perception, flexibility, decisiveness, organization and planning skills, problem solving skills, and oral and written communication skills. While many different dimensions can be assessed, organizations typically chose those which best fit their particular needs and requirements. Another defining characteristic of an Assessment Center is the complex of techniques used to evaluate each assessee. Most frequently encountered are: the in-basket, a leaderless group discussion, business games, a background interview, a competitive group exercise, a ease or problem analysis, a role play interview, and paper and pencil tests measuring aptitude or personality. Here again, the particular exercises/instruments selected by an organization are those which most appropriately measure the dimensions being evaluated. At least one exercise/instrument is used to assess each dimension. This enhances the possibility of cross- validity observations and provides additional information to feed back to assesses. Yet another distinguishing feature of an Assessment Center is that by design each assessee is observed by more than one observer. Each observer has no other role but to evaluate a participant and take extensive notes on his/her behavior. In order to perform their functions effectively, assessors are trained in observation skills and how to record and assemble what they see. The assessors are familiar with the dimensions being assessed because of their professional experience, training, or the fact they have themselves gone through the center. Frequently it is a combination of two or more of these. Before any feedback is given an assessee, the observers spend one or more hours jointly pooling their observations and identifying the specific behavior data to be discussed with the assessee. ODU’ S ASSESSMENT CENTER During the summer of 1983, the authors established an Assessment Center at Old Dominion University and built an experimental course around it. The course was made available to juniors and seniors and met two nights a week (7:00-10:00) for eight weeks. The course was taught by the authors of this paper and enrollment was limited to 20 students. The specific goals of the course were: 1. To provide information to students regarding their strength and weaknesses on 9-11 critical leadership/management skills. 2. To improve students’ observation and evaluation skills and their ability to feed information and evaluations back to others. 3. To improve students’ ability to evaluate themselves and accept constructive criticism and praise from others. 4. To familiarize students with the Assessment Center concept and how it is used. Assessment Center Format Our Assessment Center was structured as follows. On the first night of class, students were given a one hour lecture on the history and purpose of Assessment Centers. This presentation was followed by a discussion of how our own Assessment Center would be structured, what was expected of each student, and how students would be graded. During the last hour of class, an experiential exercise was used to familiarize students with each other. This “Ice Breaking” exercise required that each student remove three items contained in his or her wallet or purse such as a credit card, library card, or insurance certificate. After picking any three items he or she wanted, each student showed the class each item, stated what the item was, and mentioned its Developments in Business Simulation & Experiential Exercises, Volume 13, 1986 261 significance. For example, one student presented a sailboat club membership card and stated that he belonged to the club and sailed a 23 foot sailboat on the weekends. At the conclusion of class, students were divided into two groups of 10 each, one group of assessors and one of assessees, and were told to report to different rooms for the next class. From this point on, the students never met again as a whole group until the last day of class. The format for the next seven classes is outlined below: At this point in the class a role reversal occurred: the 10 Assessors became Assessees and vice versa. The second half of the course then paralleled the first, only this time different exercises were used: On the final day of class, all students met to debrief the Assessment Center. They also completed 2 separate questionnaires and a course evaluation form. Dimensions Evaluated During the Assessment Center, students were evaluated on nine different dimensions: 1. Oral Presentation 2. Written Communication 3. Listening Skills 4. Sensitivity to Others 5. Assertiveness 6. Decisiveness 7. Leadership 8. Motivation/Energy 9. Planning and Organizing No one exercise was designed to measure all of these dimensions. Rather, each exercise was used to evaluate assessees on from three to six of these attributes. Grading From the student’s perspective, one of the most novel aspects of the course was the absence of tests. Rather, each student’s grade was based on (1) a written self-evaluation, (2) an oral assessment of one assessee, and (3) a written assessment of’ the same assessee. The self-evaluation was submitted at the end of the course, after each student had been assessed. The oral assessment of another was conducted in private with one instructor present and was recorded on tape. The written evaluation was given to the assessee at the conclusion of the oral one, and a copy was given to the instructor. Each of the three course requirements counted for one third of the student’s grade. Evaluating The Assessment Center In industry, Assessment Centers are frequently evaluated in terms of their predictive validity, i.e. their ability to accurately predict an employee’s career success with an organization. Since our students worked for different organizations in different capacities, this approach was not possible. Rather, four different methods were used, most of which followed from the objectives of the Assessment Center described earlier. They were: 1. Each student was asked to complete a self- assessment questionnaire, covering the nine dimensions evaluated, on both the first and final day of class. They were also asked to write a self-evaluation report at the end of the course. Both of these were used to determine the effect of participating in an Assessment Center on each student’s self-evaluation. Developments in Business Simulation & Experiential Exercises, Volume 13, 1986 262 2. Students were asked to critique the evaluation they received from their Assessor. This was done to evaluate each student’s ability to accept constructive criticism and praise from others. 3. The quality of each student’s oral and written assessment of another student was used to judge each student’s ability to evaluate others and give feedback. 4. Students completed the University’s course evaluation form which rates 15 different aspects of the course. We found that student participation in the Assessment Center did change their self-evaluation. The changes, however, were not uniform across students, i.e. no particular patterns emerged. In addition, while changes did occur, we are not sure whether the post evaluations were any more accurate than the initial ones. While we surmise that they are, we had no method of judging each student’s true score on each of the nine dimensions. We also found that students effectively accepted constructive criticism and praise. These topics were discussed during the course, and the students followed the guidelines provided by the instructors. It should be mentioned, however, that we had no data on how students reacted to feedback prior to taking the course nor whether the skills they demonstrated at the end of class would transfer to actual job situations. We judged the students’ oral and written assessments of each other to be uniformly well done. Both were comprehensive in scope and followed the guidelines provided by the instructor. Finally, students as a group evaluated the course very favorably. They stated on the questionnaire that the course constituted a good learning experience and that they would highly recommend it to others. Obstacles In Conducting Assessment Centers Conducting an Assessment Center on a college campus for college students poses a number of obstacles. It requires two individuals to conduct each class, one to instruct and assist the assessors and another to instruct and assist the assessees. In addition, a 2--3 hour block of time seems essential. In fact, we found that on many nights our class extended far more than the allotted three hours, at least for some of the students. Furthermore, a minimum of two classrooms is desirable. Indeed, on some nights we used four classrooms simultaneously. Yet a further administrative obstacle relates to student absenteeism. Unlike most other classes, each day a student comes to class, he or she is either assessing another or being assessed. In any case, attendance is critical. We announced the first day of’ class that attendance was mandatory and a prerequisite for remaining in the class. Fortunately, only one student missed one class all semester. Nonetheless, where absenteeism is high, an Assessment Center would not likely be a success. In addition to these potential administrative roadblocks, there are two conceptual ones which faculty members who are thinking of conducting an Assessment Center need to address: (1) What dimensions will be assessed during the class and how will they be measured? (2) How will the effectiveness of the Assessment Center be judged? The first question involves determining which skills and abilities are to be assessed and finding exercises which can be used to measure them. We found the most difficult problem here was defining the dimensions we had picked in terms of specific observable behavior. For example, what behaviors constitute “sensitivity to others” or “decisiveness”? The second question seems to be even more difficult to solve. Indeed, we are not aware of any good solution to it. While we judged the effectiveness of our Assessment Center in the four different ways described earlier, we are aware that each has serious drawbacks to it. ASSESSMENT CENTER vs TRADITIONAL USES OF EXPERIENTIAL LEARNING METHODS We would now like to address a question which is of particular interest to ABSEL members: how does the use of experiential learning exercises/simulations in an Assessment Center compare to the traditional way in which they are used? What are the differences between the two? What are the similarities? Exhibit I shows one possible approach for comparing the two, namely in terms of (1) purpose, (2) basis for learning, (3) roles of observers, and (Ii) roles of participants (non- observers). Purpose Perhaps the most striking difference between the Assessment Center and traditional use of experiential learning exercises/simulations is in terms of the purpose of the exercises themselves. A review of prior ABSEL proceedings suggests that most faculty who use exercises/simulations do so to teach specific cognitive materials such as how to develop a weighted application blank [1] or to explain certain concepts such as operant conditioning [ 21 or reinforcement and feedback [4]. To a lesser extent faculty appear to use exercises as a way of achieving affective (process) learning such as examining people’s reactions to various communication cues [3]. By comparison, the major reason Assessment Centers use experiential learning exercises is to provide a basis for assessing participants’ skills and abilities. Exercises, then, are the vehicle used to evaluate assessees. Conveying cognitive information (of a generic nature) to participants is typically of only secondary importance or of none at all. Thus, it appears that while both Assessment Centers and traditional users use experiential learning exercises as a means to an end, the end itself differs significantly. This difference affects the basis for participant learning and the role of both the observers and other participants. Basis For Learning Perhaps the major similarity between traditional and Assessment Center users of exercises/simulations is that both view exercises as learning rather than teaching devices. They stress active versus passive learning. As such, the bulk of participant energy for learning comes from the student and not the teacher. This is unlike the lecture approach to education in which student energy is generated by the teacher, and in which high student participation occurs primarily at exam time. Since student involvement is so Developments in Business Simulation & Experiential Exercises, Volume 13, 1986 263 periodic very little gets integrated into the students’ prior knowledge base. What little does, tends to be highly cognitive and frequently so disassociated from the students’ own personal experience that it cannot be readily called upon for problem solving behavior in real life. While both Assessment Center and traditional users rely on active learning brought about by the participants’ own energy, there is one difference between the two which is important. It centers around the issue of the source and intensity of student involvement in the exercise/simulations. In an Assessment Center, student energy and involvement is typically very high because the subject matter content is the student him or herself. Keep in mind that assessees are told that they are being observed during the exercise, that their own skills and behavior are being evaluated, and that these assessments will be fed back to them individually. How can one not be personally involved under these conditions? Student involvement is high for yet another reason: the exercise/simulations used in the center simulate the business or management world toward which the student is oriented. Thus, the exercises and simulations have high face validity. Beyond that, the center provides participants with an opportunity for “testing out” their own skills and abilities. One can put into practice ones accumulated cognitive learning, ones affective characteristics, and ones skills. All of these further heighten student involvement and learning. In comparison, when exercises/simulations are traditionally used, participants are not typically being assessed on an individual basis. Instead, it is often the group which is being evaluated, or no explicit evaluations are made or fed back by anyone. Hence, the source of participant energy is not primarily to learn about oneself, although this frequently occurs. Rather, the source is typically a desire to participate and to learn cognitive and/or affective information. While one can only speculate as to the significance of this difference, a good case could be made that participating in an Assessment Center is a more intense experience than performing exercises elsewhere. Whether this ultimately results in greater learning is yet another debatable issue. Role of Observers Yet a third difference between the two is the role of the observers in each. In an Assessment Center, observers are typically instructed to evaluate only one assessee. By design, there are typically two or more observers watching each assessee. In addition, observers’ roles are exclusive ones; they do not at any time participate in the simulation itself. Furthermore, observers are told specifically what behaviors to focus on and assess in each exercise. Thus, they limit their observations to previously selected traits and abilities. Beyond that, observers in Assessment Centers typically receive considerable training as a group prior to observing participants. In fact, most companies require that observers go through an Assessment Center themselves prior to undertaking an observer’s role. Finally, observers have the task of providing feedback to participants on an individual basis, either orally, in writing, or both. By contrast, when exercises are used in traditional ways, observers typically receive little or no training. Most have not participated in the exercises previously. In addition, the observers frequently play dual rather than singular roles. That is, they simultaneously act both as participants and observers giving feedback to each other during the post debriefing period. Moreover, the assignment of observers to participants is often not as disciplined, nor are the requirements regarding which behaviors to observe. Instead, observers are frequently told to watch all of the participants and to look at all of the behaviors being exhibited. Last but not least, traditional observers do not typically give in-depth feedback to participants on an individual basis. Rather, when and if feedback is given, it is frequently given to the group as a whole or depersonalized in some way so as not to embarrass anyone. Role of Participants/Feedback Our final point of comparison relates to the role of the participants (assessees) in the exercises. In both Assessment Centers and traditional simulations the primary occupation of participants is that of engaging in a series of exercises/simulations. In Assessment Centers, an interrelated series of 4-8 exercises are typically used. The same situation can also occur in traditional settings. However, the similarities between the two often end here. The role of participants in Assessment Centers is typically quite distinct in three ways. First, Assessment Center participants often work alone on one or more exercises, not just in groups. Secondly, participants are typically not provided with feedback until all exercises are completed rather than at the end of each exercise. Third, feedback to participants in an Assessment Center is often used as a basis for discussing each participant’s career goals and how those goals can be reached. Thus, the feedback becomes a basis for action by the participant. This is not typically done when exercises are used traditionally. SUMMARY AND CONCLUSION In this paper we have described the characteristic of an Assessment Center, explained how one was implemented at Old Dominion University, and compared Assessment Centers usage of experiential learning exercises/simulations with their tradition usage. For years ABSEL members have prided themselves on being in the forefront of knowledge on the use of experiential learning methods. Many have experimented with new ways of conducting exercises/simulations in an attempt to maximize student learning. In keeping with this tradition, Assessment Centers appear to be a rich source of new ideas for ABSEL members. While some may actually want to convert their classrooms into an Assessment Center, others may prefer just to use some of its techniques. For example, one possibility is for ABSEL members to change the way they use observers, participants, or both to be more similar to those used in Assessment Centers. To illustrate, observers could be divided and told to focus on only one participant or only on certain specific behaviors. They could be trained prior to watching an exercise so as to improve their observation ability. They could also be instructed to provide participants with individual feedback. On the other hand, the participants’ role could also be modified so that they would not receive any feedback until the end of a series of exercises, or would receive both oral and written feedback. This feedback could also be used as the basis for discussing each participant’s career, or academic or personal goals. Developments in Business Simulation & Experiential Exercises, Volume 13, 1986 264 Changing the purpose of an exercise so as to incorporate an assessment objective is yet another area worth exploring. Exercises/simulations could be used not just to convey cognitive and process information but also to assess participants on selected skills and abilities. For example, ABSEL members who use an exercise designed to teach decision making skills could also use it as the basis for assessing a student on concepts taught earlier in the class such as listening or oral communication skills. REFERENCES [1] Dawson, Donald B.; Albert S. King and Ralph F. Catalanello, “Developing and using Weighted Application Blanks: An Experiential Exercise,” in ABSEL Proceedings, 1984, pp. 120-124. [2] Gomolka, Eugene G. and JoAnne Mackin, “Problem Solving: An Exercise on Learning, Coaching, and Operant Conditioning,” in ABSEL Proceedings, 1984, pp. 202-205. [3] Govahi, Gita and Sid Ward, “Intercultural Nonverbal Communication,” in ABSEL Proceedings, 1984, pp. 221-223. [4] Newmiller, Clyde E, “A Demonstration of the Effects of Feedback As A Category of Reinforcement,” in ABSEL Proceedings, 1984, pp. 206-208. Table of Contents Volume 13, 1986 How to Use Microcomputers in Human Resource Management Courses What's My Line? An Exercise in Job Analysis, Description and Classification Relationships Between Team Cohesion Dimensions and Business Game Performance Four Factors Affecting Group Performance in Business Policy Simulations Enhancing Strategic Management Goal-Setting Skills in the Business Policy Course Using a Business Simulation in the Principles of Management Course - Learning Outcomes and Perceptions Personal Computers: Drexel's Experience Using Psychological Type to Enhance Negotiation Skills The Effects of Conflict Handling Styles on Negotiation Behavior: An Analysis of an Experiential Exercise An Experiential Test of Bargaining Decisions and Representation Self-Grading of Examination by Students, A Viable Alternative to Traditional Procedures Perceived Instructor Enthusiasm and Student Achievement An Experiential Approach to Teaching Subordinate-Oriented Communication Assessing the Effects of a Computerized Study Guide in Macroeconomic Principles: A Statistical Analysis A Sales Management Simulation for the PC an Integrative Tool for the Sales Management Course Teaching Re-evaluation of Salespersons Through the Use of a Simulation Game A Test of Student Performance and Attitudes and Performance Under Varying Game Conditions Developing and Testing Airways: A Marketing Simulation Experiential Learning Revisited: Some Thoughts on Designing More Adaptive Management Education Programs Developing the Competencies of Creativity and Accurate Self-Assessment Self-View and Norms: Some Longitudinal Laboratory Results Strategic Planning with an Experiential Case The ABSEL Research Consortium: Preliminary Steps and Position Papers An Analysis of ABSEL Conference papers (1974-1985) The Creation and Operation of a Data-base System for ABSEL Research Funding for a Research Consortium on Simulation and Experiential Learning Simulation - Indoctrination or Learning The Dilemma In Evaluating Classroom Innovations Personality Variables on Group Cohesion, Team Participation and Total Learning Justifications for and Problems in Developing and Using Computerized Experiential Activities Management Competency Models and the Life-Long Learning Project: What Role for ABSEL External Validity of Business Games Restructuring the University - An Experiential Exercise The Surrogate Class Exercise Images of Effectiveness: A Classroom Exercise The Algebran Industry: A Tutorial Framework Another Chance: A Multi-Leveled, Macro Business Game for Micro-Computers A Water Quality Management Simulation Game Acquisitions? Divestitures? Progress Report on a Conglomerate Game Enterprise: A Multi-Purpose Management Development Simulation Marketer: A Microcomputer Simulation in a High Tech Industry A Pedagogical Analysis of Alternative Tax Elections for Exploration Costs in the Mining Industry Using of Multiple Microcomputers Application Programs to Teach Fundamental Business Concepts and Practices Demonstrating the Role of Simulation in Strategic Operations Planning: A Case Study in Bank Check Processing Location Analysis The Subjective Side of the Decision Support System: A Pitfall for the Panacea A Group-Based Procedure for Revealing Judgmental Heuristics and Biases An Experiential Exercise in Bayesian Decision-Making Values for Selected Parameters in Physical Distribution Simulation and Games Computerized Business Management Simulations for Tyros The Management Decision Laboratory at New York University An Example: The Use of Management Games on Microcomputers by Computer Novices Enhancing Mainframe Simulation Via Microcomputers: Designing Decision Support Systems Integrating Personal Computers Into a Course as a Decision Support Tool A Decision Support System for Capital Funds Forecasting Performance and Attitudinal Affects of a Decision Support Package in a Business Game Using the Real World: Interviews with Woman Mangers The Cooperatively-Authored Student Handbook International Buying: An Experiential Exercise The Research/Teaching Interface: Turning a Pretest into an Experiential Exercise Building Microcomputer Business Simulations Simulation with Discrete and Continuous Mathematical Modeling An Interpolation Approach to Developing mathematical Functions for Business Simulations The Design of Simulation Models Using Thought Organizers The Assessment Center as a Teaching/Learning Device An Approach to Meeting the AACSB Guidelines for Introducing Skills and Personal Characteristics (SACs) Using an Interclass Simulating and Role-Playing Pedagogy Experiential Learning about the World of Work: A Program for Primary and Secondary School Educators Assessment of Relevant Personal Characteristics of Potential Entrepreneurs