THE USE OF SELF-ASSESSMENT WORKSHOPS IN A SCHOOL OF BUSINESS ADMINISTRATION New Horizons in Simulation Games and Experiential Learning, Volume 4, 1977 369 THE USE OF SELF-ASSESSMENT WORKSHOPS IN A SCHOOL OF BUSINESS ADMINISTRATION1 Steven D. Norton, University of Missouri-St. Louis During the past twenty years assessment centers have become increasingly popular as a method of predicting managerial effectiveness. MacKinnon [61 estimates that at least 300 organizations use assessment centers, including AT&T, Caterpillar Tractor, IBM, Sears, Standard Oil of Ohio, and many Federal and State Government agencies. There is a good deal of evidence supporting the validity of the assessment center as a predictor of managerial effectiveness [3, 8]. Most articles on Assessment centers suggest that they have developmental value for both the managers who have been trained as assessors and the candidates who have been evaluated, however, there is little evidence for this developmental value. This paper will discuss the use of an self-assessment workshops designed for development, rather than for prediction, in a School of Business Administration. Evidence on the developmental value of the assessment center, based on the students’ subjective reactions, will be reported. Data on the first version of the workshop used with two undergraduate classes will be discussed in this paper. Data on the second version of the workshop used with a graduate class will be presented at the meeting itself. THE DEFINITION OF AN ASSESSMENT CENTER The popularity of assessment centers has lead to the term being applied to a rather wide variety of assessment techniques. Recently, a Task Force on the Development of Assessment Center Standards arrived at the following definition: An assessment center consists of a standardized evaluation of behavior based on multiple input. Multiple trained observers, Assessors and techniques are used. Judgments about behavior are made, in part, from specially developed assessment simulations. These judgments are pooled by the assessors at an evaluation meeting during which all relevant 1 The author wishes to thank Nicholas Di Marco, Douglas Durand, and George Witteried for their comments on earlier drafts of this paper. New Horizons in Simulation Games and Experiential Learning, Volume 4, 1977 370 assessment data are reported and discussed, and the assessors agree on the evaluation that is made. [7, pp. 2-3] A BRIEF HISTORY OF ASSESSMENT CENTERS The first use of multiple techniques, including simulations, and multiple trained observers to predict performance was carried out by German military psychologists during the 1930’s. These techniques were further refined by the British in their War Office Selection Boards. Here in the United States, the Office of Strategic Services (now the CIA) established assessment centers to evaluate the potential of agents. After World War II one of the directors of the OSS Assessment Center, Donald W. MacKinnon, established the Institute for Personality Assessment and Research at Berkeley. It was not until 1955 that the first industrial assessment center was established at AT&T by Douglas Bray [2]. Assessment centers are now very popular in industry [311. They have been used to fill positions ranging from first level manager to high level executive [8]. The agencies which enforce Equal Employment Opportunity regulations have encouraged the use of assessment centers. DEVELOPMENTAL ASPECTS OF ASSESSMENT CENTERS While working with assessment centers at the Department of Housing and Urban Development (HUD), the author noted that most candidates believed that taking the assessment center simulations was a very developmental experience. The assessor training, and then evaluating the candidates during the actual assessment centers, were experiences that enhanced their own skills as managers. The author and two colleagues at HUD designed a self-assessment workshop which gave participants the opportunity to take part in simulations and observe each other. THE UMSL DEVELOPMENTAL ASSESSMENT CENTER Upon returning to the University of Missouri-St. Louis, the author adapted the HUD developmental assessment center for use in an undergraduate industrial psychology class. Most of the 49 students in the class were majoring in business rather than in psychology, but both groups found the assessment center experience to be relevant to their needs. New Horizons in Simulation Games and Experiential Learning, Volume 4, 1977 371 The Simulations Two simulations, made available by the Internal Revenue Service, were used in the workshop. The first simulation was a leaderless group meeting titled “Supervisor of the Year.” In this exercise, which is set in IRS, the assessment center candidate assumes the role of a manager who is on a panel to select a supervisor of the year from a group of nominees. The candidate is in a conflict between supporting his or her own candidate and cooperating with the group. According to the general instructions, he or she is “responsible for evaluating all of the nominees and determining the best possible final rating.” A letter from the candidate’s supervisor, on the other hand, urges him or her to vigorously pursue the selection of the candidate from their particular district. The second exercise is a managerial In-Basket which is set in a hypothetical Bureau of Health Standards. The workshop participant plays the role of a branch chief who was just placed in this new position and who must leave town in two hours. Before he leaves, he must deal with 34 items of varying importance, supported by a 25-page packet of background material. The items cover routine personnel matters, possible EEO grievances, violations of food standards, etc. The Sequence of Events in the Workshop The students were divided into two groups, candidates and observers, before class period one. As preparation for class period one, the observers studied a detailed Guide for Observing and Reporting “Supervisor of the Year.” This guide defined the skills observed in the simulation. The students who were to be participants in the simulation were instructed to spend twenty minutes reading a packet of material on their nominee for supervisor of the year and preparing a five minute talk. During class period one, the six participants gave their five minute talks and spent about 45 minutes deciding on the ranking of their six nominees. In preparation for class period two the observers reviewed their notes on the students who they had observed in Supervisor of the Year. The students who had been participants in the exercise looked over the guide to observing and reporting the exercise. All of the students took the In-Basket. Most of the second class period was spent in discussion of what had happened during the simulation. Examples of behavior showing high and low degrees of each observed skill were discussed. During class period two the students were also told how to conduct an In-Basket interview, an interview in which a candidate must defend his or her In-Basket decisions. At this point, each student exchanged In-Baskets with a partner. The homework assignment for the third period was to prepare to interview the partner by doing a meticulous review of the New Horizons in Simulation Games and Experiential Learning, Volume 4, 1977 372 partner’s In-Basket using a detailed In-Basket Guide. During the third class period itself, each participant interviewed his or her partner and was interviewed in turn. The fourth class period included a discussion of the decisions made in the In-Basket interview itself. The use of assessment centers as a selection tool was also discussed. The Second Workshop A second workshop was recently used in a graduate class on personnel appraisal. The In- Basket was taken from Jaffee (1968 pp. 5-65.) The leaderless group meeting was written by the author. In this workshop, groups of students actually met in evaluation meetings and pooled their judgments about the groups of students they had observed. A final report, similar to that resulting from an actual assessment center, was prepared for each participant. The second workshop meets the Task Force definition of an assessment center, except for the fact that the participants alternate in the role of assessor and candidate. Results of the Evaluation Questionnaire The students filled out an evaluation questionnaire at the end of the developmental assessment center (see Table 1). All questions were answered on a scale from one (low) to nine (high). The questionnaire was anonymous. Many students gave extensive answers to an open-ended question at the end of the questionnaire. The first question of the questionnaire dealt with the student’s interest in the different parts of the assessment center. They found the two exercises, Supervisor of the Year (mean = 5.6) and the In-Basket interview (mean = 5.4) slightly more interesting than the two class discussions (both means = 4.8). However, the responses to questions lA through lD had rather high inter- correlations (mean r .422, p < .01) indicating that students were either high or low on interest in the workshop in general. Question 2 asked about the student’s appraisal of their own performance on the parts of the workshop. They felt that they had performed best as candidates in Supervisor of the Year (mean = 6.1) and poorly as interviewers in the In-Basket interview (mean = 4.6). Performance in the three aspects of the In-Basket simulation: taking the In-Basket; being interviewed by your partner on your In-Basket, and interviewing your partner on his or her In-Basket, were highly associated (mean r = .78; p < .001). However, performance in Supervisor of the Year was associated only with taking the In-Basket (r .53; p < .001). Both Supervisor of the Year and being an interviewer or interviewee are interpersonal situations and taking the In-Basket is a paper-and pencil test. However, New Horizons in Simulation Games and Experiential Learning, Volume 4, 1977 373 only in taking the In-Basket and in Supervisor of the Year did the students actually have to assume the role of a manager. It appears from the correlation that students either assumed this role well in both simulations, or in neither. It is notable that there was no association between (self perception of) performing well and either finding the workshop interesting (mean r = .065; NS) or (self perception of) how much one has learned from it (mean r = - .04; NS). The students who felt that they had performed poorly in the workshop found it just as interesting, and felt they had learned just as much, as the students who felt they had performed well. Question 3 through 8 dealt with how much the students learned from their participation in the workshop. They believed they had learned the most about 1) how management potential is measured in assessment center (mean = 5.9) and 2) what it is like to be a manager (mean = 5.8) and about their potential strengths and weaknesses as a manager (mean = 5.2). They felt that they had developed some skills in observing and recording behavior (mean = 4.9) and in being interviewed (mean = 4.5) but that the workshop had less impact on their skill as an interviewer (mean = 3.5). The responses to questions 3 through 8 had rather high inter-correlations (mean r = .411; p < .01). Question 9 asks whether or not the workshop should be included in the course permanently (mean = 6.0). A positive response was associated with finding the workshop interesting (mean r = .516; p < .001) and learning from it (mean = .385; p < .05), but not with how well the students felt they had performed (mean r = .009; NS). Question 10, asking if class attendance should be required was included as another measure of the value of the workshop (mean = 7.0). Feeling that class attendance should be required was associated with finding the workshop interesting, (mean r = .377; p < .05) but not with how well the students felt they had performed (mean r = .046; NS). Question 11, “if you could start over again as a participant, would you spend more or less time preparing for the exercises?” was included because some students seemed unprepared for their role as an observer or a candidate. Since only 12% of the students said they would spend less time preparing for the exercises, the question became a measure of how much more time they felt they should spend. Spending more time preparing for the exercises is weakly associated with finding the workshop interesting (mean r = .205; NS) and with learning from it (mean r = .241; NS) but is not associated with doing well or poorly on the exercises (mean r = .018; NS). Question 12 notes whether the student was a candidate or an observer in the group exercise. As Table 1 shows, there were no differences between candidates or observers New Horizons in Simulation Games and Experiential Learning, Volume 4, 1977 374 except that the students who were observers performed better (t = 2.79; p < .01) as interviewees in the In-Basket interview than the students who had been candidates. (Means of 6.00 and 4.13 respectively.) A questionnaire was also administered after the second workshop, used with graduate students. The results of the questionnaire, and the correlations of the questionnaire items with ratings of the participants performance during the workshop, will be presented at the meeting itself. RECOMMENDATIONS The self-assessment workshop was generally well received by the students. Students’ attitudes toward the workshop were not related to their self-perceptions of their performance on the simulations. Based on class comments, it appears that time should have been devoted to actually demonstrating as well as describing the In-Basket interview. Unlike experienced managers, undergraduate students and even the students in a small graduate class not discussed here, wanted a role model in order to carry out the interview. After the workshop was over, most students said that if they could start over again, they would spend more time preparing for the exercises. The general consensus in class discussion was that there should have been either some reward for careful preparation and good participation or some sanction against poor preparation and non-participation. The well prepared students felt cheated by partners who were poorly prepared. Although the pilot workshop was reasonably well received by the students, it might be improved if the students handed in their In-Baskets, In-Basket interview notes, and notes for Supervisor of the Year. This material could be graded on either the degree of supervisory ability shown or on whether or not the student had made an honest effort in the simulation. One of the major differences between the workshop described in this paper and the earlier ones administered by the author was the fact that the former took place in four class periods over two weeks. The emotional intensity of an assessment center was lacking. It would be highly desirable to administer a developmental assessment center in one day, perhaps on a Saturday. THE ROLE OF THE SELF-ASSESSMENT WORKSHOP IN EDUCATING BUSINESS STUDENTS This paper has described a self-assessment workshop and the students’ reaction to it. The final issue to be discussed New Horizons in Simulation Games and Experiential Learning, Volume 4, 1977 375 is the relationship of the workshop to other experiential learning techniques and its future growth. The case study was the first experiential learning technique used to teach business topics. Management games, often computerized, have recently become popular. In a case study, the student is presented with a relatively broad and general description of an organization and asked for a relatively subjective analysis and recommendation. In a business game, the student is presented with quantitative information about an organization and asked to make quantitative decisions. In a self-assessment workshop, on the other hand, the student is presented with a specific and concrete management task and must actually behave as a manager. The use of the assessment center method as an experiential learning technique is in its infancy. At least three educational institutions have well established assessment center programs: The Graduate School of Business at Stanford University [4]; The Graduate School of Business at Baylor University [9]; and Alverno College in Milwaukee [11. As more universities and colleges use the assessment center method, simulations will be refined, scheduling alternatives will be explored, and instructors will develop their skills as assessment center trainers and administrators. The purpose of both undergraduate and graduate education in business is to prepare students to be effective managers or professional employees. The assessment center method can make a unique contribution to this preparation by allowing students to experience aspects of the manager’s job which would otherwise be unavailable in an educational setting. REFERENCES 1. Alverno College. Competence Based Learning at Alverno College, (Milwaukee, Wisconsin: Alverno College, 1974). 2. Bray, D. W., R. J. Campbell and D. L. Grant, Formative Years in Business: A Long- Term AT&T Study of Managerial Lives, (New York, Wiley & Sons, 1974). 3. Cohen, B., J. L. Moses and W. C. Byham, The Validity of Assessment Centers: A Literature Review, Monography II. (Pittsburgh, PA. Developmental Dimensions Press, 1974). 4. Harrell, T. B. 371--Career Planning, Spring, 1974. (Stanford, California: Graduate School of Business, Stanford University, 1974). 5. Jaffee, C. F. Problems in Supervision: An In-Basket Training Exercise, (Reading, Mass.: Addison Wesley, 1968). New Horizons in Simulation Games and Experiential Learning, Volume 4, 1977 376 6. MacKinnon, D. W. An Overview of Assessment Centers, Technical Report No. 1, (Greensboro, North Carolina: Center for Creative Leadership, 1975. 7. Moses, J. L., et. al. “Standards and Ethical Considerations for Assessment Center Operations,” Task Force on Development of Assessment Center Standards, May, 1975. 8. Norton, S. D., “The Emperical and Content Validity of Assessment Centers vs. Traditional Methods for Predicting Managerial Success.” in press, Academy of Management Review. 9. Williams, J. C. Manager Education--A Personal Approach. (Waco, Texas: Baylor University, 1974, Abstract). Table of Contents Volume 4, 1977 Double Play for Gaming Effectiveness Adaptive Rule Changes in Computer Simulation Gaming Œ A Means of Pedagogical Reactive Interchange Monte Carlo Simulation in Personnel Management Training Teaching About the Implementation of Job Redesign Using Simulation and Group Discussions An Interactive Simulation of Private Sector Collective Bargaining Leadership Evaluation and training through Behavioral Simulations: Method, Results and Future An assessment of the Effect of Experiential, Simulation and Discussion Pedagologies Used in Laboratory Sections of an Introductory Management Course An Experiential Understanding of the Trust Dimension Using Consulting Cases to teach Business Policy An Experimental Testing of Teaching Methodologies in Marketing Interpersonal Skill Development: The Experiential Training Unit (ETU) and Transfer of Training An Analysis of the Relationship between Personality characteristics and Preferred Styles of Learning Analysis of Effective Communication Skill Development in Graduate Business and Engineering Experiential Education Changing Perceptions of Learning in a Simulated Environment Student Perceptions: Simulation and the Corporate Policy Course Degree of Uniformity in Achievement Motivation Levels of Team Member: Its Effect on Team Performance in a Simulation Game Channel Conflict, Cooperation and Control: an Experiential Learning Exercise Differences in Experiential and Non-Experiential Learners' Reactions to Conflict between Individual and Organizations Behavior The Evolution and Evaluation of a Required, Senior-Level Course in Experiential Business Applications Building Management Skills through Problem Solving A Live-Case Approach to the Business and Society Course Experiential Learning: Toward the Development of a Theoretical Base and the Identification of Variables and the Hypotheses to Guide Research The Role of the Administrator in Experiential Learning and Simulations Some Thoughts on a Theory of the Use of Games and Experiential Exercises Three Applications of the Management of Learning Grid An Analysis of ABSEL: Its Past Achievements and Future Prospects New Horizons in Simulation Research Prediction of Academic Achievement in a Simulation Mode via Personality Constructs Sex Differences in a Bargaining Simulation COM-GAME: A Commodity Trading Game for Use in an Introductory Business Statistics Course A Financial Institution Management Game with Direct Participant Interactions A Non-Computerized Marketing Planning and Strategy Game Delphi in the Classroom: A Demonstration of Forecasting Economic Activity The Potential of Programmable Calculators for Processing Small Business Simulations Can a Small Predominantly Clack University Incorporate the Computer Simulation Gaming Teaching Methodology into it Curriculum Measuring the Effect of an Experiential Exercise Experiential Learning - Analysis of a Partial Success Predicting Participants' Performance and Reactions in an Experiential Learning Setting: An Empirical Investigation A Simplified, Non-Computerized Marketing Channels Game Manufacturers and Retailers: A Negotiation Game for Beginning Management Students Petroleum Management Game A Securities Dealer Simulator SIM ECO SOC with Business Curriculum Modules: A Simulation for Business Ethics and Morals The Picnic: A Perceptual Errors Exercise Salt III; an Experiential Exercise to Highlight the Interpersonal Dynamics Involved in the Negotiation Process Experiential Exercise on Values, Attitudes and Conflict Resolution in Organizational Behavior Kick'N Go: A Product Management and Social Responsibility Dilemma The Use of Self-Assessment Work-shops in a school of Business Administration The Dilemma of Self-Perception