HIVING MODEL: ASSESSING MANAGEMENT SKILL AWARENESS Developments in Business Simulation & Experiential Exercises, Volume 10, 1983 112 HIVING MODEL: ASSESSING MANAGEMENT SKILL AWARENESS Dee W. Henderson, Brigham Young University ABSTRACT Although assessment efforts for identifying and predicting management skill levels have been in vogue for over a decade, and the use of the results has been readily adapted to skill development use, there exists an unrecognized methodological schism between identification- prediction and development. The author has not found the generally assumed transition from skill identification methodology and the skill development methodology to be congruent. The author’s research has found the development of management skills over time (over one year), with use of an assessment program, requires some significant adaptations. It requires a different model which provides cognizance and rigor to the improvement effort. The assessment model described in their paper, the Hive, provides situational adaptation, a skill improvement tracking system and more effective results. INTRODUCTION Research establishing the practical value of assessing management skills for student and manager selection and development has made considerable progress in recent years. Unusual support has been afforded assessment centers by the federal courts as well as the uncommon solidarity by practitioners, consultants, and academicians for the theoretical base and workable results of the process This paper focuses on the latter point-- the validity of using assessment centers for individual development--and presented a new model that appears to enhance this process. In contrast to most studies, specific focus will be on the criteria for development. Much attention in earlier studies has been given to predicting performance and identifying skills. Less emphasis has been placed on using the system for personal ski]l awareness and skill development. This way be the heart of the issue--how to utilize the assessment approach as a viable tool for first, management skill awareness and second, subsequent skill development. The author has designed, organized, conducted and evaluated assessment efforts for a number of large private and public organizations. He has also conducted within the Graduate School of Management at Brigham Young University a traditional and experimental development assessment center for graduate students each of the past six years. At the University, assessments are held two or three tines a year to assess full-time and middle-career graduate students in a two-year masters degree program in administration. The assessments are conducted early in the first year of a two-year masters program. During the first two days of assessment center exercises, participants are evaluated on nine skill dimensions. Each dimension includes factors that have been analyzed and found to be general prerequisites for successful performance in general administrative or mid-management positions. On the third day, the assessors converge and write final evaluations while the participants attend a career development and debriefing session. Later that day, participants receive a report on their performance at the assessment center and counseling on future development suggestions. The author has found that there are serious problems in trying to develop skills on s long ten basis through the use of a traditional assessment workshop. The source of these problems and the point of correction appears to be in the modular display of appropriate skills. Development Problems Although the graduate school assessment program has been successful in directing participants’ attention to management skills, identifying skill strengths and weaknesses, creating community support, and providing a foundation for participant development, there are critical problems encountered that are singularly unique to an assessment committed to skill development. The author has found the problems identified are interrelated and, although not systematically considered, are common to most assessment centers. They can be categorized as follows: 1. Feedback is highly dependent on the skills of the assessors. Feedback is encouraged to be, and usually is, given both orally and in writing in a constructive manner, concentrating on positive ways that the individual can use the information to upgrade his or her managerial skills. This illative feedback provides a positive result for the assessees, both in letting them know what their strengths and weaknesses are and in letting them identify ways in which they can be involved in a constructive change. However, the impact of the feedback tends to be diluted by both the varying communicative skills of the assessors and the discordant conciseness of the oral and written information transmitted to the assessees. 2. The cognizant impact on the student- participants varies considerably. flue to the nature of the feedback, some participants’ opportunities to discount and blend the discrepancies between their self- appraisals and the assessors’ feedback increased with time. During development plan counseling sessions, primarily those held toward the latter part of the two- year program, two startling comments were frequently heard; first, “I really didn't learn anything about my management skills that I didn’t already know;” and secondly, “The feedback wasn’t specific or encompassing enough for me to know exactly what skills I might work on to develop.” 3. The dimensions were limited to nine constellations of related behaviors to which the assessors and assessees had to relate. The dimensions and the rating scales had to be common and applicable to Developments in Business Simulation & Experiential Exercises, Volume 10, 1983 113 different management situations because the participants were, or would be in the future, working in different multifunctional organizations. Maintaining the scales to a minimum constellated number has research validity. However, this experience of longitudinal skill development experimentation has demonstrated that the ambiguous nature of limited constellation ratings has serious limitations over time. 4. The system lacked a lucid and relatively detailed picture of feedback from which comparisons of self-appraisal and collaborative developmental tracking could be realized. The Model, Scale Design, and Development The model. A configuration or model was needed that would be as pictorial as possible. One that would allow the scaling breakdown of behavior constellations in a plottable, graphic manner. A model which would vividly contrast the explicit discrepancies between the assessee’s self-appraisal and the assessor’s converged feedback. The format would also have to be flexible enough to adjust to rating scale changes, behavior category expansion, or reduction. The Hive mode], a construct created by World-Wide Achievements, Inc. for other learning purposes, fulfilled all these requirements plug being relatively easy to apply and to understand. The method used was adapted to scales designed by the author to measure observable behavior rather than personal traits. The method was also more precise than typical assessments, yet its specificity did not appear to make it inapplicable to the development of organization-wide scales in a multifunctional management environment. Scales design. The scales in this study were developed after a series of meetings with groups of managers who were asked to list the characteristics that were considered essential for advancement in most managerial jobs. The researchers refined the managers’ lists by eliminating obvious overlaps and ambiguities and eliminating personality traits or other characteristics that did not represent observable behavior. The refined list was pilot tested by having managers use the scales to rate themselves and a few of their subordinates. The scales were to be used for developmental purposes and designed to describe behavior in highly structured simulations where assessors could be trained to relate many scales to behavior commonly elicited by these simulations. In a sense, the new design and model more fully complied with good psychometric practice in keeping the scales unidimensional. For example, scales designed for assessment centers usually describe only one or two constructs. However, the number of scales, although more precise, increased by six times. The scales. The nine dimensions and their scaled behavior clusters are as follows: 1. Work Characteristics Involved in task immediately; operates at a fast pace; does not disturb any others with work; checks progress. 2. Organization and Planning Considers the total situation; identifies the elements of the situation; arrays alternatives; analyzes consequence of alternatives; prioritizes alternatives; considers precedent for future problems; strategizes before taking action; sets goals and objectives; prioritizes objectives; breaks items into double parts; distributes resources; uses organizational aids if possible; considers contingency actions; plans for short-range objectives; plans for long-range objectives. 3. Leadership Others react positively toward the person as a leader; others give the person attention; others listen to person’s inputs; gives information to others; seeks information from others; seeks to influence others; can be very directive if needed; can be very nondirective; builds supportive relationships; pursues own point of view with others; pursues other point of view with others. 4. Oral Communication Attempts to speak; speaks loud enough; pronounces words distinctly; pronouncements noticed by others; effective eye contact; effective voice inflections; energy effectively put into speaking; listens to others; responds effectively to what others are saying; impact on others appears to be what has been intended; concise. 5. Decision Making Searches for alternatives; decisive; displays good judgment. 6. Perception Sees critical elements of problem; sees relationships among elements of a problem; evaluates people accurately; sees the obvious; sees underlying causes; identifies practical/political limitations on various solutions; identifies creative possible approaches to problems. 7. Delegation Delegates when appropriate; clarifies given expectations; holds others accountable with follow- through; extends degrees of freedom. 8. Written Communications Content clear and understandable; concise; organized; appropriate word choice; neat. 9. Response to Change Adaptable; maintains calm leadership under unusual or adverse conditions; maintains own decision! when appropriate. The hive. Figure 1 shows the format and content of the nine dimensions and their behavior clusters within a Hive matrix. The skills are plotted within specific cells in the Hive. There are blank cells in the Five which allows for easily adapting the skills dimensions to the needs of different organizations or unique organizational unit situations. If needed, each column or individual cell of the Hive Matrix can he expanded into its own Hive matrix to include specific skill behaviors or conceptual skill breakdowns. Thus, the Hive becomes a dynamic model simply adjusted to the needs of the user. The Study and Results After the scales were developed and assessors trained, two groups were assessed. They were: Developments in Business Simulation & Experiential Exercises, Volume 10, 1983 114 Group I -- Thirty-eight mid-career managers seeking graduate degrees in administration. Group II -- Forty-seven full-time graduate students in administration. Both groups were asked to self-appraise their performance on the assessment workshop prior to receiving the converged results of assessors. The skill behaviors were scaled on a five-code system: (1) weak, (2) questionable, (3) satisfactory, (4) more than satisfactory, and (5) outstanding. Figure 2 shows the contrasting mean results between the assessors’ “converged" assessment of the two groups collective skills and the groups’ own “self”- appraisals of their performance. Over seventy-five percent of the management skills for both groups combined were assessed by the assessors as significantly (p < .01) lower than the self-appraisals of the assessees. Collectively, none of the skills were assessed significantly (p <.05). Figure 3 shows that when comparing separately the graduate students and mid-career managers' self appraisal and assessors’ converged rating, mid- career managers saw themselves significantly (p <.01) more effective than the converged in 80 percent of the matrix cells or the separate skill behaviors. The full-time graduate students collec- Developments in Business Simulation & Experiential Exercises, Volume 10, 1983 115 tively saw themselves significantly (p< .01) more effective in only 15 percent of the skill behaviors. The comparison between the managers’ and the full- time graduate students’ self-appraisals revealed 17 percent of the skills being viewed as significantly different. The managers saw ten of their skills as being significantly more effective than the full- time students, only three skills were seen by the full-time graduate students as being significantly more effective and they were in the leadership dimension of “building supportive relationships” and the communications dimension of “being concise” and “pronouncing words distinctly.” Table 1, using just the mid-career managers, illustrates the wide discrepancy between the assessees’ collective view of their performance on specific skills and those converged appraisals by the assessment teams. With the pictoral Hive this form of specific contrast and congruence in a modular and specific manner is available for the individual assessee, treating a rigor of awareness unobtainable with former methods. Table I Skills collectively assessed as being significantly less effective by assessor teams than perceived by the performing mid-career managers. Skills Significantly Different at the .001 level Maintains own decisions when appropriate Identifies creative possible approaches to problems Identifies practical/political limitations on various solutions Sees the underlying causes of problem Sees the obvious Evaluates people accurately Sees relationships among elements of a problem Developments in Business Simulation & Experiential Exercises, Volume 10, 1983 116 Identifies practical/political limitations on various solutions Sees the underlying causes of problem Sees the obvious Evaluates people accurately Sees relationships among elements of a problem sees critical elements of a problem Displays good judgment Decisive Searches for alternatives Responds effectively to what others are saying Pursues others’ points of view with others Seeks to influence others Seeks information from others Gives information to others Others listen to person’s inputs Others react positively toward person as a leader Plans for long-range objectives Considers contingency actions Prioritizes objectives Sets goals and objectives Considers precedent for future problems of alternatives Prioritizes alternatives Analyzes consequence of alternatives Arrays alternatives Considers the total situation Checks progress of work Does not disturb effectiveness of others with own work Involved in work immediately Skills Significantly Different at the .01 Level Extends degrees of freedom in delegating Holds others accountable with follow-through dates Gives expectations clearly Listens to others Speaks loud enough Builds supportive relationships Receives attention from others Plans for short-range objectives Distributes resources Strategizes before taking action Identifies problems Determines problem possibilities Identifies problem elements Skills Significantly Different at the .05 Level Maintains calm leadership under unusual or adverse conditions Adaptable Energy effectively put into speaking Effective voice inflections Pursues own point of view with others Can be very directive Breaks items into do-able parts Conclusion The research showed that assessees' self-appraisal rating is consistently higher than the converged assessors’ view of their performance on the same activities. This tendency to rate higher was much greater for the mid-career manager than for the full-time graduate students. Perhaps Managers tend to assume their work success is based on a higher Quality of management skill than they possess. The results of the Hive as a pictorial means of specifically rating behavior is encouraging. The most observable reversal in behavior has been the evidence of a cognizant impact on the assessees. When the feedback is received there are a few initial sightings, denials, of rationalizations; but preeminently a discomforting awareness that perhaps their preconceptions of personal management skills might need adjustment. Those participants whose skills are assessed core favorably than they had appraised themselves appear more confident. Students have tended to engage in studying their results and formulating more specific and systematic development plans than were before possible. Elaboration on the follow-up development strategies is not possible in this paper. However, periodic development counseling encourages assessees to take no more than three priority skill areas at a time on which to concentrate their Improvement plans. Since we are now considering a much more comprehensive and specific development plan, and awareness of a need for development consideration is heightened, a limitation in sequential development strategies is critical to avoid overwhelming the student. Perhaps the most rewarding benefit of the Hive assessment occurs when promising students ask “How do I build those strengths’? Row can I overcome those weaknesses? How can I develop those skills?” Thanks to Hiving there is a more systematic way to analyze the feedback, visualize the skills, and track the growth. REFERENCES 1. R. J. Klimoski and W. 3. Strickland, “Assessment Centers--Valid or Merely Prescient,” Personnel Psychology, 1977, 30, pp. 353-360. 2. L. C. Nichols and J. Hudson, “Dual-Role Assessment Center: Selection and Development,” Personnel Journal, May 1981, pp. 380-386. 3. D. F. Hoyle, “AT&T Completes Assessment of Nearly 1700 Women Under Comment Agreement,” Assessment and Development, 1975, 2, pp. 4-5. 4. D. W. McKinnon, “An Overview of Assessment Centers,” Center for Creative Leadership, Technical Report: No. 1, 1975. 5. A. Howard, “An Assessment of Assessment Centers,” Academy of Management Journal, Vol. 17, No. 1, March 1974, p. 116. 6. D. Strausbaugh and B. Wagman, “An Assessment Center Examination to Select Administrative Interns,” Public Personnel Journal, July-August 1977, pp. 263- 268. 7. D. Henderson. “Learning Public Management Skills Through an Assessment System,” Innovations in Teaching Public Affairs and Administration, ed. by R. Heimovics and A. Rizzo, University of Missouri and Florida International University, Kansas City; Miami, 1981, pp. 50-59. 8. S. Norton, D. Gustafson and C. Foster, “Assessment for Management Potential: Scale Design and Development, Training Effects and Rater/Ratee Sex Effects,” Academy of Management Journal, Vol. 20, No. 7, 1977, pp. 117-131. 9. D. Stosich and D. W. Henderson, Organizational Development Hive, (Idaho Falls: Worldwide Achievements Corporation, 1981). 10. Campbell, M. Dunnette, R. Avery, and L. Hellewik, “The Development and Evaluation of Behaviorally Based Rating Scales,” Journal of Applied Psychology, Vol. 57, 1973, pp. 15-22. 11. Development Dimensions, Catalog of Assessment Development Exercises, Pittsburgh: Author Publishing, 1975. Table of Contents Volume 10, 1983 Is the Computerized Business Simulation Relevant? Business Professionals Play a Student Game A Methodology For Assessing the Internal Validity of Business Simulations A Longitudinal Study of the External Validity of a Business Management Game Policy Analysis and Decision: A corporate Relocation Simulation Exercise Effective Listening: An Exercise in Managerial Communication The Symbol Exercise: An Initial Group Activity Concept Based Simulations Institutional Users of Experiential Learning Packages: A Preliminary View from Publishers' Adoption Lists How to Use Business Games in the Business Policy Course: The Students' Perspective Professors' Ratings of Business Policy Learning Methods Moot Trial: An Exercise in Trial Procedure and Evidence The Advertising Agency Game: An Experiential learning Exercise The Use of Videotaped Cases in Teaching Information Acquisition and Decision-Making Skills Experiencing Information Processing Strategies as a Means to Explore Decision Making Importance Ratings and Operations Data as Predictors of Business Game Performance Determinants of Performance in Computer Simulations Predicting Business Game Performance form Perceptions of Manager Information and actions Business Consulting: A Practicum for Undergraduate Internship as a Contingency Based Experiential Learning Program for More Effective Organizational Socialization: A Conceptual Framework MANSYM III Decision Support System Demonstration Learning the Concept of market Value Through Simulation Conflict Management for Economic Developers Development of Data Analysis Units Designed to Enhance Reasoning and Knowledge Transfer in the College Level Course BOSS: A Behavioral-Quantitative, Computer-Supported Game Entrepreneurial Potential: An Experiential Exercise in Self Analysis and Group Assessment Development of Strategists: Simulated Cases BANKRUPT: A Deceptively Simple Business Strategy Game Simulating Market and Firm Level Demand - A Robust Demand System COMPSIM A Computer Center Management Simulation Hiving Model: Assessing Management Skill Awareness The Johari Window, A Reconceptualization Role-Playing Based on Video-Tape Scenarios: An Application of Modeling to Building Supervisory Skills How to Internationalize Your Curriculum A Computerized Model of Human Behavior in a Total-Firm Management Simulation the Worksheet Approach for Simulation Game Strategy Analysis Teaching Competitive Bidding Using a DSS Generator Do We Learn from Experience? The Use of Theory Power for Increased Research Momentum in Business Simulation and Experiential Exercises Research Report on Programmatic Research on Perceived Learning Barriers with Simulation and Experiential learning An Empirical Examination of Conflict - and Nonconflict - Oriented Problem-Solving Technologies Management Curriculum: 1982 Humor as a Management Tool: Use in Formal Game Presentations Student Behavioral Change Through Teacher behavioral Change