TRAINEE V. TRAINEE SUBORDINATES EVALUATION OF EXPERIENTIAL LEARNING Developments in Business Simulation & Experiential Exercises, Volume 9, 1982 178 TRAINEE V. TRAINEE SUBORDINATES EVALUATION OF EXPERIENTIAL LEARNING Lane Kelley, University of Hawaii ABSTRACT Student evaluations. They are “objective”. The professor’s mean response on “how useful the material covered is for career preparation” is 3.21 on a numerical scale of 4.00 which is, significantly different than the colleges mean of 3.11 at the .01 level of significance. Is this really significant? In our experiential courses, we are probably evaluating our courses by this same method - the student evaluation. The purpose of this paper is several fold. Primarily it is to address the questions of evaluating training and secondly to describe a training model which seemingly is appropriate within an organizational behavior context. The research design illustrates the weakness of student/trainee evaluations and the evaluations of others. It also shows the necessity of pre and post measurements of change with control groups. The purpose of this study was to evaluate the effect of a three-day session using an integrated didactic and experiential approach for training in interpersonal skills for supervisors. The integrated didactic and experiential training model is outlined in Toward Effective Counseling and Psychotherapy: Training and Practice by Charles B. Truax and Robert R. Carkhuff and also in Truax’s “The Training of NonProfessional Personnel in Therapeutic Interpersonal Relationships,” American Journal of Public Health, October, 1967. The objective of this training is to increase the trainees’ communication of accurate empathy, non- possessive warmth, and genuineness. The three central elements in this training approach have been summarized by Truax as (1) a therapeutic context in which the trainer communicates high levels of accurate empathy, non- possessive warmth, and genuineness to the trainees themselves; (2) a highly specific didactic training using research scales for “shaping” the trainees’ responses toward high level of empathy, warmth, and genuineness; and (3) a quasi- group therapy experience which allows the emergence of the trainee’s own idiosyncratic therapeutic self through self-exploration and the consequent integration of the didactic training with personal values, goals, and life styles. The two trainers who conducted the session had considerable experience with the training model. Both trainers also had experience in rehabilitation services and counseling. A questionnaire administered at the end of the training to the trainees indicated that the trainers satisfied the requirement of communicating high levels of the therapeutic triad. In order to evaluate the training, the Barrett-Lennard “Relationship Inventory" was administered to the subordinates of “trained’ and “untrained’ supervisors before and thirty and ninety days after the training. The “Relationship Inventory” is designed to gather data and provide measuring scales for four variables: empathic understanding, level of regard, unconditionality of regard, and congruence. The trainees also evaluated the training in terms of its ability to increase the three behavioral objectives. THE TRAINING MODEL The training model for this study was constructed by Truax and Carkhuff and is described at length in their text, Toward Effective Counseling and Psychotherapy: Training and Practice. The three central elements in this training approach have been summarized by Truax as (1) a therapeutic context in which the supervisor communicates high levels of accurate empathy, non- possessive warmth, and genuineness to the trainees themselves; (2) a highly specific didactic training using research scales for ‘shaping” the trainees’ responses toward high levels of empathy, warmth, and genuineness; and (3) a quasi-group therapy experience which allows the emergence of the trainee’s own idiosyncratic therapeutic self through self-exploration and consequent integration of his didactic training with his personal values, goals, and life styles. Since the purpose of training for interpersonal relationships clearly involves attitudinal and behavioral change, the variables found to be effective in psychotherapists are logically applied to trainer-trainee relations. This means that the trainer should provide the conditions of empathetic understanding and non- possessive warmth for the trainee in a relationship characterized by genuineness. The didactic training places emphasis upon the direct teaching, structuring, or shaping of the thinking and responding of the trainee. Learning and training of the therapeutic triad takes place within a structured situation using the “Accurate Empathy Scale,” the “Genuineness Scale ,“ and the “Non-Possessive Warmth Scale.” The aim of the quasi-group therapy experience is first to give the trainees experiential meaning for their role and by their own participation. The second purpose is to provide an opportunity for self-exploration of their own goals, values, and experiences. it is hoped that this encounter will allow the trainees to move toward integrating their personality, values, and goals with the didactic and cognitive learning. This experience is not aimed at uncovering deep emotional problems, thus the term quasi-group therapy. ANALYSIS OF DATA The research for this study involved a pre-training, post- training (thirty days after) , and follow-up (ninety days after) administration of the “Relationship Inventory” in order to determine the effectiveness of the Truax-Carkhuff Training Model for a three- day training session. The “Relationship Inventory” was administered to subordinates of trained and untrained supervisors. The training session was also evaluated by the trainees. The symbols for each test were specifically, F = empathy, UR = unconditionality of regard, LR level of regard, and fl genuineness mean scores. “A denotes the experimental group and “B” the control group. The number “1” stands for the pre-training test, “2” for the post-training test, and “3” for the follow-up test. Developments in Business Simulation & Experiential Exercises, Volume 9, 1982 179 Table 1 is a summary of the statistical test of the subordinates of trained and untrained supervisors for the specified variables. Empathy. There were no significant difference between the experimental and control group empathy scores for pre, post, or follow-up testing. There was change in the experimental scores between the pre, post, and follow-up period but when you compare the change in the experimental versus the change in the control, it is not a significant change. Level of Regard. There was no significant difference between the experimental group’s level of regard scores and the control group’s level of regard scores before or thirty days after the training. There was a difference ninety days after the training. This change was also evident within the experimental group and the change was significant with the experimental group showing positive change the control group negative change. Unconditionality of Regard. The experimental and control groups unconditionality of regard scores were not significantly different for the pre, post, or follow-up measurements. The changes within groups and change versus changes measurement also were not significant. Genuineness. There was no significant difference between the genuineness measurement of experimental group and control group for the pre, post, or follow- up tests. There was no significant difference between the pretest and posttest scores of the experimental group, but the difference between the experimental group’s pretest and follow-up scores was significant at the .05 level. The experimental supervisors were perceived as being more genuine ninety days after the training. The statistics for testing the difference in the change in means of the experimental versus the change in means of the control group are presented in Table 1. The change in scores was significant at the .05 level of confidence. A questionnaire was completed by the trainees so that there could be a comparison between the “trainees” assessment versus the changes in behavior as assessed by subordinate - the ultimate objective of the training. The trainees assessment was that it was effective in terms of changing behavior for all three variables and perhaps a little more effective for genuineness than empathy or communication of warmth. It is important to compare these findings to the results of the trainees’ evaluation questionnaire. If this training session had been evaluated only by the responses of the trainees, it would have to be judged very successful. All but one of the trainees rated this training session at least above average in its effectiveness in improving the communication of empathy, non-possessive warmth, and genuineness. Evaluating this training in terms of the perceptions of trainees’ clients did not indicate the same results. SUMMARY The statistical tests did not support the effectiveness of the Truax-Carkhuff Training Model used in a three-day training session for sheltered workshop supervisors. In terms of communication of the therapeutic triad, there was a significant increase in the trained supervisors’ communication of empathy, level of regard, and genuineness, but when compared to the untrained supervisors (control group), only level of regard--which is one of the two requirements of non- possessive warmth--was significant. Conversely, the trainees’ evaluation of this training was very favorable. The trainees felt that the session increased their communication of empathy, non-possessive warmth, and genuineness, but their clients’ feeling of their communication of these three variables did not substantiate the effectiveness of the training. What, does this mean? First, that we can’t evaluate training without the use of control groups. In some cases, the changes of the control group’s measurements allowed for the significance of change between the two groups. It is interesting to note that the control group subordinate perception tended to become more critical perhaps more honest as time passed. Second, that we need to differentiate between possible learning objectives. Are the changes expected cognitive knowledge, acquisition behavior, attitude, etc? In this study, the objective of the training was the communication of empathy, warmth, and genuineness the behavior of the supervisor as perceived by his/her subordinate. A differentiation in this context is very important because if we had attitude change as the primary objective, it would have been much more difficult than behavioral change. I can learn to maintain eye contact which communicates interest in the individual without changing my feeling about the individual. Changing my feelings takes a greater depth of intervention which means changing me, how I deal Developments in Business Simulation & Experiential Exercises, Volume 9, 1982 180 and feel about the world and others. Third, it forces the question of the objectivity of trainee evaluations. Previous research by this author indicates that the relationship of the trainer-trainee effects the trainee's assessment of the training. Given that most training is evaluated by the trainees, instructors/trainers must be aware of the impact of that relationship and their training success. REFERENCES (1) Truax, Charles B., “An Approach Toward Training for the Aide-Therapist: Research and Implications,” invited paper, Symposium on Non-Traditional Preparation for Helping Relationships, American Psychological Association Convention, Chicago, Illinois, September 5, 1965. (2) Truax, Charles B., Counseling and Psychotherapy: Process and Outcome, Arkansas Rehabilitation Research and Training Center, Fayetteville, Arkansas, 1966. (3) Truax, Charles B., “The Training of Non-Professional Personnel in Therapeutic Interpersonal Relationships,’ American Journal of Public Health, Vol. 57, No. 10, October, 1967, 1778-1791. (4) Truax, Charles B., Robert R. Carkhuff, and John Douds, “Toward an Integration of the Didactic and Experiential Approaches to Training in Counseling and Psychotherapy,” paper presented at the Summer Workshop in Psychotherapy and Psychotherapy Research, Wisconsin Psychiatric Institute, University of Wisconsin, June 26, 1963. (5) Truax, Charles B. and R. Carkhuff, Toward Effective Counseling and Psychotherapy: Training and Practice, Chicago, Aldine Publishing Co., 1967. (6) Truax, Charles, B., ‘The Training of Nonprofessional Personnel in Interpersonal and Counseling Skills,” unpublished paper. (7) Truax, Charles B. and Jeffrey G. Shapire, “Training Non-Professional Personnel in Interpersonal and Counseling Skills,” unpublished paper. Table of Contents Volume 9, 1982 The Value of Conjoint Analysis in Enhancing Experiential Learning The Effect of Participation in a Collective Bargaining Simulation on the Expectations and Attitudes of Union and Management Representatives Conflict Resolution in Experiential Learning The Use of Simulation to Test Theories of Bargaining in a Business Context The Nominal Group Technique: A Vehicle for Improving Case Method Courses A Framework for Developing a Business Policy Case Produce Your Own Video Cases for Classroom Use: A Demonstration An Experiential Effect from Charismatic Encounters Nine Topic Oriented Mini Simulations: Descriptions, Purposes, and Observations A Hospital Simulator (HOSPSIM) A Report of the Model and Results Expected from Field Testing An Evaluation of SLIM (A System Laboratory for Information Management) An Experiential Exercise Introducing Students to the Role Ambiguity Faced by Salespersons The Advertising Research Project as an Innovative Experiential Learning Technique The Small Group Research Project: An Experiential Learning Approach for Undergraduate Marketing Research Students The Relationship of Cognitive Style Maps to the Preference for Experiential Learning of Undergraduate Students The Learning Style Inventory Debate Revisited: An Empirical Assessment fo the Construct Validity Issue Related to Experiential Learning Theory Conducting a Classroom to Facilitate Career Goal Attainment Corporation Executives' Ratings of Policy Learning Techniques The Generation and Application of Evaluation Criteria for Management Policy and Strategy Simulation Games Experiential Opportunities with Microcomputers Leading Students to Learning: The Teacher's Obligation An Experiment in Teaching Principles Courses: The Mini Debate Formula Developing Creative Thinking Through Experiential Learning Heuristic and Systematic Evaluation of Policy: Exercise in Decision Making A Case Study Approach for the Litigation Decision: Employing Decision Analysis to Determine When a Business Should Settle or Go to Court Learning Negotiation Skills Through Simulation Union vs. Management: A Simulation of Collective Bargaining in Action Inside the Black Box: An Analysis of Underlying Demand Functions in Contemporary Business Simulations A Review of Channel Exercises and the Description of a New Alternative SIMCON I: A computer Based Simulation Model for Evaluating Physical Distribution Strategies Involving Order Consolidation Toward Competency-Based Management Education: The Interpersonal/Communications cluster The Value of Pre-Teaching in Role Playing Some Effects of Positive Personal Reinforcement upon Socializing Students in an Experiential Learning Course Who Gains and Who Does Not from Experiential learning Toward the Ultimate Experiential Exercise, the Student View Simulating Professional Writing Experiences in the Classroom The Johari Window as a Measure of Personal Development Windows Into Management: A Participative Aid to Learning Get Your Faculty Involved A Pedagogical Approach to Business Gaming for the Commuter Student Super Service for Computer Game Administrators Expand the Role of Simulation with Creative Scenarios The Delivery, Administration, and Evaluation of an Executive Development Program Using a Total Enterprise Business Game An Application of Experiential Learning in International Trade and Foreign Direct Investment International Management: Building Bridges Analysis of a Business Simulation Exercise: Organizational Survival and Success The FALRIS Organizational Scavenger Hunt Trainee V. Trainee Subordinates' Evaluation of Experiential Learning Longitudinal Analysis of an Innovative Teaching Intervention Student Perceptions of Effective Teacher Behaviors Revisited Realism and Learning: The Evolution of a Management Game A Merger and Acquisition Simulation A Stock Market Investor Simulation Experiential Learning in Consumer Behavior: Perceptual and Attitude Change Exercises The Recycling Industry Problems of Women in Management: A Role Playing Exercise for a Course in Contemporary Organizational Problems Experimental Three Weekend Course: Empirical Results A Process for the Analysis and Development of Course Content and Instructional Methodology for Large Class Sections Problems Associated with the Assessment of Experiential learning Using the Multiple Choice Test Combining Lecture and Simulation Teaching Methodologies Qualitative Determinants of Team Performance in a Simulation Game The Effects of Different Team Sizes on Business Game Performance Comparison of Problem-Solving Technologies: A Free Simulation Approach Consistency in Business Games A computer Simulation of Personnel Selection Decisions Giving Praise Exercise Job Enrichment A Look at the Spoken and Written Word in Organizations A Pattern of Group Communication A New Generation in Business Simulation Adapting Mainframe Business Simulations to Min Computers File Access is the Essential Prerequisite to Time-Flexible and Interactive Computer Simulation A Microcomputer Simulation for Teaching Retail Location Strategy Blocks & Chips: A computer-Assisted, Geno-Typical Entrepreneurial Game Development of a Self-Paced Course in Business Statistics The Involvement of Student Bodies in the Teaching of Advanced Technical Concepts Systems Learning Sequence: An Experiential Course Module for Management Information Systems An Experiential Approach to Developing Managerial Competencies Communication Research, Inc. An Experiential Learning Activity Developed as a Practicum for a Course in Organizational Communication