Voices from the Trenches: Personal Reflections of an MBA Program’s Experiment with an Experiential Learning Course in Executive Skills Page 29 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 ABSTRACT Three instructors with extensive experience teaching a behaviorally-based required MBA course in executive skills utilized whole person experiential learning methodologies over a period of years. This paper discusses the rewards, opportunities, challenges, and frustrations these instructors encountered. Borrowing from the format employed by the Academy of Management Learning and Education’s inaugural issue article on the impacts of 9/11 on instructors and classes (Fukami, 2002), extensive questionnaire responses were pooled and reported in detail. Brief conclusions and implications for educators were drawn from these in-depth responses. However, we leave it to the informed ABSEL reader to make a more detailed assessment of how well such a concept might apply at any given institution and whether or not to adopt such a model wholesale or with pragmatic modifications. OVERVIEW Pursuant to a framework developed by Hunt and Sorenson (2001) which was itself an extension of Kolb’s (1984) experiential learning theory, a large southwestern university implemented a behaviorally-based MBA core course in executive skills. Once implemented in the MBA program the course methodology and mechanisms for delivery took a few years to converge into a consistent experiential learning format (Hoover, 2005). This format featured the use of assessment centers in pretest/posttest fashion, with experiential and behavioral modules between the pretest and posttest as the educational “treatment condition” being utilized (Hoover, Giambatista, Sorenson, & Bommer, 2006 and 2010). This course has been alternatively the most rewarding and, at times, the most frustrating, course that nearly every instructor associated with it has taught. Much like the Reynolds (2007) mixed outcomes model of harnessing Pygmalion while restraining Golem, we found aspects of the course’s development to be some of our most creative and enjoyable teaching experiences, combined with some of our greatest teaching challenges. It has also proven fruitful for empirical research testing the pedagogy via improvement in assessment center scores (Hoover, Giambatista, Sorenson, & Bommer, 2006 and 2010), testing the efficacy of a traditional 15-week semester versus an immersive 4-week summer format (Giambatista & Hoover, 2007), and examining the role of direct versus vicarious learning and for individual differences correlating with proficiency gains (Hoover & Giambatista, 2009; Hoover, Giambatista, & Belkin, 2012), among other projects. Despite this research, we who have been personally involved in both teaching and empirical investigation have been left with an empty taste in our mouths, in the sense that we have not found an outlet to convey our most personal and deeply held thoughts and reflections on being “in the trenches,” so to speak. Further, most literature we read either takes an empirical approach to pedagogy or tends to advocate particular pedagogies and approaches, while glossing over potential costs and challenges associated with teaching methods. This article captures our determination to report in depth our experiences with and reflections of the pedagogy, the good, the bad, and sometimes even the ugly. We believe ABSEL educators would potentially benefit through exposure to such challenges and rewards to make informed pedagogical and curricular judgments. This paper seeks to rectify such omission. Using a format effectively employed by the Academy of Management Learning and Education (AMLE) in its 2002 premiere issue (Fukami, 2002) , we developed a structured questionnaire featuring open-ended items regarding experiences with the course and received feedback from three instructors, which we are reporting with minimal editing. We then use these data to develop a model of factors that we propose to be correlated with the success of such a course. This conceptual model, we hope, will prove fruitful for both ABSEL educators and ABSEL researchers interested in experientially- or behaviorally-based managerial education. Voices from the Trenches: Personal Reflections of an MBA Program’s Experiment with an Experiential Learning Course in Executive Skills J. Duane Hoover Texas Tech University duane.hoover@ttu.edu Robert C. Giambatista University of scranton robert.giambatista@scranton.edu Emilia Westney texas tech university e.westney@ttu.edu mailto:duane.hoover@ttu.edu mailto:robert.giambatista@scranton.edu mailto:e.westney@ttu.edu Page 30 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 BACKGROUND OF THE COURSE The course in executive skills that provided the basis of this questionnaire and model features the following structure: a) An assessment center spanning a 3 hour class session serving as a pretest and baseline of behavioral skills in the following five areas: communication, decision making, teamwork, leadership initiative, and planning and organizing. b) Behavioral skills teaching interventions spanning 8-10 modules emphasizing these content areas, approximately one content area per week, and augmented by other related behavioral skill areas such as conflict resolution and ethical decision-making. c) A second administration of the assessment center using the same basic format but using a different workplace scenario as its basis serving as a posttest measure of behavioral skills. The course concludes with a behavioral final utilizing teamwork and presentation skills and an individual assignment labeled the Individual Development Plan (IDP) that affords each student the opportunity to tell the story arc of the skills acquired and personal development experiences over the course of the semester. The assessment center pretest/posttest format possessed the benefits of: 1) establishing a baseline of skill measurement, 2) focusing upon the selected skill areas as learning targets, 3) establishing post treatment levels of skill measurement and improvement, and, 4) reinforcing student appreciation of the importance of these behavioral skills. This format allowed for consistent skill assessment while providing a differing set of performance challenges, and also minimized test-retest contamination with true skill acquisition. The specific assessment center we employed has been validated and employed in other published studies (Hoover, Giambatista, Sorenson, & Bommer, 2006 and 2010) and is discussed in greater detail by Rode et al. (2005). In summary, the assessment center comprises a 3-hour videotaped behavioral sample and employs trained raters to reach consensus on the presence, absence, and sometimes magnitude or effectiveness of specific behaviors against a behavioral checklist. THE INTERVIEW DATA Below we report responses to the interview questions by Instructor 1 (I1), Instructor 2 (I2), and Instructor 3 (I3). Over a seven-year period, I1 taught 8 sections of the course, I2 taught 27 sections, and I3 taught approximately 30. In the first half of this time period, the class was taught in standalone sections, but in the latter years it mutated into a lecture and lab format. This mutation affected only some delivery mechanisms such as course scheduling, and did not alter the basic course format or the course’s experiential pedagogy. Question 1: When you taught Exec Skills, what were the specific behavioral skills you aspired for students to acquire? Were there any behavioral skill inadequacies or deficiencies that you hoped the course would curtail or eliminate? I1: Over time, I came to believe active listening represented the single most important skill set. It has pervasive applications to decision-making, conflict resolution, teamwork, and leadership of teams. It also runs counter to most students’ natural communication styles, so it is quite difficult to justify and to help students internalize. One crucial deficiency of our students was the lack of a process orientation, which affected several areas, particularly teamwork. Students seemed to possess implicit theories of teamwork revolving around hard work and submission, rather than facilitating the contributions of others and moving the process forward constructively. Student overconfidence combined with this demonstrated simplistic mental model to produce a formidable yet imperative obstacle to overcome. I2: I want students to get a better feel for what it is like to deal with managerial issues and problems that hinge around the realities of interpersonal relationships and dealing with people in a work setting. It seems to me that a lot of the business school and MBA program content proceeds on the unrealistic assumption that decision making occurs in an orderly environment with few disruptions, and that there is plenty of time to get it all done. Most MBA decision-making scenarios have the people involved either unmentioned, dormant or cooperatively providing all of the needed data. My managerial experience is that there is never enough time, people are usually not dormant, but are more in your face, and the needed data is either unavailable or is being withheld by people with a bevy of differing motivations for doing so. I would hope that real time experiential exercises in a skills oriented class would put at least some premium on the full realities of organizational decision making. I am also hopeful that students could acquire specific skills that have real world application. An important skill set is having what it takes (and being willing to do it) in order to actually listen to another person as well as the capacity to be heard by others when it’s important to get a message across. A capacity to make effective presentations is also valuable---this includes the ability to be persuasive as well as the ability to demonstrate personal knowledge and competencies. I hope that students could get a real world feel for the challenges of using skills from a personal maturity perspective. This is especially important for younger students or for students with limited work experience. The challenges of skill execution, over time, give a manager a better sense of self and, hopefully an enhanced sense of humility. I am concerned that many of our MBA graduates come out of the MBA program armed not with perspective, but with arrogance. As such, they may become ticking time bombs in the work place, waiting to make errors of judgment with real and lasting consequences. I3: The majority of students are highly task oriented, which can present problems when involved in teamwork. I hoped that the course would curtail the instinct to merely divide up team assignments and parcel out a collection of tasks, and encourage more dialogue, sharing of information, consensus decision making, and true collaboration within teams. Page 31 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 Question 2: In the effort to ensure students’ behavioral skill development, did you try to apply specific principles of experiential/ behavioral learning? If so, please describe how you applied them. I1: I tried to use all four elements of the Kolb framework with mixed success. I would generally open with 20-30 minutes where I discussed skill elements and their mastery, and their basis in OB theory and practice. I would provide multiple examples of important points that students could relate to. In conducting role plays, students were encouraged to experiment with different methods of interacting to triangulate with and against their dominant styles, and to compare the outcome of the role play versus how things might have turned out otherwise. Reflective observation, however, was problematic. I am a firm believer in vicarious learning, but when I attempted fishbowl-style role-plays, non-participants generally seemed disinterested. The Individual Development Plan (IDP) integrated many elements of Kolb, but as it was due at the end of the semester, feedback from the IDP often did not reach the student. I2: I prefer the term andragogy to pedagogy because I think it is important to treat MBA students as adults (Forest and Peterson, 2006). If nothing else it can function to create a sort of self-fulfilling prophecy of student maturity and personal responsibility. I also think that our methodologies should be focused as much as possible on learning impacts that affect the whole person, including emotion (affect) and behavior in addition to cognitive stimulation (Hoover, 1974; Rogers, 1980; Hoover, 2007). To the extent that our educational systems function in this fashion, we are able to combine the emotional/affective and behavioral domains with the cognitive domain always found in educational processes (Boyatzis, Stubbs and Taylor, 2002). There are several tools that can facilitate this process. One is what we call an IEBF paper. After each experiential exercise, students can write a first person paper in which they dissect and react to the just completed exercise by writing mini-essays on the Intellectual, Emotional, Behavioral and Future Focus dimensions. Personal Learning Journals can also be used as a record keeping device, allowing students to record their reactions to just completed exercises (Kalliath and Coghlan, 2001). And, finally, an Individual Development Plan paper can be used to focus the entire course (Hoover, 2005). Modeled after corporate performance appraisal models, the IDP becomes the student’s report of their individualized journey of executive skill development. It begins with a Personal Mission Statement, integrates course experiences into individualized learning, and concludes with a 2 to 5 year plan looking forward. I3: A major guiding principle I have tried to apply is the widely accepted notion that students learn more effectively when they are actively engaged during class. When I became involved with the course in fall 2004, the curriculum had already contained many activities that incorporate some active learning techniques such as small group discussion, role plays, etc. I have attempted to build on this aspect of the course by encouraging class discussion as a mechanism for sharing experiences and perspectives and for probing and clarifying their own thoughts about a subject. I also introduced several worksheets that serve to focus the students’ attention on a particular problem or concept; or that get them to apply certain principles taught in class to specific situations; and/or that help them examine their attitudes and form their own conclusions about a subject. I also endeavor to apply two adult learning principles that I believe are particularly important in this class. The first is the principle that learning is more likely to take place when students perceive that the subject is relevant, that it has real and direct application in their lives. I have addressed this primarily through the prebriefing before an exercise. This can sometimes mean walking a fine line during a prebriefing because it is important not to ruin the element of surprise and discovery essential to experiential learning. At other times, I attempt to engage them in discussion during the debriefing to help them process their experience of the exercise and to see its relevance. The other adult learning principle I have tried to apply is the need for specific feedback. Student presentations and written papers provide excellent opportunities for feedback specific to a student’s performance. Question 3: From your experience, what if any critical incidents have been most influential in shaping your opinions about the course, its potential, and its effectiveness? I1: Watching the students genuinely immerse themselves in the assessment center was probably the most rewarding critical experience I had with the class. Most of my other positive opinions stem from numerous small and non-critical incidents, such as seeing students engaged in the role playing tasks. When the course evolved into a pre-test, teach, post-test design, improvements on the post-test were quite gratifying. On the negative side, I distinctly recall a student scoring an 88% on some trivial assignment and confronting me in an extremely threatening, Columbine-esque tone and snarling, “I BETTER get an A in this course,” before storming out of the room. That seemed to typify the attitude many students had of the class, summarized as follows: It’s B.S. and remedial in nature, therefore I deserve an A, therefore any feedback below “A” is a personal insult and should be blamed on the teacher and the material. This is classic self-serving bias, but very difficult to contain and very demoralizing to me personally. It also belies an underlying attitude that many students are not here to learn, but to write their ticket to a post-degree job. I2: I’ve seen an overemphasis on cognitive frameworks by some well-intentioned professors drive the value out of the experiential approach. I’ve also seen insufficient cognitive structuring that negates the accomplishment of whole person learning. Whole person learning has to be whole and balanced to be effective. The critical incident for me in the entire experience, however, revolved around students without sufficient personal maturity or life experience to both perceive and harvest the benefits of the course. Klimoski (2007) has suggested that we can do only so much during the brief period of time the average MBA program represents. Thus, “assuming that we want such qualities in our graduates, they might be better seen as factors for selection into a program rather than something to be developed within the curriculum experience” (Klimoski, 2007: 434). But for many of us who work in universities that are enrollment driven or operate with funding formulae, there are no such options. Thus we find ourselves with a situation where “schooling” has been defined by the processes of crunching, processing and Page 32 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 regurgitating data for many of our students, and they are frustrated by a learning environment that requires them to focus on the process as well as the content of learning. Greenberg, et al (2007) labeled this cognitively limited framework as an Athenian orientation; it is not compatible with a whole person experiential class such as executive skills. I3: Two critical incidents come to mind. The first was when we experimented with having students, rather than instructors, grade their own short-essay quizzes in an attempt to enhance their own and each other’s’ learning of the material. After taking the quiz, students met with teammates to pool their knowledge and determine what would constitute the correct answer, assessed each other’s’ answers and assigned a grade to each individual’s quiz. The result, however, was an average quiz grade much higher than any instructor-graded quiz; they were clearly far more lenient in their grading and less discerning than the instructors would have been had they graded the quizzes. The result was some unforeseen behaviors we do not want to encourage, such as opportunism rather than self-development, and group dynamics that promote cohesion at the expense of maintaining standards. This experience brought to mind the importance of structuring reward systems so that they motivate the behaviors we want to encourage. Another aspect of the course that has shaped my opinions is the students’ own action plans for their further development. (This is more a series of observations and less a “critical incident” but it has had a strong influence on my views nonetheless.) Those students who create what I consider to be a good action plan are the minority; the majority of goals are too general or vague (e.g., “improve communication skills”), and don’t have clearly expressed action steps for accomplishing them. I have concluded that this is the result of several factors. Once the IDP is written, there is no follow up or accountability for meeting goals, unlike in the workplace setting, where employers provide motivation by encouraging, monitoring, and rewarding self- development. The act of identifying strengths and weaknesses may be fundamentally uncomfortable for many students (particularly the weaknesses part), so they do not delve deeply enough to identify goals for themselves that would address these. Many of the students in our class have little or no significant work and life experience and lack a perspective in which they can apply the skills taught in the class. Finally, some students are fundamentally not goal oriented, so the act of setting goals and striving to achieve them has little significance. Question 4: What are the biggest obstacles and challenges you have faced in trying to deliver a high-quality experiential/behavior learning experience in the course? I1: By far the biggest problem was students’ resistance to change and unwillingness to open up to the possibility that the course might transform them in a positive way. This seemed to affect at least half the class and was much more pronounced among men than women. Readers surely know the type: undifferentiated baseball-hat-in-the-back-rows guys. This is in part a problem with our student body, as we had a disproportionate number of students who began the class barely three months past their graduation from our undergrad business school. These students arrived in the MBA program generally as a second or third resort. Their preference was clearly either to get a desirable job out of the undergrad program and/or to land a more prestigious MBA placement. Thus, they carried palpable chips on their shoulders. There was also the related issue of maturity. Later in their careers, they might be more open to change and improvement on their interpersonal skills, but many people that age simply don’t see how these skills are important or require improvement. Another related issue was the compulsory atmosphere in the required class that incrementally contributed to negative attitudes. But if we dropped the requirement, few would take the course and those that needed skill improvement most desperately would generally be those least inclined to enroll. I2: Taking students from the state we find them in and then facilitating their transformation into something “new and improved” has been called a Promethean approach (Greenberg, 2007). But Promethean efforts are thwarted when students have little interest in learning skills. Many, perhaps even most of our students seem more interested in taking the kind of traditional courses that they are used to; or, more specifically they are more comfortable in those areas in which they are already proficient. Student burnout turned out to be an ongoing and unanticipated challenge. Most of the literature of experiential learning seems to assume that the greater levels of personal involvement afforded by experiential learning keeps student interest piqued. However, in our executive skills class, by the end of the semester even those who were “getting it” seemed to flame out a bit at the end. Perhaps this was from emotional overload or the burden of continuing self-inspection. Those in the course just to get a grade or those who wanted to just get past the course perhaps came to perceive that they were being force fed. In any event, they also seemed to grow increasingly weary of the process. One semester we tried a new schedule of breaking in the middle. That seems to have lessened this problem at least to some extent. A final challenge was generating appreciation by colleagues into what it takes to do Exec Skills correctly. This is a factor even in our strongly collegial Management Area, and a substantial hurdle when presenting to the multi-disciplinary College MBA Committee. I3: My answer to this question falls into two broad categories: (1) the problems inherent in teaching a course of this nature in the academic setting, and (2) the nature of the student body in our classes. CHALLENGES WITH THE COURSE ITSELF GRADING One of the major challenges is the issue of how to evaluate our students. But which skills do we evaluate? Some skills are fairly easy to observe and measure; presentation and writing skills are good examples. However, other skills, such as decision making and teamwork, are more difficult to observe although they may have specific behavioral skills associated with them. As a result, some of the key components of the course are either not assessed at all, or are assessed on unrelated criteria. For example, the Page 33 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 grades for teamwork are based on a team paper and several team presentations; it is the team’s output that’s evaluated, not the teamwork and processes that went into creating the output. Other skills, such as interpersonal skills, conflict management, and decision making, are not assessed at all. (Although some of these skills are observed during the assessment center that students take part in, and students are provided specific behavioral feedback, that aspect of the course is not given a grade per se.) Related to the challenge of what to grade is knowing on which criteria to grade. Should we grade on actual skill execution? Effort? Improvement? Good intentions? Grading on performance improvement presents a challenge because it requires multiple instances of observable behaviors on which to base a grade; this is not always possible given course time constraints. Grading on effort comes down to assessing the person’s attitude (“Who cares?” vs. “I tried, I really did!”) which may be satisfying for an instructor who wants to reward effort, but which doesn’t really measure what it’s supposed to measure and hence is not valid. There is also the issue of managing students’ expectations regarding grades based on skills. In applied, skill-based programs of study there exists the expectation of grading for accurate performance or skill execution. A music student studying to play an instrument, for example, or a medical student studying to be a surgeon understands and accepts that he or she must demonstrate proficiency, and that although deep knowledge may be required, the person’s skill level will also be assessed and may comprise the major component of a grade. This is not the case for students in business schools who are used to being graded primarily for their knowledge, not for their skills. Ironically, the very skills we teach in the course are often criteria employers use in their performance evaluations of employees (for example, the ability to work with others in teams, to communicate effectively, and to manage interpersonal relationships and conflict). So a challenge with the course becomes getting them to understand the difference between knowing something and being able to do it. SKILL PRACTICE It is difficult, if not impossible, to create the conditions in an academic setting that would allow for sufficient repetition of all the skills we teach. In addition, we cannot provide the kind of ongoing, individualized feedback that would promote skill mastery. Feedback is typically a “one shot deal” with little or no opportunity to try the skill again and get additional feedback or coaching to reinforce the skills. COURSE DURATION Another challenge is related to the length of the course. During the fall and spring semesters the course is taught over a 13- week period. I have found that students appear to be quite interested and stimulated in the earlier weeks of the course, but grow weary and irritable in the last third of the semester. The course’s sustained focus on self-knowledge and self-improvement may be tiring, especially for the younger student who is unused to so much self-examination and who does not fully perceive the need for self-improvement THE NATURE OF THE STUDENT BODY AGE AND AMOUNT OF WORK EXPERIENCE The MBA program at our university does not require work experience for admission to the program. A significant number of students enter the program immediately upon graduation from college and are relatively young. Those with little or no professional work experience are at somewhat of a loss because they have neither the perspective nor experience to apply what is taught, nor an appreciation of the value of the skills taught in the class, all of which makes it very difficult to keep them motivated and engaged. “STUDENT” MENTALITY It is often difficult for the young, inexperienced students to adopt a “professional” attitude about the course because they are too used to being students. Student attitudes and behaviors still prevail because they have not been away from student life long enough to adopt a different way of thinking. The fact that the course takes place in a familiar school environment only helps to perpetuate the student outlook that manifests in bargaining for grades, resisting criticism, adopting an adversarial attitude toward the teacher, and fulfilling assignments out of a desire for a grade rather than for the satisfaction of learning or self-development. PERSONALITY Approximately seventy-five percent of the MBA student body falls into the “SJ” temperament category, according to the Keirsey Temperament Sorter, a personality instrument administered in the class. SJs tend to be highly task and results oriented, like concrete information, and want quick answers. They work best when tasks and assignments are clear and unambiguous. All these qualities make SJs terrific managers (and students) in general. However, these very qualities can cause them to be uncomfortable with some aspects of the Executive Skills course. A good deal of the course involves exploring one’s own strengths and weaknesses, and reflecting on class experiences; there are no right or wrong answers (which SJs often seek) when the subject is oneself. This may cause the SJ some anxiety and resistance, as they are required to venture into this unfamiliar territory. Question 5: What are/were the most successful aspects of the class? I1: The assessment center instrument was received very positively. I’m not sure if students enjoyed the post-test as much as the pre-test, but the assessment center lent credibility and cachet to the class. Another element we used to enjoy a great deal was the Page 34 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 paint ball team building experience. While some students, and myself initially, had doubts with the paint ball concept, the experience itself was exhilarating to most students, even most skeptics. The session paid continual dividends through endless student comments and interpretations in various unfolding class contexts. We have dropped paint ball because a few students were entrenched against it and there was always the minor possibility of an injury. I always felt, however, that a unique, immersive, and metaphorical offsite teamwork opportunity was important to the course. A few of the exercises we did in class were more successful than others, and generally it was important to experiment until finding one you felt confident with. When I was there, the Starfish leadership exercise and the Ethical Decision Making group role play exercise were quite effective. I2: The most rewarding experiences of my teaching career have come from teaching courses such as Executive Skills utilizing whole person experiential learning and seeing students truly “get it.” To me it is like watching a flower bloom in a slow motion film---students truly growing in demonstrated skills over the course of the semester. The double kick comes when they also come to see that they did it, see how they did it, and then finally come to appreciate why they did it. Designing and executing a whole person learning environment is a challenging and rewarding experience for me as a professional educator, but the true realization manifests in the stories of positive change related by students. In retrospect, I have also come to feel that the philosophy behind, design parameters executed, and skillful execution of this course is something that all of us who have been involved can rightfully experience as a feeling of pride. I3: The fundamental thrust of the course is the most successful aspect of the class. Students acquire skills they can apply immediately, and gain self-understanding they will have for life. These skills and self-understandings give them something real, practical, and useful in all aspects of life, not just school or work. In this way, the course is distinguished from many other, more traditional academic business subjects. The subject matter of the Executive Skills course transcends fields of professional study; it is applicable to anyone in any career, and at any point in a career. These really are life skills. One of the more successful aspects has been the grading system for written work and presentation skills. In order to receive a grade for their first paper or presentation, students must demonstrate a minimal level of competency (effectively a B-plus); those who do not meet this standard must revise and resubmit the paper, or redo the presentation, to earn a grade. The highest grade they can receive for a resubmit/redo is B-plus. Since this policy has gone into effect, there has been discernable improvement in papers and presentations in general, and more students achieving the minimal standard. The Iliad assessment center is another of the more successful aspects. Question 6: Discuss the strengths and weaknesses of using the assessment center as a lynchpin for the course. I1: The strengths are many – the assessment center is rigorous, energizing, has instant credibility with most students, provides accurate process and outcome feedback, and helps justify many modules in the class. However, the three-hour format collects a relatively small sample of behavior, thus minor behavioral changes can parlay into significant score changes in the center. Clearly, we were quite dependent on the quality of the raters. This is in part a perceptual issue with students, but can cause some student feedback acceptance issues. Along similar lines, students often blamed the instrument or otherwise externalized their performance, rather than learn from it. The feedback can be quite dissonant from the positive feedback they earned from family, friends, and even educators throughout their lives. A lesser concern was that the assessment center binds us somewhat. We focused on the modules that are emphasized in the instrument, and students wanted us to “teach to the test,” which we assiduously avoided. Also, the assessment center made apparent which modules were central to the post-test versus those that were not, thus non-assessed modules were stigmatized. Finally, confusion often arose when the pattern of scores across the assessed dimensions changed on the post-test. We attributed this to students shifting their attentional focus to one or two dimensions they struggled with on the pre-test, but it served to undermine post-test credibility, particularly to students prone to making self-serving attributions of their feedback. I2: Strengths: Setting a skill focused behavioral baseline at the start of the learning program is invaluable. It gives face validity to much of what follows in the course. The self-efficacy lesson from the goal setting exercise is valuable because it helps some students to be more realistic about their true skills versus their perceived skills. When the feedback is positive from the assessment center results, students perceive this to be meaningful feedback to them. Weaknesses: When the feedback from the assessment center is negative (low or no improvement in scores or low scores overall), students question the legitimacy of the instrument. With that in tow, they can easily transmute to questioning the legitimacy of the course. It’s necessary to avoid “teaching the test” versus teaching the skills and having the test serve as the instrument of measurement. Many students are conditioned to ask “What do I need to do to score well on the test?” as opposed to the more relevant question “What do I need to do to improve my personal set of skills?” I3: Strengths: 1. The course identifies concrete and specific behaviors the students can exercise. 2. The simulated situation approximates a manager’s environment and the demands the manager faces daily, reinforcing the importance of the skills taught in the course. 3. Students learn from the first assessment center which skills they already have strengths in, and which need further development. 4. Participants can instantly apply the skills and behaviors taught in the course, particularly during the second assessment center. 5. Evaluation of the student’s performance is by neutral raters which enhances its credibility. Page 35 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 6. It provides a departure point for the course as a whole and introduces the students to the types of skills they will be working on during the semester. 7. It provides a framework throughout the course that students can relate to. 8. The repeated assessment center gives students specific goals for improvement. Weaknesses: 1. Students who do not perform well may become discouraged about their capabilities. 2. Students who receive lower scores than they expect may discredit the assessment center and reject the results. 3. Resource requirements (facility, equipment and personnel) as well as cost may be prohibitive for some institutions. 4. Course instructors have no input on rating criteria. Question 7: What, if anything, would you change about the course to improve its ability to deliver experiential/behavioral learning? What would you want to remain the same? I1: One thing we’ve already done is to staff the course with educators skilled in experiential methodology. Also, when I served at this university, we relied heavily on the micro skills concept espoused by Hunt & Sorenson (2001). While I liked the concept, I was never able to achieve student acceptance of it in the classroom. It seemed to play into preconceived biases against the class and undermine their potential effectiveness. When I was teaching the class, we were expected to rely heavily on drill and repetition of extremely specific behaviors and verbalizations, and student annoyance and boredom was painfully apparent, thus I eventually refused to do it. I felt it was crucial to student acceptance to provide richer role plays allowing for more improvisation, naturalism, and behavioral complexity. The assessment center should not be altered because of the nexus of its domain, evaluation, rigor, and credibility. Before we incorporated the assessment center, student apathy towards the class was extreme and made the environment almost impossible to overcome. The assessment center provided a critical mass of content and legitimacy that could sustain the course if we didn’t screw the rest up too badly. I2: Smaller sections are needed for effective delivery of the methodology. This became a problem for us when FTE (full- time employee) allocation requirements caused us to change to a lecture-lab format. As a result, for the “lecture” component, we were faced with dealing with a large lecture hall filled up with as many as 100 students. On the other hand, we feel that it is realistic to assume, at least in a state university setting, that faculty resources are not and will not be available to teach a number of smaller sections because of: a) the cost and FTE allocations required, 2) the lack of a sufficient pool of faculty expertise for successful execution, and 3) a lack of willingness of many faculty to undertake the time requirements and challenges involved in successfully executing an experientially based course such as Executive Skills. I3: In an ideal world, we would individualize a program of learning tailored to each student, according to the student’s work and life experience, the student’s own desire to develop skills he or she identifies, and the student’s current level of skill and self- understanding. This is what many employers do with employees they wish to develop for further levels of responsibility. However, given the size of the student cohort this is not practical. In the past, I have met individually with students at the start of the semester to learn more about them, and to give them a sense of what they can expect in the course. However, as subsequent cohorts have increased in size, this has become too time consuming. Other ideas for change to the course: 1. Engage students who have already taken the course (whether still in the MBA program or graduated) as spokespersons to current students to increase appreciation of the course’s relevance. 2. Include more complex case studies and real situations for students to study and analyze, and propose how they would handle such situations in terms of course skills. This could be expanded to include role-plays to demonstrate how they would address the problems in the case. 3. Provide students with more comprehensive practice at interpersonal skills, including listening, empathic communication, assertive communication and conflict management. Evaluate them on these skills through direct observation (live or videotaped). Regarding what should remain the same, I believe we need to keep the two assessment centers as the foundation of the course. Question 8: What is/was your overall satisfaction/frustration with the course when you taught it and why? I1: For me, it was always centered on student affect and mood. Each section had three types of students: those who were engaged, those who were beyond redemption, and those who could go either way. It always came down to critical mass, contagion, and tipping points. Good sections would have sufficient good students to sway the fence sitters favorably and sustain a favorable class climate; bad sections went the opposite way. Further, one negative experience could sometimes turn a class ugly, and it was always hard to retrieve a class from the abyss. One of our colleagues once had a problem operating a troublesome video camcorder during the assessment center pretest, and the class never forgave him for it the entire semester. Tough crowd! If I had a difficult section, I always felt I was walking on eggshells and out of my comfort zone as an educator, and I would fall victim to Golem effects (Reynolds, 2007). Page 36 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 I2: Upon reflection, I now see that an interesting paradox arose in the execution of my teaching effectiveness in the Executive Skills course. When I began to observe that student evaluations were starting to slip, I purposely and systematically “turned up the wick” on the experiential aspects of the course. Whole person learning became more “whole” as emotional and behavioral components were designed to produce learning experiences with more impact. I assumed, perhaps naively, that this would result in more student positive change. What did result, however, was more student negative reactions. So the main frustration for me was that as I produced better methodological executions (at least from a design perspective), I got less student satisfaction. What I perceived to be increased opportunities for personal insight and meaningful skill development was perceived by many students to be more “busy work” at the best and something rather more intrusive at the worst. I3: A major frustration stems from the fact that so many of our students don’t see the need for these skills in their lives, at least not at this point. I believe the general lack of work experience in our students accounts for this. It means having to frequently “sell” the course to them to get them to be interested in what we do. It is frustrating to facilitate an experiential class with so many students who are simply going through the motions to achieve a grade. Question 9: What is/was your overall assessment of the class’ effectiveness in meeting its goals of developing behavioral skills? I1: Difficult to characterize. On the positive side, we have empirically measured improvement in assessment centers that appears to be attributable to pedagogy (Hoover, Giambatista, Sorenson, & Bommer, 2006 and 2010). Alternatively, structural difficulties surrounding student demography and attitudes perpetually hamstrung the class. Nevertheless, I believe we need to keep up the good fight and hope that we are planting seeds that will pay off for our students down the road, even if they are not as receptive to the class as we would like them to be. We are all concerned about long-term retention and internalization of skills, and also about overcoming the significant chunk of students who never truly “buy in.” I2: From reading the content of hundreds of Individual Development Plans, I conclude that the effectiveness of our results in the Executive Skills course probably falls into a bimodal distribution. I estimate that the majority, maybe up to 60 to 65%, meet criteria of accomplishing the development of a meaningful level of behavioral skills. A third of that group--- about 15% of the students in total--- accomplish something resembling a positive transformative experience. Ten percent of the students seem to be relatively unaffected. But, some distance removed in perceived outcomes, we also have about 25% of our students who seem to just be going through the motions. I suspect that a good portion of this phenomenon is a function of student characteristics that are generational. I think that many members of the current generation, raised on the internet, have neither the mindset nor the patience needed for the exploration of behavioral skills or for a program of individual growth and personal effectiveness. They are programmed for instant feedback and results without significant effort (maybe the old American work ethic belongs to prior generations). When they do not get instant results--- rather than looking to their personal shortcomings or their lack of effort as the culprits---, they reject the message as maybe the messenger as well. I3: I believe the course does an excellent job of giving students enhanced self-understanding, including their personality types, leadership and followership predispositions and capabilities, and general work style preferences. I also think the course does an excellent job developing many behavioral skills. For example, the modules on group facilitation and presentation skills have been effective; each of these modules provides specific feedback to the student (and in the case of presentations, a grade) which solidifies learning. The course would be even more effective if we could do this with some of the other skills taught, such as interpersonal skills. Question 10: Finally, are there any logistical or resource management issues (common to all courses), that were confounded, highlighted or particularly challenged by the Exec Skills teaching experience? Examples could include grading issues, lecture/labs, using TAs, etc. I1: If we had more doctoral-trained and pedagogy-trained faculty, I would like to see more behavioral observation and assessment of students in groups. Using groups can be powerful because you can provide a naturalistic task and dialog which yields rich observation and assessment potential. Broad tasks can allow for simultaneous assessment of many class components. Also, some economies of scale in recording the groups can be achieved, and faculty members can assess offline and provide individual feedback and view recordings with feedback for the next class session. I was quite successful with this method for the module on decision- making, but it was extremely time-consuming. If a single faculty member were asked to do this for every module, that scholar would have to put his or her research agenda on hold for the semester. As a tenure-track junior faculty member, I simply could not do this. I also believe breakout rooms are essential for some exercises; a change of environment seems to stimulate students and take them out of their adversarial mindset. I2: Students seem to only read materials that they know they are going to be tested on. This leads to a “testing trap.” If you do not test, they will not read it. If you do test, the class becomes all about the cognitive exam (an arena students are well practiced in) and not whole person learning. This leads to a “lecture or don’t lecture trap.” If you know the students are not reading it, then the cognitive material and conceptual frameworks have to be covered in a lecture. But lecturing tends to damage the experiential format by overemphasizing the cognitive element. However, sans a lecture, students may go into experiential exercises with insufficient sensemaking tools due to inadequate cognitive frameworks. I3: A major issue is the need for small lab classes that encourage more interaction and discussion, and allow for more individualized feedback. A good size is about twenty students. Page 37 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 Boyatzis, R.E., Stubbs, E.C., & Taylor, S.N. (2002). Learning cognitive and emotional intelligence competencies through graduate management education. Academy of Management Learning and Education, 1 (2): 150- 162. Forrest, S.P. & Peterson, T.O. (2006). It’s called andragogy. Academy of Management Learning and Education, 5 (1); 113-122 Fukami, C.V. (2002). 9/11 Montage: Professors remember. Academy of Management Learning and Education, 1 (1), 14-37. Giambatista, R.C. and Hoover, J.D. (2007). An evaluation of ‘intensity’ in skill development using experiential learning methodologies, National Academy of Management Meeting, Philadelphia, PA. Goleman, D. (1996). Emotional intelligence. London: Bloomsbury Greenberg, D.N., Clair, J.A. and MacLean, T.L. (2007). Enacting the role of management professor: Lessons from Athena, Prometheus and Asclepius. Academy of Management Learning and Education, 6 (4), 439-457 Hoover, J. D., (1974). “Experiential learning: Conceptualization and definition” in James Kenderine and Bernard Keys (Eds.). Simulations, Games, and Experiential Learning: On the road to a new frontier. ABSEL Proceedings, Vol 1. Hoover, J. D. (2007) “How “whole” is whole person learning? An examination of spirituality in experiential learning,” Developments in Simulation and Experiential Learning, Vol. 34, 324-330, Reprinted in the Bernie Keys Library [Available from http:// ABSEL.org] Hoover, J. D., & Giambatista, R. C. (2009). Why have we neglected vicarious experiential learning? Developments in Business Simulation and Experiential Learning, 36, 33-37. Hoover, J.D., Giambatista, R.G., Sorenson, R.L. & Bommer, W.G. (2010). Assessing the effectiveness of whole person learning pedagogy in skill acquisition. Academy of Management Learning and Education, 9: 192-203. Hoover, J.D., Giambatista, Robert C. & Belkin, Luiba (2012) Eyes On, Hands On: Vicarious Observational Learning as an Enhancement to Direct Experience. Academy of Management Learning and Education, 12 (4). Hoover, J.D. (2005). A prescription for experiential executive skill development in MBA programs. National Academy of Management Meeting, Honolulu, Hawaii. Hoover, J.D., Giambatista, R.C., Bommer, William, and Sorenson, R.L. (2006). Effectiveness of experiential/ behavioral skill pedagogy in skill acquisition via assessment centers, National Academy of Management Meeting, Atlanta, Georgia, 2006 Hunt, J. G., & Sorenson, R. L. (2001). A learned behavior approach to skill development. Journal of Management Education, 25 (2): 167-190. Kalliath, T. and Coghlan, D. (2001). Developing reflective skills through journal writing in an OD course. Organization Development Journal, 19 (4), 61-70. Klimoski, R.J. (2007). From the editor: Becoming a prophet in our own land. Academy of Management Learning and Education, 6 (4), 433-438. Kolb, D. A. (1984). Experiential learning: Experience as the source of learning and development. Englewood Cliffs, NJ: Prentice-Hall Lewin, K. (1951). Field theory in social science, selected theoretical papers, in D. Cartwright (Ed.) New York: Harper and Row Rode, J.C., Arthaud-Day, M.L., Mooney, C.H., Near, J.P., Baldwin, T.T., Bommer, W.H., and Rubin, R.S. (2005). Life satisfaction and student performance. Academy of Management Learning and Education, 4 (4): 421-433. Reynolds, D. (2007). Restraining Golem and harnessing Pygmalion in the classroom: A laboratory study of managerial expectations and task design. Academy of Management Learning and Education, 6 (4), 475- 483. Rogers, C.R. (1980). A way of being. Houghton-Mifflin REFERENCES Another issue is the classroom itself. It should allow students to move into different configurations for small group work, dyads, etc. A flat room (not tiered) with moveable chairs and enough desk space for students to work on exercises using large flipchart paper is ideal. CONCLUSION As scholars and educators, we tend to be quite critical regarding our reading of research and of our own efforts in the classroom. Thus, the interviews above seemed to strike mostly critical, if not downright pessimistic tones. All three interviewees, however, were passionate about the course and remained committed to the course’s basic premise of whole person learning (WPL) through experiential- and behaviorally-based skill development. If anything, our commitment to the course might have clouded our judgment regarding the implications of working in a less -than-munificent environment. Realities about administration, student body, resource constraints, and similar factors in retrospect might have been better considered in tweaking the course design while remaining true to the WPL concept. Educators wishing to dramatically transform management education into a skills-based approach need to consider such environmental factors into pedagogical and androgogical design. Yet, it would be easy to compromise away the essence of a behaviorally-based approach, and for this reason we recommend that ABSEL educators with high aspirations “dive in”, but with open eyes and a willingness to subsequently modify and adapt to student needs and motivations as well as administrative pressures. http://ABSEL.org http://ABSEL.org