Transformational Thinking in the Organizational Behavior Course: The Use of Metaphor as an Assessment Tool Developments in Business Simulation and Experiential Learning, Volume 26, 1999 TRANSFORMATIONAL THINKING IN THE ORGANIZATIONAL BEHAVIOR COURSE: THE USE OF METAPHOR AS AN ASSESSMENT TOOL David Szczerbacki, Alfred University Martha Spears, Winthrop University Dolun Öksoy, Alfred University ABSTRACT The paper discusses an effort to use metaphori- cal imagery in completing an outcome assess- ment exercise for the basic course in organiza- tional behavior. This imagery is drawn from a popular video shown at the beginning and end of the course. The outcomes assessment exer- cise focuses on value added measures of trans- formational thinking relative to student ability to conceptualize organizational phenomena. The paper reports partial results of this pilot study. INTRODUCTION Contemporary interest with curricular outcomes assessment has called into question time hon- ored techniques used by Instructors and their respective departments to generate student evaluations of individual courses. Concern largely focuses on the extent to which courses “add value” to the learning process. Interest has applied equally to curricula in general education and to professional fields such as business and engineering. Implicit in the various efforts designed to en- hance curriculum-based outcomes assessment is the desire for transformational learning: funda- mental changes in skill, behavior, or thinking. This paper reports on a pilot effort to use meta- phor as a basis for generating self-reported evaluations of learning in the undergraduate Or- ganizational Behavior (OB) course typically found in most business administration pro- grams. Specific interest here is in testing and developing a technique for assessing transfor- mational thinking where the intent of a course is “to change the capacity to understand and com- prehend rather than just impart information and facts to be comprehended.” Morgan (1988). Establishing value-added measures and proto- cols to assess transformational thinking about organizations represents special challenges. Clearly, development of higher order critical thinking skills is appropriate for the OB course. For example, leadership development, typically suggested as one of the core objectives of the OB course, assumes an emphasis on the acquisi- tion of skills that go beyond the technical and interpersonal level to include the level of ab- stract conceptualization about organizational systems. This being said, it is the experience of the au- thors that students in the OB course often lack frames of reference conducive to organizational conceptualization and abstraction. Of course, there is no lack of curricular material available to instructors as they seek to create appropriate frames of reference. Witness the robust reper- toire of experiential materials available to OB instructors. However, there appears to be little evidence indicating use of such materials for the establishment of base-line measures supporting self-reported, value-added assessment. The research described below is motivated by the perceived need for inquiry and experimenta- tion with assessment techniques linking student ability to conceptualize organizational phenom- ena at the beginning of the OB course to the student’s conceptualization skill at the end of the course. Such techniques need to pass two 244 Developments in Business Simulation and Experiential Learning, Volume 26, 1999 initial tests. First, are they conducive to creating an initial conceptualization opportunity for the student? Second, do the techniques allow re- flective, self-recorded, value-added assessment indicative of transformational thinking? LITERATURE ON METAPHORS Metaphors are often viewed as just a device for embellishing language discourse. Their signifi- cance, however, is far greater. The use of a metaphor implies “a way of thinking and a way of seeing that pervades how we understand our world” Morgan (1997). Metaphors are direct comparisons that form a conceptual frame for ideas. When problems cannot be solved or events cannot be explained, metaphors are used to bring meaning out of chaos and dispel the mystery. Metaphors make confusion compre- hensible Ortony (1974). By including metaphor within the frame of knowledge process that de- scribes a form of epistemology, the metaphor constructs a bridge from the mind to culture. The metaphor thus changes the culture in which we live and thereby affects the ways in which humans interact with their environment. The use of metaphors to understand organiza- tional processes is well-documented. Bolman and Deal (1991). However, there is little evi- dence that students use metaphorical language in the study and mastery of organizational proc- esses. Knowledge based on metaphors is both descriptive and generative and Dunford and Palmer (1996). By using metaphors, the situa- tion is named, framed, and the problem set Schon (1993). The use of a familiar structure provides “for guiding inferences, sanctioning actions and setting goals.” Boland and Green- berg (1988). If metaphor is a basis for action, it is reasonable that it can act as a basis for learn- ing and the evaluation of learning. METHODOLOGY To provide an appropriate metaphorical image, the popular movie “Babe” was shown to two OB classes. The movie was viewed as an appro- priate mix of fantasy, allegory, and playfulness to invite and engage creative thinking. The plot of the movie basically revolves a story line in which a pig is raised amidst rich organizational dynamics - complete with barnyard politics, in- ter-species conflicts, and the intricacies of agrarian production systems. Through a combi- nation of acculturation, swine spunk, and neces- sity, Babe is entered into a prestigious sheep- dog competition. As an allegorical tale, Babe is in the tradition of Orwell’s Animal Farm and Chaplin’s Modern Times. To test the acceptance of the video in a class- room setting, the video was first shown to a graduate course in Total Quality Management consisting of MBA students. The video was shown late in the course and the assignment given to the students was to write a one-page “reaction paper” responding to the movie based upon total quality management techniques and concepts which had been covered in the course up to that point. While these graduate students were quite skeptical with the assignment, the papers submitted suggested an encouraging re- sponse in terms of the movie’s capacity to en- courage conceptualization - and to generate lively discussion. Based on this initial class test- ing of the video, the authors proceeded to de- sign and implement an initial assessment in- strument and process in the OB course. The video was then shown to two undergraduate OB classes. The students were predominately third year business administration or accounting students. This was the first course in organiza- tional behavior for these students. 245 Developments in Business Simulation and Experiential Learning, Volume 26, 1999 Students from each class were shown the video during the regular class period in the second week of class meetings. Prior to viewing the video, students were provided with a brief lec- ture on using metaphor as a learning tool. Stu- dents were told that the purpose of the exercise was to develop sensitivity to basic organiza- tional behavior concepts through the use of metaphor. Students were informed that the placement of the exercise at the beginning of the course was intended to enhance receptivity rela- tive to the subsequent demands of concept learning. Finally, prior to viewing the movie students were given this assignment: Thinking about what organizational behavior is, write a one-page reaction paper. The paper should con- sist of three paragraphs: What was the most important thing you learned about (1) Individual Behavior, (2) Group Behavior, (3) Organiza- tional Behavior. The use of specific examples from the video (i.e. think metaphorically) is preferred. After viewing the movie, students submitted the reaction paper in the following class meeting. Students were asked to sign their name to the paper and the papers were returned with mini- mal comment and without grade. (Students had not been advised that the papers would not be graded. Expectation of receiving a grade likely increased motivation for completing the as- signment.) In addition to providing an initial course orientation, the purpose of generating these reaction papers was to create a vehicle for completing the self-assessment, value-added exercise at the conclusion of the course. Stu- dents were not informed that they would be completing the second phase of the project at this time. Copies of the original submissions were retained for distribution at the end of the course. No explicit or structured reference to the video was made over the remaining duration of the course. In a very few instances for each class, students made reference to the video as part of general class discussion on a particular topic. During the last week of each class and prior to regularly scheduled student evaluations, the second part of the value-added exercise was im- plemented. Each class viewed the film again. Students were asked to repeat the written exer- cise provided early in the semester. In an effort to minimize the potential for resubmitting or rehashing the first assignment, the students were reminded that the instructors had copies of the first paper. On the day the second reaction paper was submitted, the students were asked to com- pare their first reaction paper (provided to them in class) with the second reaction paper. To fa- cilitate this comparison a questionnaire was dis- tributed and collected in this same class period. Students were advised that the information from the questionnaire would not be used in deter- mining class grade. Furthermore, the students were advised that the instructors would not re- view the questionnaires until final grades were determined and submitted to the University Registrar. This counsel was provided in order to encourage frank and critical responses to the questionnaire. The questionnaire consisted of three sections as described below. The first section of the questionnaire has three sets of questions each responding to the various levels of organizational analysis: individual, group and organization. For example, for the individual level of analysis, students were asked to respond to the following: 1. In the context of individual behavior, please rate the extent to which your paper represents the improvement in the quality of your insight relative to the insight demonstrated in paper number 1 (response ranging from no improve- ment to high improvement; 5 point scale). 2. Please rate to which extent this course has contributed directly to the level of improvement 246 Developments in Business Simulation and Experiential Learning, Volume 26, 1999 identified above (response ranging from has not contributed to contributed greatly; 5 point scale). 3. Think about “Babe.” What one specific con- cept did you learn in this course which most di- rectly strengthened your ability to understand individual behavior in the video? This pattern of questioning was repeated for the group and organizational levels of analysis. A second question sought to assess the extent to which the initial viewing of Babe created an ac- tive metaphorical lens which students used as a learning tool as the course progressed. Specifi- cally students were asked the following ques- tion: As the course progressed over the semes- ter, approximately how often did you find your- self reflecting back to ‘Babe’ and thinking about a point made in class or your textbook? (Re- sponse ranging from never to frequently; 5 point scale). A third set of questions asked students to as- sume a value-added perspective based upon work submitted in the reaction papers. From this perspective, students were asked to self- assess learning for these nine organizational be- havior concepts: attitudes/values, motivation, decision making, groups, leadership, work- design, culture, communication, and con- flict/change. The specific question as it ap- peared on the questionnaire read thusly: Refer again to your ‘Babe’ paper # 2. Think for a moment about your level of understanding of organization behavior at the beginning and now, the end of the semester. With this perspective in mind, please assess your learning (measured from where you were in August and where you are now) for each of the following concepts. (Response ranging from no understanding to high understanding; 5 point scale). FINDINGS The paper provides a partial reporting of ques- tionnaire findings. In the interest of space, qualitative data is not reported. Two questions are assessed: what is the relationship between self-reported improvement in “insight” and the “contribution” of the course in shaping this out- come; and, the exercises utility in creating a re- flective transformational learning opportunity. Questionnaire responses were assessed using the Tukey Mean-Difference Plot technique. Tukey (1977). A mean-difference plot can add substantially to our visual assessment of a shift between two distributions. As depicted in Ex- hibits 1 and 2, the differences, (1) ii yx − are graphed against the means . (2) 2/)( ii yx − The line of the quantile-quantile plot be- comes the zero line on the mean-difference plot. Shifts are assessed by judging deviations from the zero line. This data visualization technique tends to enhance our perception of effects since we can more readily judge deviations from a horizontal line than from a line with a non zero slope. yx = Q1. Did the course contribute to self-reported insight at the individual, group and organization levels? A1. The interest here is whether, in completing the exercise and in engaging in critical assess- ment, students connected self-reported im- provements to work completed in the course. Exhibit I depicts “Relative Contribution of OB Course to Self-reported Insight” (where Insight = INS; Contribution = CON). Mean difference data is plotted for three levels of reflection: In- dividual (IndIns-IndCon), Group (GroupIns- GroupCon) and Organization (OrgIns-OrgCon). 247 Developments in Business Simulation and Experiential Learning, Volume 26, 1999 Exhibit 1 – “Relative Contribution of OB Course to Self-Reported Insight.” Visual inspection of the data plots shown in Ex- hibit 1 indicate that, for all three levels of reflec- tion, there is a strong correlation between self- reported improvements in insight and the con- tributions of the course in shaping this insight. In short, as students reflect upon their written work relative to the “Babe” exercise, they were able to critically assess learning realized in the course. As a tool, the “Babe” exercise appears to offer promise in providing an assessment op- portunity. Q2. Did the exercise create a reflective, trans- formational learning opportunity? A2. Exhibit 2 provides a mean difference plot for two variables: frequency of reflection by students to the "Babe” video over the course of the semester; and, a self-assessment of learning for each of nine OB topics: culture, communica- tion, conflict, groups, leadership, work-design, values, motivation, and decision making. Exhibit 2 – “Self-Assessed Learning for Organ- izational Behavior Concepts” -1 0 1 2 3 4 ININS~INCON 1 2 3 4 5 -1 0 1 2 3 4 GRINS~GRCON -1 0 1 2 3 4 ORINS~ORCON mean di ffe re nc e Tukey Mean-Difference -2 -1 0 1 2 3 4 Values 1 2 3 4 5 Motive Decision 1 2 3 4 5 Groups Leader -2 -1 0 1 2 3 4 Work -2 -1 0 1 2 3 4 Culture Commun 1 2 3 4 5 Conflict mean di ffe re nc e Tukey Mean-Difference Interest in Exhibit 2 lies in the fact, as indicated via inspection of the data plots, that responses for each of the nine OB topics differ in an ex- ceedingly simple way from responses to the re- flect question. In short, students reported a gen- erally modest level of reflection on the exercise over the course of the semester, and, based upon a completion of the exercise at the end of the course, students reported a generally high level of learning. DISCUSSION The use of metaphor as an assessment tool is predicated on the assumption that students need an interpretive context to self-assess learning. Self-assessment, per se, is valued in that it is a form of self-disclosure, providing a reflective opportunity. Such critical reflection can drive transformational learning as defined in the pa- per’s introduction. The “findings” shown in Exhibit 1 suggest that students are both comfortable in reporting learn- ing utilizing metaphorical imagery as an as- sessment tool and, perhaps more importantly, the use of such imagery provides an opportunity to reflect upon the impact of the course. The “findings” shown in Exhibit 2 suggest that students respond well to the use of pre- and past 248 Developments in Business Simulation and Experiential Learning, Volume 26, 1999 course imagery as a vehicle for making summa- tive evaluations of learning. While students did not report a great deal of such reflection during the course, they responded to the opportunity for such reflection provided by the completion of the exercise at the end of the course. REFERENCES Bolman, L. E. and Deal, T. E. (1991). Refram- ing Organizations. San Francisco, CA: Jossey- Bass Publishers. Boland, R. J., Jr. and Greenberg, R. H. (1988) ‘Metaphorical Structuring of Organizational Ambiguity,’ in L. R. Pondy, R. J. Boland and H. Thomas (eds) Managing Ambiguity and Change. New York, NY: John Wiley & Sons Publishing. Cormac, M. (1985). A Cogntive Theory of Metaphor. Cambridge, MA: The MIT Press. Dunford, R. & Palmer, I. (1996) ‘Metaphors in Popular Management Discourse: The Case of Corporate Restructuring.’ In D. Grant and C. Oswick, (1966). Metaphor and Organizations. Newbury Park, CA: Sage Publishing. Morgan, G. (1988). ‘Teaching MBAs Trans- formational Thinking,’ in R. E. Quinn and K. S. Cameron, (1988). Paradox and Transformation. Cambridge, MA: Ballinger Publishing Com- pany. Morgan, G. (1997). Images of Organization. Newbury Park, CA: Sage Publishers. Ortony, A. (1979). Metephor and Thought. Cambridge, MA: Cambridge University Press. Schon, D. (1993) ‘Generative Metaphor: A Perspective on Problem Setting in Social Pol- icy’, in A. Ortony (1993). Metaphor and Thought. Cambridge, MA: Cambridge Uni- versity Press. Tukey, J.W. (1977). Exploratory Data Analy- sis. Reading, MA: Addison-Wesley. 249 Table of Contents Volume 26, 1999 ABSEL's Historical Research Interests Back From the Future: An ABSEL Merlin Exercise for the Year 2005 The Contributions of ABSEL During the 1980's ABSEL's Contributions to Experiential Exercises in the 90's ABSEL's Contributions to Experiential learning/Experiential Exercises: The Decade of the 1970's Business Simulations - Algorithms and Model Enhancements: A 25 year Review A Study of the ETS General Field Test as an AACSB Assessment Tool and the Impact of Experiential Exercises and Simulation on Learning A Framework for Assessing the Competencies Reflected in Simulation Performance Developing A Learning Culture: Assessing Changes in Student Performance and Perception Financial Simulation Using Distributed Computing Technology Analyzing Managers' Judgements and Decisions with an Educational Business Simulation Understanding Your Business through Home-Made Simulator Development An Examination of a Reanalysis of the Impact of a Market Leader on Simulation Competitors' Strategies Applying Shocks to TE Simulations: A Demonstration Increasing Efficiency of Management Skill Assessment A Testbank for Measuring Total Enterprise Simulation Learning Developing Leadership Skills - Video Live! LEADSIMM: Collaborative Leadership Development for the Knowledge Society A Team Approach to Producing Multi-Media Laptop and Video Formatted Presentation Tinkertoys Revisited: Exploring Trust Based Relationships Overall Dominance in Total Enterprise Simulation Performance Success or Bankruptcy: The Relationship between Personal and Goal Orientation and Simulation Performance Assessing the Effects of Feedback: Muti-method and Muti-directions in Multi-pedagogical Courses The Missing Ingredients in Experiential Learning Purpose and Learning Benefits of Business Simulations: A Design and Development Perspective Building Capabilities for Change through Laboratory Simulations Modeling Innovation as a Process of Design in Educational Business Simulation The Need to Measurer variance in Experiential Learning and a New Statistic to do so Assessing Effectiveness of an Experiential Oriented Course Over Time Developing Participant Satisfaction Models of Experiential Exercises in Business Education Perceptions of Learning in TE Simulations Students' View on the Use of Business Gaming in Hong Kong Management Gaming's Lost Opportunity: Meditations about the Russian Experience and Prospects for the Future Unanticipated Enhancements in the Business Strategy and Policy Game when Running in Windows 95 Using a Business Game to Demonstrate Broad Business Concepts A New Model for Business Courses (Getting the Student Connected) Seeing the Forest and the Trees: Integrating Knowledge Using Large Scale Simulations in Capstone Business Strategy Classes Is It Here To Stay? A Roundtable Discussion of the Inter-Group Interaction Interactive Tools Used in Applying Financial Concepts Strengthening Essential Skills through a Finance Exercise: Calculation of Beta A Model of Currency Exchange Rates Understanding Currency Exchange Rates: A three-part Exercise Student Experiences in the World Intercollegiate Business Game Competition Student Experiences in the International-Collegiate Business Policy Game Competition Using Boards of Directors in Simulation Environments: Comments From Board Members Sharing Best Practices: Teaching Smarter, Not Harder Star Power: A Simulation for Understanding Power and Empowerment The Marketing Game: A Marketing Principles Simulation Alexander Islands: GSSM Tiny Business Simulator on the WWW Putting Strategy into Strategic Business Games Industry Analysis, Porter's Five Forces Model and Strategic Group Maps in the Business Strategy Game Simulation The Use of Concept Mapping to Improve Student Performance and Understanding of Strategic Management Concepts: A Comparison of Techniques Transformational thinking in the Organizational Behavior Course: The Use of Metaphor as an Assessment Tool Creation of a Virtual Learning Community for the Global Virtual Enterprise Project Inter Institutional use of Educational Resources: Joint Use of Management Simulation Games Business Insights: Theory and Practice with the Aid of a Business Simulation Progress: An Experiential Exercise in Development Marketing The ABC's of teaching the Theory of Constraints to Undergraduate Business Students Demonstrating Principles of Organizational Purchasing Behavior through an Experiential Exercise The Virtual Manager: A Different Simulation for Managing Complexity When the Rules are changing and Chaos Breeds Innovation: Recapturing the Value of Constructive Thinking and Play in Simulation Training The Pitfalls, and Potential, of Actual Events as Problem Drivers Using Business Games to teach Environmental Awareness and Green Management: The International Experience with the ENSIM Game A Review of my ABSEL-Related Work Simulation of Government: A Workshop Using GEO Creating Internet-Based Games Using Perl and JavaScript Who's on First? Exploring the Concepts of Problem-Based Learning, Experiential Learning, and Lifelong Learning Students Learn Customer Service and Selling while Conducting Research So You Want to Run an NFL Football Team–An Honors Interdisciplinary Project Supervised Internship: The Employer's Perspective Using Computer Assisted Simulation to Teach International Business Strategy: A Case of the Multinational Management Game (MMG) Multi-Cultural Experiential Learning: A Computer Simulation in China An Appreciative Stance on Diversity as We Move into the 21st Century: A Timeline Exercise to Identify Key Experiences in Good Work Relationships between Black and White Peers A Day in the Life of an Interactive, Real Time, and Internet Delivered Course: A Demonstration Comparing Internet Search Engines: An Experiential Learning Exercise Total Enterprise Simulations and the Internet: Assessing Student Perceptions and Preferences Teaching Accounting Information Systems in a Practicum Format Providing Experiential Learning in Accounting through a Field Study Payroll Project A Spreadsheet Based Business Simulation Game Computer-Behavioral Simulations Training for Project Managers Simulation Scenarios - Rationale and Illustration