AN EXPERIMENT IN ASSESSING THE THEORETICAL RECOGNITION AND APPLICATION LEARNING SKILLS OF STUDENT CASE WRITERS Exploring Experiential Learning: Simulations and Experiential Exercises, Volume 5, 1978 146 AN EXPERIMENT IN ASSESSING THE THEORETICAL RECOGNITION AND APPLICATION LEARNING SKILLS OF STUDENT CASE WRITERS James E. Weir LEARNING AS A MODEL BUILDING PROCESS For eight weeks of a sixteen week semester, students had been analyzing short prepared cases dealing with managerial incidents. They had weekly opportunities to sharpen their cognitive assimilation skills in studying and discussing various textbook concepts and propositions dealing with human behavior, interpersonal behavior, and group behavior. It appeared the weekly case analysis activities afforded opportunities to transfer these theoretical constructs to textbook type organizational and managerial situations. In each case they attempted to answer a critical question: What are the characteristics of effective managerial behavior depicted in the case situation? The experiment described below, in part, was meant to demonstrate to them that they already possessed assumptions, and even complex implicit models, of how effective managers should behave in various circumstances. Learning Skills Sought This experiment was an attempt to demonstrate the operation of theory-practice transference processes and meaning abstraction processes which occurs in learning situations. The experiment sought to demonstrate to MBA students that they were capable of applying the concepts, theories, and working propositions concerning effective managerial behavior to their own organizational experiences. The second major purpose of the experiment was to assess the degree to which students could make situationally-based value judgments. While value judgments are usually considered to be neither right or wrong as they apply to some aspects of personal behavior, this cannot be held true from a managerial point of view. Managers must learn to make judgments of value concerning the worth of a worker’s task behavior, as well as the worth of programs, products, and services. In a similar manner, students were encouraged to develop the skill of abstracting meaning from their work situations and to review the value judgments of managers in situations they were familiar with. Instead of making a task decision, as a manager might, they were required to make an assertion dealing with one of the two major theme categories, effective or ineffective managerial behavior. As will be discussed, this involves cognitive transitioning through stages of analysis under the assumption that some traces of those cognitive transitions would be present in their written case analyses. Four learning skills became the target for assessment in this experiment. The Cognitive Assimilation Skill is the ability to mentally grasp and understand a particular theory or set of Exploring Experiential Learning: Simulations and Experiential Exercises, Volume 5, 1978 147 concepts comprising a conceptual framework [3, p. 73]. The Transformation Skill is the ability to relate specific organizational or managerial behaviors, attitudes, or expectations with those presented in particular theories [3, p. 73]. Third, Cause and Effect Inferential Skill is the ability to attach meaning to disjointed facts presented in an organizational or managerial situation and to perceive a principle of uniformity operating therein. Fourth, Situational Value Judgment Skill is the ability to make reasonable decisions concerning the short-run or long-run worth of an employee or manager’s behavior in accomplishing the organization’s goals. This skill requires that abstractions be made from facts and cause- effect relations at the level of the analyst’s value system. In short, students were taught to learn to reason inductively to obtain answers to the question: What constitutes effective managerial behavior in specific circumstances? Abstracting Meaning From the Work Environment Figure 1 hypothetically depicts the levels of knowledge which mark the cognitive processes of a manager constantly abstracting meaning from the work environment. These four levels roughly correspond to the four levels of human abstraction specified by language expert Hayakawa [6] and further clarified by Reynolds [8, p. 49]. The model in Figure 1 assumes that the status passage of a decision issue through a manager’s thought processes typically may be marked by four increasing levels of knowledge. At the first level he recognizes symptoms of a possible non-standard condition in his system. At the second level he has collected and analyzed facts about the symptoms in an attempt to discover the existence of meaningful patterns. At the third level the manager has penetrated uncertainty enough to establish a new level of knowledge. He has verified the existence of a reasonable cause and effect set of relationships which appear to explain the presence of the symptoms. The fourth stage contains processes in which an informal criteria function is compared to the causal model for the purpose of discovering alternatives to satisfy the decision issue now stated in a problematic form. At this stage the decision issue is translated into the form of a solvable problem simultaneously with the discovery of a satisfactory alternative and one or more reference alternatives. Assuming that this scenario has some degree of validity in depicting the way effective managers cognitively process a symptom condition, the model contains two major implications for educating managers. First, the management teacher should verify that students can proceed through these four stages in a relatively efficient manner. Second, the model demonstrates how concepts and theory propositions are used as tools to facilitate the transition from symptoms to decision issue to solvable problem. According to this model theory becomes a necessary tool for completing each phase of the transformation. It should be possible to assess the theory content used in these transitions Exploring Experiential Learning: Simulations and Experiential Exercises, Volume 5, 1978 148 by means of the residue present in the stage four decision framework. METHODOLOGY Twenty-three evening MBA students in a graduate management and organization theory course were asked to write a take-home case describing a managerial experience of which they had firsthand experience. They were required to describe an incident or situation which illustrated the behavioral difference between effective and ineffective managers. This assignment was designed to have them relive their past organization experiences for the purpose of viewing it from new perspectives, aided by the course concepts and theories. The students were asked to analyze this familiar data base and to apply the appropriate theoretical concepts and principles previously covered. They were to state summary working propositions which distinguished how effective Exploring Experiential Learning: Simulations and Experiential Exercises, Volume 5, 1978 149 or ineffective managers did behave in the circumstances depicted in their case analysis. The cases were not to include the names of real persons, work groups, or companies. They were told that they would be graded on their use of concepts, theoretical frameworks, as well as their ability to identify working propositions which describe how effective or ineffective managers behave in the situation described. A method called thematic content analysis was used to analyze and score each case. This method parallels the method of “qualitative content analysis” suggested by Berelson [1, pp. 114- 134] and especially described as “thematic analysis” by Budd, Thorp, Donohew [2, pp. 47-49]. It is based upon the presence or absence of themes which may be subsumed under two theme categories: 1) characteristics of effective managerial behavior, and 2) characteristics of ineffective managerial behavior. These content categories are described as theme categories because they act as a larger “compartment” within which themes may be organized. A theme is defined as an assertion about a subject matter (Berelson, 1952, 18). The subject is the referent (e.g. the effective manager) and must be present or implied in the assertions. Figure 2 identifies the structural components which were experimentally employed in the analysis of themes from student cases. The analysis consisted of attempts to score concepts, theoretical propositions, and propositions concerning effective or ineffective managerial behavior. A set of scoring rules were also developed to avoid double counting and improve the meaningfulness of the resulting scores. FIGURE 2 THE STRUCTURE OF THEMATIC CONTENT ANALYSIS Thematic content analysis consists of the following skill elements: 1. Theoretical Recognition and Application Skill. A skill which defines the student’s overall cognitive abilities to apply organization theory and management principles to themes from real life situations. 2. Concept Application Skill. A skill which defines the student’s cognitive abilities to properly transfer a learned theoretical concept to themes derived from real life situations. 2.1 Concept Application Scores (CAS). A score used to assess the extent and quality of concept recognition and application learning skills consisting of: a) Stated Concept (SC), b) Implied Concept (IC), c) Direct Application of a Concept (DAC), and d) Misapplied Concept (MC). 3. Proposition Application Skill. A skill which defines the student’s overall cognitive abilities to properly transfer a learned theoretical relationship between two or more concepts to themes derived from real life situations consisting of: Exploring Experiential Learning: Simulations and Experiential Exercises, Volume 5, 1978 150 3.1 Proposition Application Score (PAS). This score is based upon the student’s correct use or misuse of identified working propositions based upon themes from real life situations as follows: a) Stated Proposition (SP), b) Implied Proposition (IP), c) Direct Application of a Proposition (DAP), and d) Misapplied Proposition (NP). RESULTS The merit scores derived from thematic content analysis of the twenty-three cases are presented in Table 1, ranked according to Theory Recognition and Application Score (TRAS)~ These scores range from 155 to 40 points. The median score is 74 points. These scores represent ordinal or rank data. Therefore it is admissible to perform nonparametic statistical tests only. According to nonparametic testing, the addition of the Concept Application Score and the Proposition Application Score to arrive at the TRAS score is not mathematically valid. However, it is convenient to have one number represent the ranking of student cases by a rough measure of relative degree of theoretical content. It is arbitrary to designate the cut-off scores for high, medium, and low achievement of theory recognition and application skills. However, using quartiles the lower quartile score (Ql) was 57, while the upper quartile score (Q3) was 94, excluding the 155 extreme score. It appears that the sample scores were distributed uniformly from a low of 40 to a high of 110. One very gifted student was able to achieve a very high score of 155. Clearly, his degree of achievement does not appear to fit well into this sample. DISCUSSION The two ranking scores were useful in distinguishing those students who appeared to possess different kinds of meaning abstraction skills. Some students apparently had not internalized the concepts and theory propositions presented in the text and in class discussions. Their CAS score was low, ranging from 10 to 50 points, compared to those who had mastered the theory and had scores ranging from 50 to 75 points. The PAS score helped to differentiate those students who could abstract meaningful propositions from their work experiences from those who could not. The latter students were usually able to describe phenomena in general terms but sometimes these descriptions demonstrated failure to penetrate deeply into what was happening in the work situations. Those students who fail to describe their work situations well also appeared to be unable to apply the concepts usefully. They failed in either of two ways. Either they failed to go into enough detail to apply the concepts well (low level abstraction failure), or they failed Exploring Experiential Learning: Simulations and Experiential Exercises, Volume 5, 1978 152 Exploring Experiential Learning: Simulations and Experiential Exercises, Volume 5, 1978 153 to abstract propositions of effective managerial behavior even though they seemed to be able to apply concepts (high level abstraction failure). It appears that to be able to do an excellent job on both conceptual and propositional skills one must transition through multiple levels of abstraction without premature jumping levels or without excessive dwelling on lower levels. The results of the experiment demonstrated the following tentative conclusions. When students are required to analyze situations existing in their own work environment, they must come to grips with reality in a much more detailed and comprehensive manner than they usually do. In a sense, by defining aspects of their work situations they come to understand their own motives, expectancies, goals, and fears more comprehensively. Second, they learn more about the manager’s job they seek to depict descriptively in their cases because they learn to put themselves into this actor’s shoes--an experience they may never have attempted before. Third, they come to learn that concepts and working propositions derived from the theories of human behavior are necessary in order to successfully carry out the reality abstracting processes. Fourth, abstraction reaches its highest point when the student learns to apply a series of value judgments explicitly to the motives and behaviors of actors, especially the managerial actors. This level includes judgments that the manager should not have acted in a particular manner, or that he failed to act to meet a critical issue in the decision situation. Finally, the comparative method of case analysis allows the student to learn by either the positive comparative method which deals with an instance of effective managerial behavior, or by the negative comparative method which analyzes how ineffective managers behave in specific circumstances [9]. The student has implicitly learned how to build a model of events within a social system without even realizing that he was engaged in building empirically grounded theory [5]. CONCLUSIONS This model-building learning process is important for practicing managers from four points of view. First, it fits the naturally inductive and experimental mode of managerial thinking and acting. It allows room for informed intuition and the application of various heuristic decision rules which managers are known to employ in complex decision situations. Second, there is no intellectual forcing of an elaborate prepared (and possibly elective) model of effective managerial behavior upon the student. He is required to make a little more explicit what his experience has already taught him about how managers behave and how things operate in work environments, but which has remained until now, tacit knowledge [7]. Third, this learning process is useful in teaching managers the value of theory in helping them clarify their own actions and in understanding the behavior of subordinates, peers, and superiors. Exploring Experiential Learning: Simulations and Experiential Exercises, Volume 5, 1978 154 It aids them by providing a rich language of concepts and systems of propositions which may be applied as tools in abstracting meaning from reality. Fourth, it aids the teacher of management in pinpointing deficiencies in the student’s ability to conduct reality abstraction processes at increasingly different levels. REFERENCES 1. Berelson, Bernard, Content Analysis in Communication Research (Glencoe, Ill.: The Free Press, 1952). 2. Budd, Richard W., Robert K. Thorp, Lewis Donohew, Content Analysis of Communications (New York: Macmillan Company, 1967). 3. Certo, Samuel C. and Kenneth E. Newgren, “Interpersonal Skill Development: The Experiential Training Unit (ETU) and the Transfer of Training,” Proceedings of the National Conference of Association For Business Simulation and Experiential Learning, Wichita, Kansas, 1977, 72-81. 4. Drucker, Peter F., The Effective Executive (New York: Harper and Row, 1966). 5. Glaser, Barney and Anselm L. Strauss, The Discovery of Grounded Theory: Strategies for Qualitative Research (Chicago: Aldine Publishing Company, 1967). 6. Hayakawa, S. I., Language in Action, (New York: Harcourt Brace and Company, Inc., 1941). 7. Polanyi, Michael, The Tacit Dimension (Garden City, N.Y.: Anchor Books, 1966). 8. Reynolds, John I., “Cases Which Meet the Student’s Needs,” Academy of Management Proceedings 1976, 48-52. 9. Smelser, Neil J., “The Methodology of Comparative Analysis” in Donald P. Warwick and Samuel Osherson (eds. Comparative Research Methods (Englewood Cliffs, N.J.: Prentice-Hall, 1973). Table of Contents Volume 5, 1978 Debriefing: The Key to Effective Experiential Learning A Kiss Before Debriefing Debriefing with Serialized Theory Development for TaskTeam Learning Insights Into Debriefing Experiential Learning Exercises The Two-Step Flow of Experiential Learning: A Preliminary Investigation Programmatic Experienced-Based Learning in an MBA Program The New Research Focus: An Analysis of the Simulation Game User An Analysis of Performance in Simulation Games Compared to Performance on Structured Course Criteria: A Case Study Le's Simplify the Administrative Requirements of Computerized Educational Simulations Assessment of Sex Stereotypes Within Task Group Simulation The Use of Program BAYAUD in the Teaching of Audit Sampling The Effectiveness of Progressively Complex Accounting Simulations in Increasing Decision-Making Performance The Natural Learning Project The Application of a US-USSR Trade Simulation to the Reaching of Business Russian The Use of Semi-Autonomous Work Group in Graduate Management Education: An Australian Experience Evaluation of an Experimental Course in Organizational Behavior for Managers of Japanese Banks Berkshire II: An Experiential Decision Making Exercise: An Exercise in Hegelian Inquiry Creating a Responsible Managerial Experience A Framework for Determining the Pedagogical Value of Simulation Gaming: Implications for Future Simulation Gamin Research Emergent Simulations in Administration Courses Measurement of Administrator Role for Feedback on Structure and Goals An Experiment in Assessing the Theoretical Recognition and Application Leaning Skills of Student Case Writers Some Game Information Systems Experience Using Job Control Language (JCL) for Group and Individual Simulation Considerations for the Use of Computerized Business Simulations Assessing the Effectiveness of Learning Styles as Predictors of Performance within Three Distinct Pedogogic Methodologies Learning Style and Performance A Psychometric Analysis of Kolb's Learning Styles Inventory Learning to Identify and Satisfy consumer Wants: A Classroom Game An Experiential Exercise in Product Benefit Segmentation Sales and Sales Management: A Case/Simulation Approach The SBI Case as Experiential Learning The Use of a Live Case in Teaching Organization Theory and management Principles to Graduate Students The Residency in Hospital and health Administration as Experiential Learning The Extended Live Case Approach to Teaching Marketing Research Experiential Learning in Marketing: Student Consultants The Brazil-Columbia Coffee Marketing Negotiations: An Experiential Exercise Aimed at Highlighting the Dynamics Involved in the International Business Negotiation Process Educational Values of Simulation Gaming Business Simulation Re-Revisited Simulation and Experiential Learning as Human Subject Research An Experiential Evaluation of a Didactic - Experiential Apporach for Teaching personnel Management Experiential Learning in Wage and Salary Administration Integrating Structured Experiences into Personnel Simulation Locus of Control and Performance in a Management Simulation Correlations Between Academic Achievement, Aptitude, and Business Game Performance Suggestions for Student Input of Decisions in General Management Simulations An Exploratory Investigation of Student Perceptions of Computer Simulation as an Educational Tool Time to Sell?: An Experiential Learning Approach to Stock Market Decisions through Interactive Gaming