USING JOURNALS TO ENHANCE COMPUTER SIMULATION BASED LEARNING Developments in Business Simulation & Experiential Learning, Volume 27, 2000 USING JOURNALS TO ENHANCE COMPUTER SIMULATION BASED LEARNING Paul McDevitt, University of Illinois at Springfield ABSTRACT This paper presents the results of a three year experiment using student journals as a means for weekly debriefing in a simulation based graduate marketing class. The journals provided a rich, productive and individualistic debriefing methodology; they also produced two unanticipated benefits. First the journals provided relatively unambiguous feedback about what students were learning from the simulation; this was not always what was expected or anticipated. Second, the journals provided an excellent vehicle for clarifying and more effectively – if not efficiently – addressing learning outcomes. INTRODUCTION This paper reports the results of a three year experiment involving the use of student journals to help understand and direct student learning using a marketing simulation. The first section of the paper describes the learning environment and identifies the simulation that has been used for many years (albeit with several short lived bouts of brand switching behavior). The second section describes the nature, purposes and use of the journal and the information that it has provided about student learning. The third section describes how this information has been used to help frame specific learning outcomes. Finally, the last section relates how the journal is currently used to help direct student learning towards those outcomes. The Learning Environment The setting is a four credit hour introductory marketing management class at the University of Illinois at Springfield. The goals of this required class are to introduce students – most of whom have never had a prior marketing class – to the knowledge base of marketing and to the management of a marketing program. The student body are part-time, fully employed persons in their late twenties and early thirties. The two most heavily represented occupations are engineers and accountants, constituting approximately 25 percent of enrollment. Classes meet once per week from 6:00 to 9:30 p.m. The first half of each class is devoted to a short lecture or cases addressing the assigned material for the week from Kotler’s Marketing Management (1999). The second half is devoted to COMPETE by Faria, Nulsen, and Roussos (1994). Students often stay until 10:00 p.m. or later to complete the COMPETE decisions to their satisfaction. Each student serves as a product manager and is solely responsible for the performance of his or her item. Individuals are judged on the basis of two equally weighted performance measures: a short-term measure (cumulative profit contribution) and a long-term measure (brand strength). Using Journals to Understand Learning The use of journals to learn (the student) and to understand what is being learned (the instructor) is not widely practiced in business education, but it is not new. McDevitt (1997) proposed that, with clearly defined learning outcomes, journals could be used to determine what is being learned by computer simulation participants and (through content analysis) to measure learning. 57 Developments in Business Simulation & Experiential Learning, Volume 27, 2000 Briefly, the case for journaling rests on the premise that it constitutes the highest order in a three-stage hierarchy of experiential learning. The first stage, participation, generates learning through a process model proposed by, for example, Kolb (1976). The second stage, debriefing, includes a purposeful discussion of an experience, usually intended to find out what a participant learned and to test this against the instructor’s learning objectives (Lederman, 1992). Finally, writing is thought to produce an effect that exceeds both participation and oral debriefing. Writing commits the learner to actively and formally organizing and expressing the results of an experience (Petranek, 1994). The purpose of the journal discussed here is to enable students to organize and communicate their thoughts about why they did what they did as a product manager in the COMPETE simulation. Students are encouraged throughout the grading procedure to address the intended purpose of the journal in their entries. Journals are written after each of twelve rounds and submitted to the instructor. At the end of each semester, the complete journal of each student is reviewed and observations about his/her class experience are recorded. The contents of these journals were revealing, and several recurring themes stood out. First, students often were not learning what the instructor had hoped they were learning. Rather, students were often learning things that were substantially different than expected or intended. We believe this was to be expected. Whetten and Clarke (1996) observed that “because the inductive process is inherently less predictable and reliable, students whose learning is restricted to experiential exercises often arrive at invalid conclusions.” Those authors further contend that “in unguided experiential learning situations less experienced students are particularly prone in the biases that Tversky and Kahneman identified in decision makers who use “small sample sizes” to generate conclusion: in particular, they tend to be biased in judging the representativeness of their experience, leading to overconfidence in spurious conclusions.” (p. 155). We are also reminded of the memorable introduction in a paper by Beale (1993): “Are your students learning what you’re supposed to be teaching? Are you sure? How do you know?” (Beale, p. 18). The second impression was that the journal’s greatest contribution appeared not to be as a vehicle for recording learning or even to support measuring it. Rather, its most productive use seemed to be as a mechanism for debriefing and directing learning. Consider some specific lessons that students were learning, as evidenced in journal entries and confirmed by COMPETE decisions. Many students were becoming strong – though often narrowly focused – tacticians but poor strategists. For example, learning often tended to consist of observation, reflection and experimentation in the pricing and advertising arenas, to the exclusion of sales force and R & D considerations. In turn, when competitors were considered in students’ analysis and planning, it was often in terms of their pricing and advertising actions only. Secondly, non-sequiturs of the worst sort were rampant. For example, if sales volume didn’t suffer when prices were increased, demand was dismissed as inelastic over that range. Or if sales volume matched that of a competitor who had a higher quality index, quality was judged relatively unimportant. Discussions with these students often suggested that they were classic examples of Kolb’s Accommodator learning style (1976). These are persons who learn best through doing rather than observing and who rely on feeling rather than thinking when deciding. Analyzing and decision making in a numbers dominated, unstructured problem solving environment are 58 Developments in Business Simulation & Experiential Learning, Volume 27, 2000 difficult for such participants. Ceteris paribus is a puzzle. Finally, many of the conceptual traps against which marketing texts and educators wage war are chronically apparent in journal entries and the actions that they explain. To wit: 1. the share trap: gaining share means gaining profits 2. another share trap: “buying” share rather than building strong brands 3. the accountant’s trap: increasing earnings by reducing marketing expenses 4. the research trap: undervaluing information 5. the quality trap: customers will only buy the highest quality brand. Many of these same conceptually flawed behaviors have been observed and are discussed periodically by consultants in, for example, the McKinsey Quarterly. One wonders how many of them were learned participating in business school simulations. Learning Outcomes Reconsidered The experience with student journals underlined the importance to this instructor of (1) more clearly articulated learning outcomes and (2) course and simulation designs intended to achieve precisely those outcomes. Among the many benefits of learning outcomes (Banta, 1993, 1996; Loacker, 1988), several were immediately germane. Thoughtfully formulated and faithfully followed learning outcomes should offer: 1. A means for clarifying what should be learned. This instructor’s learning outcomes were too broad and overly ambitious, resulting in inadequate instructional support. 2. A refinement of expectations regarding the level and quality of student learning and demonstrated performance. A logical consequence of narrowing learning outcomes should be, ceteris paribus, better learning and demonstrated performance. 3. A means for actively engaging students in the learning process and reflection on the learning they have experienced. Clearly a vehicle was needed for guiding the reflection (and subsequent “mislearning”) that was evident in the content of the journals and manifested in decision making behavior. 4. An adaptation of pedagogy to the learning needs of the students. The most needy students were those whose learning styles seemed least compatible with the learning environment present by COMPETE. 5. Feedback on the effectiveness of instructional strategies for individual students. As learning outcomes are more clearly defined, game administration procedures should evidence progress or lack of progress towards mastery. The need was clear: a set of simulation learning outcomes that contributed to the broader curricular goals of the class and that might be used to evaluate student achievement. Possible learning outcomes for simulations are numerous. Anderson and Lawton (1997) identified eight classes of learning outcomes, including 76 possibilities. Upon reviewing the goals of the course in the context of the broader curriculum and the outcomes addressed by the non-simulation part of the course, the following modest learning outcomes were selected. 1. Develop strategic analysis, planning and implementation capabilities. 2. Understand the contribution of each marketing mix activity to marketing position strategy. 3. Understand the complementarities among marketing mix activities. 4. Develop a balance between a customer and a competitor orientation. 59 Developments in Business Simulation & Experiential Learning, Volume 27, 2000 5. Understand the case for building strong brands rather than cutting costs to enhance earnings. 6. Understand the difference between share gains and earnings gains. The most difficult of these is the first one. Computer simulations effectively create a learning environment within which higher order cognitive skills may be challenged and enhanced (Anderson and Lawton, 1997). However, thinking strategically is clearly not the usual student’s first inclination; thinking tactically is. As Whetten and Clarke’s (1996) “quintessential inductive learning activity” (p. 154), simulations also have potentially strong downside learning potential. Even disciplined tactical thinking can be seriously flawed on conceptual grounds. Using a Journal to Enhance Administrative Effectiveness Journal feedback convinced this instructor that what was needed to achieve the outcomes cited above were more frequent and more effective interventions throughout the simulation. In short, the administrative style being used was not capable of achieving the desired ends. Resolving these needs involved redesigning the content and administration of the journal and using it as a debriefing tool. Debriefing is an activity in which people who have had an experience are led through a purposive discussion of that experience (Lederman, 1991). Lederman proposed that the goals of debriefing are to facilitate understanding of what happened, to find out what the participant learned and to test the latter against the instructor’s learning objectives. This instructor had been debriefing primarily for these ends, at the conclusion of the simulation. In retrospect, we believe that this administrative style matched Keys’ (1989) Free Thinker: strong on content and experience but short on student feedback. A combined journaling/debriefing procedure was implemented to provide more timely and effective interventions. As before, students prepare and submit journals immediately following each decision round. In a more hand on approach, these are now read by the instructor and, as appropriate, written feedback pertinent to learning outcomes is provided. More importantly, journal entries signal the need for oral debriefings throughout the simulation. These are known as “walkarounds,” since they include a walk around the building to discuss COMPETE decisions relating to learning outcomes. The format for walkarounds has been heavily influenced by the concerns of Baker et al. (1997) for using conversation as a means for transforming experience into learning. While the instructor’s facilitating role is not the one envisioned by Baker, their contextual considerations are certainly relevant, we believe, for enhancing learning. These include: making an effort, creating a safe space, moderating the energy, confronting conflict in ways that are growth promoting, engaging with the head and the heart and valuing the reflective listening as highly as the active speaking. These guidelines are especially important to the debriefing function as described below, since it involves virtually continuous dialogue between the instructor and individual students throughout a semester. A Socratic approach to debriefing is used during a walkaround. This is a questioning technique that seeks to lead a student to discover an analytical or planning flaw apparent in a journal entry…perhaps a market share trap. A predetermined line of inquiry, usually based on a sequence of very general to increasingly more specific questions, is used. When executed successfully, discovery is as rewarding an experience for the instructor as for the student. It is one of those rare opportunities 60 Developments in Business Simulation & Experiential Learning, Volume 27, 2000 to actually participate in a “reorganization of reality” (Piaget, 1973). The Two Commandments govern written journal feedback and oral debriefings. First, students are never told what to do. The challenge is to use the questioning technique to trigger student discovery. Second, discovery is couched within the context of Kotler’s marketing management process model (1999). This reinforces the course goal which is about learning to manage marketing rather than how to win a simulation. Using this journaling/debriefing procedure, the administration for a typical twelve round COMPETE session is as follows: 1. Each student’s decisions are recorded and summarized after each round in a format that eventually becomes an individual data base file. 2. A student journal is required after each round and is returned prior to the next round with feedback addressing (typically) objectives 2 through 6 above. 3. Students calculate profit contributions for their products after each round so that profitability and market share may eventually be examined (objective 6). 4. Oral debriefings begin usually after the fourth or fifth round. These focus on all learning objectives as necessary, but primarily on objective 1. Typically four or five oral debriefings are held each round. 5. After each fiscal year, students are given a record of profitability by product class, i.e., for all TSTs, CVEs and SSLs, so that they understand their earnings standing. 6. At semester’s end, a final class debriefing occurs, including data documenting cumulative profitability and quality/cost measures by product. There is rarely any disagreement about (but often much discussion over) product rankings by profitability or brand strength. Our sense and our hope is that this administrative style fits Keys’ (1989) Manager of Learning: “well educated in content, familiar with many methodologies for creating experience, and perceptive at drawing out feedback information that clarifies student misunderstandings and validates content” (p. 8). This educator has reached two conclusions on the basis of the administrative experience cited above. Each of these holds for a given setting, including the environment, the learning outcomes and a choice of simulation. First, virtually any committed MBA student can experience the satisfaction of managing marketing strategically. This is often both a cognitive and an affective learning opportunity, which explains why it tends to be so lasting and memorable. Some students “get it” sooner than others and receive more positive feedback throughout a twelve round session. Perhaps these will be future line managers. Mission accomplished. Others will take longer to “get it” and may feel more frustration along the way. Perhaps these students will never experience line management responsibility, but they will understand it. Mission accomplished. The second conclusion is that the primary determinant of the richness of the student experience is the management style of the administrator and his or her level of involvement. REFERENCES Anderson, P. and Lawton, L. (1997). Demonstrating the Learning Effectiveness of Simulations: Where We Are and Where We Need To Go. Developments in Business 61 Developments in Business Simulation & Experiential Learning, Volume 27, 2000 Simulation and Experiential Learning, 24, 68-73. Baker, A. C., Jensen, P. J. & Kolb, D. A. (1997). In Conversation: Transforming Experience into Learning. Simulation and Gaming. 28(1), 6-12. Banta, T. W. (1993). Summary and Conclusions: Are We Making a Difference? In T. W. Banta and Associates (Eds.), Making a Difference: Outcomes of a Decade of Assessment in Higher Education. San Francisco: Jossey-Bass Publishers. Beale, A. V. (1993). Are Your Students Learning What You Think You’re Teaching? Adult Learning. January-February, 18-26. Faria, A. J., Nulsen, R. O. & Roussos, D. S. (1994). COMPETE: A Dynamic Marketing Simulation. Burr Ridge, IL Richard D. Irwin. Keys, J. B. (1989). The Management of Learning Grid for Management Development Revisited. Journal of Management Development. 8(2), 289-297. Kolb, D. A. (1984). Experiential Learning: Experience as the Source of Learning. Englewood Cliffs, NJ: Prentice Hall. Kotler, P. (1999). Marketing Management. Upper Saddle River, NJ: Prentice Hall. Lederman, L. C. (1984). Debriefing: A Critical Re-examination of the Post Experience Analytic Process with Implications for Its Effective use. Simulation and Games. 15, 415-431. Loacker, G. (1988). Faculty as a Force to Improve Instruction Through Assessment. In J. H. McMillan (Ed), Assessing Students’ Learning. San Francisco: Jossey-Bass Publishers. McDevitt, P. (1997). Measuring Student Learning using Business Simulations: A Theory Based Perspective. In John K. Butler Jr. and Nancy H. Leonard (Eds.), Developments in Business Simulation and Experiential Learning. 24, 247-252 Petranek, C. F., Corey, S. & Black, R. (1992). Three Levels of Learning in Simulations: Participating, Debriefing and Journal Writing. Simulation and Gaming. 23(2), 174- 185. Piaget, J. (1973). Psychology and Epistemology. New York: The Viking Press. 62 Table of Contents Volume 27, 2000 Internet International: A Simulation Exercise for Understanding Technological Innovation and customer Service In a Rapidly Growing Internet Server Company Simulations and Learning: Dialog and Directions Endnote Activity: A Tool for Integration of Course Content and Communication Skill Practice Incorporating Video as a Teaching Strategy in Interpersonal Communication Vision Quest: An Alternative Approach to Industry Analysis for MBA Courses in Strategic Strategic Management: An Evaluation of the Use of Three Learning Methods Trainer, Mentor, Educator: What Role for the College Business Instructor in the Next Century? Using the Internet and Shareware to Facilitate Computer Simulation in Distance Learning Classes Visual Modeling of Business Simulations Teaching about Information with Management Games A Self-Evaluation Based on the Discussion and Decision in Experts' Business Gaming The Restaurant Game Using Journals to Enhance Computer Simulation Based Learning Exercises to Facilitate Better Student Writing in the Undergraduate Strategy Class Identifying, Resolving, and Managing Common Ethical Dilemmas in the Workplace: An Experiential Approach Integrating the Digital Revolution into the Classroom The Wheel of Learning: An Integrative Business Curriculum Experiment The Changing Nature of Simulation Research: A Brief ABSEL History Perspectives on Simulation & Gaming's Review Process Experiential Learning Across Disciplines: Mixing International Business and Accounting Simulating Governmental Effects on Economic Development Internationalizing the Introduction to Business Course Using an International Text and Domestic Simulation with a Twist Using Stock Value as the Performance Measure in a Business Simulation Game Introducing Cross-Elasticities in Demand Algorithms Validating a Model of Currency Valuation An Exercise for Exploring the Relationship between Jungian Psychological Types and Organizational Politics Exercise: Preparing Financial Reports Using the Group Categorizing Technique Effect of Trust and Cultural Beliefs on Negotiation Processes: Data from an Experiential Role Play Experiential Learning Gets Stamp of Approval From the Boyer Commission Talent Search 2000 - An Experiential Activity to Help Strengthen Skills in Employee Recruitment and Selection Clemson University's Collaborative Learning Environment Initial Data on a Test Bank Assessing Total Enterprise Simulation Learning Changing the Assessment Paradigm: Using Student Portfolios To Assess Learning from Simulations How We Learn and Why We Don't: The Cognitive Profile Model: A Workshop in Teaching to Reach Your Students Knowing Thyself: A Portfolio Approach to Student Self-Assessment Collaborative Learning and Web-Based Instruction in a Cognitive Apprenticeship Model Teamwork Attributes in a Classroom Simulation Virtual Teams: Meeting the Next Challenge for Experiential Education New Age Learning: Nuance or Nonsense Developing Charisma: An Experiential Exercise in Leadership Problems and Solutions in Going Web-based with an Agribusiness Simulation Creating a Comprehensive Web-Enhanced Classroom Your Class is in Session, Now What? The Challenges of Teaching On-line An Application of Process Control Charts for Attributes as a Form of Classroom Assessment for Experiential Learning Work Goals and Life Aspirations: Do You Have What it Takes to be an Entrepreneur An Exercise to Develop Initiative: Possible Dream? The Ball Point Pen Assembly Company Management Game Review System Development Total Enterprise Simulations and Optimizing the Decision Set: Assessing Student Learning Across Decision Periods Facilitating Learning in the New Millennium with the Complete Online Decision Entry, System (CODES) The Marketing Management Experience The Right Venue for Your Simulation One More Time: Overall Dominance in Total Enterprise Simulation Performance A Profile of ABSEL Conference Attendees Learning Readiness: An Underappreciated Yet Vital Dimension in Experiential Learning Active Learning in a Professional Undergraduate Curriculum The Problem Is - They Think Differently! Cultures Integration in Mergers and Acquisitions: Putting Managers Together in a Business Simulation The Global Business Game: A Strategic Management and International Business Simulation