AN APPLICATION OF PROCESS CONTROL CHARTS FOR ATTRIBUTES AS A FORM OF CLASSROOM ASSESSMENT FOR EXPERIENTIAL LEARNING Developments in Business Simulation & Experiential Learning, Volume 27, 2000 AN APPLICATION OF PROCESS CONTROL CHARTS FOR ATTRIBUTES AS A FORM OF CLASSROOM ASSESSMENT FOR EXPERIENTIAL LEARNING Douglas L. Micklich, Illinois State University ABSTRACT The use of Total Quality Management Techniques in classroom assessment aids in determining whether a particular process (experiential exercise) is either in or out-of- control and to assist in identifying possible causes of variation in various aspects of this process. Control charts for attributes (non- conformance) were used to assess this process. It was found that sources could be identified where adjustments can be made that would enhance the process. Limiting the number of standard deviations associated with the control also assisted in identifying where the process would considered to be in or out-of-control. INTRODUCTION In an effort to improve the learning process and enhance experiential learning in the classroom the application of Total Quality Management (TQM) concepts can assist in determining where changes to the learning process can be made. Every process involving humans and/or machines displays variations. Excessive variations, however, causes the process and systems to be erratic and unpredictable and mediocrity and poor quality result. (Cornesky, 1993) By applying the concepts of TQM to classroom assessment, we can attempt to identify causes of variation in the process of classroom learning and begin to make any adjustments necessary. The causes of variation can be identified as being either "common" or random causes or "special" or assignable causes of variation. TQM's goal in classroom assessment is to ensure that students work together to create the best possible system of learning rather than assuming the current one is the only way. (Cornesky, 1993) Research was conducted in the fall of 1995 to explore faculty and student perceptions of the effectiveness of the use of the Cross-Angelo Classroom Assessment Techniques (CAT's) as a method of improving the quality of teaching and learning process. (Soetaert, 1998) From this study four areas were identified where assessment techniques were found to be particularly helpful. These areas are: 1) student learning, 2) the process of instruction, 3) the classroom environment, and 4) promotion of student cognition. (Exhibit I) (Soetaert, 1998) In the exercise under study in this paper, concept mapping, control charts for attributes were used to test the stability of the system and measure variations within that system. Statistical Process Control (SPC) for attributes uses qualitative characteristics and measurements, in determining the success or failure or a product or service. Whereas the normal application of process control is to check for defects or errors occurring in a process, the application in this instance is to test the stability of the process in relation to whether student's realized benefits from the exercise. METHODOLOGY Students (n=32) were exposed to a series of three concept mapping exercises over the course of a semester. They were then asked to complete a survey questionnaire that included in their responses the following five areas. These are: 1) when they completed the exercise in relationship to an exam, 2) how much time they spent working on the exercise, 3) whether they worked on it individually or in a group, 4) the instructions concerning the exercise, and 5) which did they find most beneficial in 219 Developments in Business Simulation & Experiential Learning, Volume 27, 2000 developing their understanding, primarily individual work or group work. A C-chart was used to test the stability of the process. This type of chart is useful when the characteristics or attributes under study are too complex for just a simple yes/no type answer. This complexity is found in non-conformance attributes. Non-conformance attributes are defined as those that do not demand the yes/no response. Those attributes and their response choices are found in Exhibit II. Charts were constructed and upper and lower control limits were calculated for each exercise and then for the exercise as a whole. Totals for each person for each time were plotted on the chart to determine if there were any instances where the process may be considered out of control. These were then compared to what was most beneficial in developing their understanding. (Exhibit III) The upper and lower control limits were set to two standard deviations rather than three because narrower limits make it easier to detect changes in the process that are not random and have assignable causes. Wider limits (three standard deviations) make it less likely to erroneously conclude that the process is out of control when points outside the control limits are due to normal, random variations. RESULTS There were 26 responses to the survey, of which 23 were valid because three people did not complete the second page of the survey. For each separate iteration of the exercise, the process was found to be in control as no points exceeded either the upper or lower control limits calculated. It was also noticed that for each subsequent iteration of the exercise, the range between the control limits became smaller. When the separate times were aggregated, and then combined with their perceptions of which mode was most beneficial, there was only one instance of the upper control limit being exceeded and none that fell below the lower control limit. A majority (n=19) were found to be within 1 standard deviation of the mean. EXHIBIT I Assessment Area Student Learning Identify concepts that are unclear, confused, or missed completely. Process of Monitor new Instruction Instruction strategies Classroom Recognizing environment student's opinions Student cognition Enable students to organize concepts and clarify their thinking about the concepts. EXHIBIT II Non-Conformance Attributes 1. Was the map completed either: - Before attempting the exam - While completing the exam - After completing the exam 2. Time spent completing the exam: - Less than 1 hour - 1 to 2 hours - 2 to 3 hours - More than 3 hours 3. Mode used when completing the map: - Worked individually - Worked individually, then in a group - Worked in a group, then individually - Worked in a group 4. Were the instructions concerning the exercise: - Poor - Fair - Adequate - Good - Excellent 220 Developments in Business Simulation & Experiential Learning, Volume 27, 2000 EXHIBIT III Which did you find most beneficial in developing your understanding? - Working individually - Working individually, then in a group - Working in a group, then individually - Working in a group REFERENCES Cornesky, Robert., (1993). The Quality Professor: Implementing TQM in the Classroom, MAGNA Publications Inc. Soetaert, Elaine, (1998). Quality in the Classroom: Classroom Assessment Techniques as TQM, NewDirections for Teaching and Learning, #75, 47-55 221 Table of Contents Volume 27, 2000 Internet International: A Simulation Exercise for Understanding Technological Innovation and customer Service In a Rapidly Growing Internet Server Company Simulations and Learning: Dialog and Directions Endnote Activity: A Tool for Integration of Course Content and Communication Skill Practice Incorporating Video as a Teaching Strategy in Interpersonal Communication Vision Quest: An Alternative Approach to Industry Analysis for MBA Courses in Strategic Strategic Management: An Evaluation of the Use of Three Learning Methods Trainer, Mentor, Educator: What Role for the College Business Instructor in the Next Century? Using the Internet and Shareware to Facilitate Computer Simulation in Distance Learning Classes Visual Modeling of Business Simulations Teaching about Information with Management Games A Self-Evaluation Based on the Discussion and Decision in Experts' Business Gaming The Restaurant Game Using Journals to Enhance Computer Simulation Based Learning Exercises to Facilitate Better Student Writing in the Undergraduate Strategy Class Identifying, Resolving, and Managing Common Ethical Dilemmas in the Workplace: An Experiential Approach Integrating the Digital Revolution into the Classroom The Wheel of Learning: An Integrative Business Curriculum Experiment The Changing Nature of Simulation Research: A Brief ABSEL History Perspectives on Simulation & Gaming's Review Process Experiential Learning Across Disciplines: Mixing International Business and Accounting Simulating Governmental Effects on Economic Development Internationalizing the Introduction to Business Course Using an International Text and Domestic Simulation with a Twist Using Stock Value as the Performance Measure in a Business Simulation Game Introducing Cross-Elasticities in Demand Algorithms Validating a Model of Currency Valuation An Exercise for Exploring the Relationship between Jungian Psychological Types and Organizational Politics Exercise: Preparing Financial Reports Using the Group Categorizing Technique Effect of Trust and Cultural Beliefs on Negotiation Processes: Data from an Experiential Role Play Experiential Learning Gets Stamp of Approval From the Boyer Commission Talent Search 2000 - An Experiential Activity to Help Strengthen Skills in Employee Recruitment and Selection Clemson University's Collaborative Learning Environment Initial Data on a Test Bank Assessing Total Enterprise Simulation Learning Changing the Assessment Paradigm: Using Student Portfolios To Assess Learning from Simulations How We Learn and Why We Don't: The Cognitive Profile Model: A Workshop in Teaching to Reach Your Students Knowing Thyself: A Portfolio Approach to Student Self-Assessment Collaborative Learning and Web-Based Instruction in a Cognitive Apprenticeship Model Teamwork Attributes in a Classroom Simulation Virtual Teams: Meeting the Next Challenge for Experiential Education New Age Learning: Nuance or Nonsense Developing Charisma: An Experiential Exercise in Leadership Problems and Solutions in Going Web-based with an Agribusiness Simulation Creating a Comprehensive Web-Enhanced Classroom Your Class is in Session, Now What? The Challenges of Teaching On-line An Application of Process Control Charts for Attributes as a Form of Classroom Assessment for Experiential Learning Work Goals and Life Aspirations: Do You Have What it Takes to be an Entrepreneur An Exercise to Develop Initiative: Possible Dream? The Ball Point Pen Assembly Company Management Game Review System Development Total Enterprise Simulations and Optimizing the Decision Set: Assessing Student Learning Across Decision Periods Facilitating Learning in the New Millennium with the Complete Online Decision Entry, System (CODES) The Marketing Management Experience The Right Venue for Your Simulation One More Time: Overall Dominance in Total Enterprise Simulation Performance A Profile of ABSEL Conference Attendees Learning Readiness: An Underappreciated Yet Vital Dimension in Experiential Learning Active Learning in a Professional Undergraduate Curriculum The Problem Is - They Think Differently! Cultures Integration in Mergers and Acquisitions: Putting Managers Together in a Business Simulation The Global Business Game: A Strategic Management and International Business Simulation