EXPERIMENTATION WITH ASSESSMENT TECHNIQUES: A PROPOSAL FOR PANEL DISCUSSION Developments in Business Simulation and Experiential Learning, Volume 31, 2004 EXPERIMENTATION WITH ASSESSMENT TECHNIQUES: A PROPOSAL FOR PANEL DISCUSSION Khushwant K. S. Pittenger Ashland University kpitten@ashland.edu Sandra Morgan University of Hartford Joshua Mott Ashland University Diana Page University of West Florida Constance Savage Ashland University ABSTRACT 1. Sandra Morgan - University of Hartford ASSESSMENT TECHNIQUES Experimentation in student assessment is just as critical as it is in teaching pedagogy. Actually the two are closely intertwined. Assessment is a critical part of the teaching process. The purpose of the proposed panel discussion is to share creative practices in the assessment process. I use the standard AAHE definition of assessment as a “systematic collection, review and use of information to improve student learning and development” and I try to assess students by both understanding what they learned and didn’t learn. Most of my assessment is embedded in assignments, both written and action-oriented in the classroom (e.g., presentations, group activities). Of course, I also use tests—students seem comfortable with them, and some of my undergraduates probably would never read the text if there weren’t tests scheduled. Two standard tools I use are Reflection Papers and Fast Feedback. Guay (1997) believes, “how you assess determines how you teach.” In recent years there has been a great deal of discussion in education literature about the need to re-think how we teach and how we assess our students (Thrust for Educational Leadership, V. 26, February/March 1997). Much is being written on the assessment reform (Walberg, H.J., 1994; Kane, M.B.; Khattri, N. 1995.) The purpose of the proposed panel discussion is to share creative practices in the assessment process. Experimentation and creativity in assessment has the potential to drastically reform the quality of teaching and learning in the classroom (Kerrins, 1997). REFLECTION PAPERS These are short (1-2 pages) papers required either weekly or every other week from undergraduates. The assignment is spelled out in general on the syllabus and I post (on the blackboard) and hand out the topic a week before the due date. In a class that meets twice a week, I make the paper due the first class of the week. The objective of a reflection paper is for the student to integrate theory and practice, to see how concepts from the course readings and class activity relate to his or her own experience. I ask a specific question for students to respond to although the idea of the RP has evolved from what I used to call “journal notes,” which were more free form. I found that with undergrads, the more specific I am, the better the learning. The panel consists of five members. Each member will give a brief presentation. Listed below is a brief description of what each member is expected to share. There will be at least a 20- minute question and answer session with the audience. The remaining session will be used to mine innovative assessment practices from the audience. The session is planned to be interactive and primarily discussion based. 146 mailto:kpitten@ashland.edu Developments in Business Simulation and Experiential Learning, Volume 31, 2004 Fast Feedback Two examples of RP questions in a leadership/problem-solving and decision-making course appear below: [Course number] 1 Of what you learned today, what did you find most interesting and/or useful? “Write a description (story) of a time when you had to make a major decision. Include all the story components—setting, characters, “plot” or processes used, action, results, and what you learned from this experience that will affect (or has already affected) your future behavior.” (I leave plenty of space between questions for their responses.) 2 What points need more clarity? 3 What did you like/dislike about the class process? 4 What single change could have improved today’s session the most? “In RP 4 you wrote about a major decision. In this reflection paper, you will view the decision in a different way. Analyze your process of framing and making the decision using the four elements on pages 2-3 of your text [framing, gathering intelligence, coming to conclusions, learning from feedback]. Ideally, you would name the element, (e.g., framing), then present your way of framing the decision and then give concrete support to your statement using the components of framing from page 2 (e.g., tell what your viewpoint was, list your primary criteria that would help you choose one option over the other, etc.). At the end of presenting the four elements with support from your actual decision, write about your reflection now—was it a well-managed decision based on these elements? If not, how could it have been improved?” 5 What can we (you and I) do to help you get more out of this course? 6 Overall, how would you rate today’s class? (Circle a number.) Terrible Not very good Okay Good Great 1 2 3 4 5 The responses help me determine how much the students have absorbed of the content and also figure out better ways to design the class for more learning. The forms and my response to them in the following class also model acceptance of and use of feedback, a skill we hope to inculcate in our graduates. In reading and grading (i.e., assessing) these reflection papers, I can see whether students have absorbed the conceptual material and are able to apply it in their own lives. I grade on both content and presentation, thus encouraging students to write clearly and professionally. I find that most improve measurably about half way through the semester and the few who continue to struggle I require to get tutoring from our Learning Center to improve the mechanics of writing. 2. Joshua Mott – Student, Ashland University The College of Business & Economics at Ashland University implemented competency-based curricula six years ago. This has resulted in programmatic assessment of competencies in areas such as written and verbal communication as well as course-wide experimentation by individual professors. The purpose of my presentation will be to share the course-wide experimentation. I will review all the course syllabi in the College of Business & Economics; identify the non-traditional/ innovative practices listed on these syllabi and then interview those individual professors to learn the details and specifics of those non-traditional assessment techniques listed on their syllabi. As a member of this panel, I will share the results of this qualitative survey with the audience. I usually allot 30-40% of the grade to the reflection papers in an undergraduate class. I like this method much better than requiring a research paper, which they could download from the internet! FAST FEEDBACK I use the following questions in about 3-4 classes in each course every semester; I print them out on an 8 1/2 x 11” piece paper and hand it out 10 minutes before the class is over. I ask students to respond honestly and to turn the forms in anonymously. The following class I respond to the comments and, as a class, we try to improve the process and content for the remainder of the term. This is both an assessment of student learning and satisfaction as well as an assessment of my design and presentation of the material. 3. Diana Page – University of West Florida FINAL EXAMS AND CONCEPT TESTING I will address two successful experiments with assessments in an Organizational Behavior (OB) Class. The first assessment technique concerns a final exam. This assessment was designed to determine individual student, comprehensive knowledge of OB. The instructions are quite simple, and allow students maximum flexibility and creativity in demonstrating their knowledge of OB. Students say this is the most enjoyable final exam they have ever completed. The second experimental assessment technique offers a model to determine individual application of specific concepts and readings also in an OB course. For this assessment, students are exposed to the relevant theory, they apply the theory to the classroom experience and/or their group, and finally, students describe the implications for them in the future. This technique begins with and goes beyond Kolb’s (1984) journal suggestions. Suggestions for substitutability are offered for classes other than OB. Samples of both assessments will be provided to 147 Developments in Business Simulation and Experiential Learning, Volume 31, 2004 148 participants. In addition, guidelines for evaluating both assessment techniques are offered. 4. Khush Pittenger – Ashland University As the Chairperson of the panel, I plan to serve as the moderator for the discussion. In addition, if time permits, I will share two experiments I am currently conducting in my Introduction to Management class. Both experiments are meant to encourage learning of the course content in an interactive, creative and fun manner. One involves a group project while the other is a class competition about the nature of the final exam. 5. Constance Savage – Ashland University After some experimentation, I have developed a working model for assessing student participation. This model clearly delineates faculty expectations for student participation in class and provides a process for promoting on-target class participation. I will share with the audience participation guidelines that outline the factors of class participation, their definitions and examples. In addition, I will share the mechanisms that I use for students to self-assess their class participation. I use tangible reinforcements early in the semester to encourage class participation. I use participation contracts to engage silent students. Kolb’s (1999) Learning Style Inventory helps me individualize my practices and I provide coaching throughout the semester to shape student behavior. All these techniques will be shared with the audience in as much detail as the time permits. This is definitely work-in-progress. I plan to harvest the audience’s wisdom to refine the model further. I expect the dialogue to be engaging and vigorous. REFERENCES Guay, D. (1997). Creative approaches to assessment. Thrust for Educational Leadership, Vol. 26, pages 12-14. Kerrins, J. (1997). The 4R’s of assessment reform. Thrust for Educational Leadership, Vol. 26, pages 10-11. Kolb, D.A. (1984) Experiential learning: Experience as the source of learning and development. Englewood Cliffs, NJ: Prentice Hall. Kolb, D. (1999) Learning style inventory. (version 3) Boston: Hay/McBer Walberg, H.J. (1994). Assessment Reform: Challenges and opportunities. Evaluative Report, ISBN: 0-87367-377-8, ERIC number: ED378243. Kane, M.B.; Khattri, N. (1995). Assessment reform: A work in progress. Phi Delta Kappan, Vol 77 (1) pages 30-32. Table of Contents Volume 31, 2004 Controlling the Complexity and Orenting Target Groups by a Modular, Server-Based Business Game System Learning Network Demonstration: Delivering Business Education in a Distance Learning Environment Economic Evolution, Human Capital Investment, and Adult Distributed Electronic Learning: A Literature Review Designing a Globalization Simulation to Teach Corporate Social Responsibility Developing and Teaching an Online / In-Class Hybrid: A Demonstration A Model for Evaluating Online Instruction An Evaluation of a Distributed Learning Course: A Students'-Eye Perspective Blended Learning Strategy Improved Business Writing Skills How to Receive and Process Attachemnts while Greatly Reducing the Risk of Viruses and Trojans Introducing Online Components to a Class: How to Increase teh Likelihood of Success Teaching Strategic Communications Online: Using Learning Outcomes to Develop a Case-Based Course Implementing Distance Approaches to Education: A Panel Discussion for ABSEL: Las Vegas, 2004 MANDI: Learning Management Through Field Sales Experience An International Capital budgeting Experiential Exercise A Primer To Combating Terrorism: Playing It Safe While On Overseas Assignment (An Experiential Exercise) Integrating The Business Curriculum With A Comprehensive Case Study: A Prototype The Case Brief: A Model For Case Analysis, Writing And Discussion Technology Infused Pedagogy And Delivery – A Sure Bet? A Proposal For Panel Discussion Absel Conference 2004 Research Strategy And The Bkl: Getting The Most From The Absel Archives Simple But Effective: Rediscovering The Class Discussion Needle And Thread: An Activity For Examining Various Management Behaviors A Customized Excel Data Analysis System For Use In Undergraduate Marketing Research Team Leader Selection - Does It Matter? The Power Of Perspective: Reframing Your Framing Skills For Innovative Instruction In Leadership And Influence Exercise: How Should Merit Raises Be Allocated? An Online Situation For Problem-Based Learning In A Junior-Level Management Course The Eden Alternative As A Roadway For Change: A Service Learning Quality Improvement Project Avoiding Catastrophe: The Role Of Individual Accountability In Team Effectiveness Omega Systems: A Change Management Exercise The Risks And Rewards Of Providing Students A Structured Cheating Opportunity Experimentation With Assessment Techniques: A Proposal For Panel Discussion Using A 2 - Page Case To Introduce Concepts Of Business Strategy Interactive Session The Integration Of Appreciative Inquiry And Experiential Learning For Peak Performance Appreciative Inquiry Case Story: New York City Leadership Challenge Individual Achievement Versus Team Performance: An Empirical Study With Business Games Some Strategists Don't Learn Or Can't Learn Computer Simulation: A Design Architectonic On The Value Of Bugs In Simulation Environments Online Sales Forecasting With The Multiple Regression Analysis Data Matrices Package Simulation Exercises And Problem Based Learning: Is There A Fit? A Study Of Business Game Stock Price Algorithms Assessing Individual Performance In A Total Enterprise Simulation Information Use In A Business Game Determining The Value Of A Firm Unsorting Algorithms For An Ordered List And Its Application To Business Simulations Teaching Public Finance Management Through Simulation Antecedents Of Game Performance Student Expectations Of Classroom Teaching Practices In Developing And Presenting Course Information In Hong Kong Implementation And Impacts Of The Balanced Scorecard: An Experiment With Business Games Impact: Shocking The Legacy Mindset Implementation Of The Eepad Framework Of Business Processes In An Accounting Information Systems Course Are Business Games Really Delivering What Students Are Led To Believe?? Reporting Lessons Learned: What Gets Reported; Who Gains Value Teacher Expectations Of Classroom Teaching Practices In Developing And Presenting Course Information In Hong Kong Student Reactions To The Use Of A Computer-Based Simulation As An Integrating Mechanism For A Mba Curriculum A Cognitive Investigation Of The Internal Validity Of A Management Strategy Simulation Game The Casino Challenge: Making Simulation Delivery A Safe Bet! Accounting For Company Reputation: Variations On The Gold Standard Foreign Currency Hedging: A Simulation The Influence Of Variables Easily Controlled By The Instructor/Administrator On Simulation Outcomes: In Particular, The Variable, Reflection. Absel Awareness Among Business School Faculty Validating Business Simulations: Does High Market Share Lead To High Profitability? Simulation Debriefing Procedures Coaching And Business Simulations: A Formula For Success? A Seminal Inventory Of Basic Research Using Business Simulation Games The Influence Of Scorecard Evaluation On Decisions And Outcomes