ASSESSING AND INCENTIVIZING LEARNER CONTRIBUTION AND PERFORMANCE EXCELLENCE: CREATING WIN-WIN EVALUATION OPTIONS Developments in Business Simulation and Experiential Learning, Volume 34, 2007 32 ASSESSING AND INCENTIVIZING LEARNER CONTRIBUTION AND PERFORMANCE EXCELLENCE: CREATING WIN-WIN EVALUATION OPTIONS Nick Maddox Stetson University nmaddox@stetson.edu ABSTRACT This Experiential Track submission outlines two techniques used to encourage active student participation and contribution to both in-class dialogues and discussion board forums run utilizing the Blackboard learning platform. The author first explains the auditing system he uses to evaluate day-by-day class contribution and on-going contributions to existent discussion forums. The author then explains his system of Incentive for Excellence points (IEPs) used to reinforce high motivation, performance and achievement in his classes. INTRODUCTION I am a strong advocate and practitioner of dialogic and experiential teaching methods. In that context, I believe that it is important to de-adversarilize “grading” as much as possible. The methods I share herein are designed not only to prompt enhanced learner performance, but to also take such “intangible” performance factors like contribution and participation and make them real and evident to learners in classes I teach. I rarely lecture in class, preferring to give reading and/or experiential learning activities for homework preparation. The focus of my learning facilitation is open class discussion and interaction engaging everyone in the learning enterprise. Beyond that, I have regularly used Blackboard Discussion Forums as a formal and graded dimension of individual learner performance. I enact the methods described herein in Organizational Behavior (undergrad and MBA), Creativity Enhancement, Managerial Ethics and Decision Making and Organizational Development courses. I have been evolving this system over the past decade and it works most optimally in classes with less than 30 students. My undergraduate classes are comprised of typical, traditional undergraduate learners. My graduate classes are also fairly typical of MBA demographics, although our programs do tend to attract a smaller percentage of experienced learners than do regionally competitive programs. Teaching at a small, private Southern University that promotes and “features” small class size as an institutional attribute, I usually have classes with between 18 and 30 students both at the undergraduate and graduate levels. Clearly given the labor-intensiveness of the methods described in this piece, large class sizes would not be the ideal setting for the methods I describe in this piece. In my upper division Management and MBA classes, in-class contribution and contribution to Blackboard discussion forums are two independent, significant and graded dimensions of individual learner performance. As clearly established in my syllabi, learners are expected to be prepared for class, ready to add their perspectives to class discussion and highly active on posted Blackboard discussion forums. Herein, I will specify how I evaluate both domains of contribution quantitatively and qualitatively to render each learner an accurate, and largely objective, grade related to their semester-long performance in both domains. Learners who remain in my classes clearly understand that a large percentage of their final class grade will be determined by their performance in these contribution/participation domains. Additionally, in this piece, I will explain my Incentives for Excellence system of extra credit points and its impact on learner performance in all of my classes. This practice, analogous to “pay for performance methods” in the workplace is designed to enhance motivation and performance by providing additional incentives for both individual and team excellence. IN-CLASS CONTRIBUTION I encourage a learning climate wherein learners feel prepared, safe and comfortable with adding their perspectives to experiential dialogues and exercises that are the focus of class time. Ultimately I am seeking to create a climate that engages all learners collaboratively in forwarding our learning efforts. In so doing, I avoid micromanaging the dialogue and emphasize the synergistic power of collaborative learning.. While I generally guide class direction and facilitate dialogue, my learners’ contributions, individually and in groups, drives class process and progress. Learners, of their own free will, and cognizant of the contribution expectation, respond well to this approach. As discussion progresses, I make note of the quantity and quality of everyone’s individual efforts. While I acknowledge to learners that one’s extroversion versus introversion preference may influence their orientation to a demanding expectation of mailto:nmaddox@stetson.edu Developments in Business Simulation and Experiential Learning, Volume 34, 2007 33 contribution, most learners prepare and contribute regularly to our class dialogues. On average, 20-25% of a learner’s final grade is based on class contribution. During the first class of the semester, I address my performance expectations in this and all performance arenas. I clearly explain the contribution grading system and the rationale behind the heavy weighting of this performance dimension. I point out that in the world-of- work, to succeed, one must be able to articulate and justify one’s perspectives within team and problem-solving contexts. At the conclusion of this orientation discussion, students individually sign a statement indicating they clearly understand the nature and expectations for the course. I point out to them that we now have a psychological contract and if they remain in the class, they have agreed to perform within the parameters of that contract. On occasion, a student or two will drop a course because of the class contribution expectation. Below is an excerpt from a recent syllabus that defines this performance expectation: SYLLABUS EXCERPT – Class Contribution Classroom Contribution – I know everyone will actively contribute to class on a regular basis. Hence, I encourage you to envision yourself behaving in a positive, energetic, enthusiastic, and meaningful manner. Contribution greatly involves the way you present yourself and your ideas in class. You are building an impression of yourself in my mind to which I will attach a number (This is exactly what you’ll be called upon to do within the world- of-work). We will use open-ended dialogues, topical discussions based on readings and experiential activities to study and evaluate various perspectives on the issues pertinent to our class. I will use the role sheet each class to audit your contributions to individual class discussions. I will use a 6- point rating scale to establish your actual class-by-class contributions. If you are absent from class, you will receive a zero (0) for that day. If you show up for class and contribute nothing you will receive one point. If you have an exceptional contribution grade, you can receive up to six points. Otherwise, you will receive between two and five points depending on the level of your contribution. You will receive a numeric score for each class you attend. Throughout the semester, I will audit your numbers and in combination with my qualitative assessment of your performance, at the end of the semester, assign you a contribution grade. Hence your final contribution grade will be a combination of qualitative (impression-based) and quantitative evaluation based on your class-by-class actual quantity and quality of contribution. INCENTIVE FOR EXCELLENCE POINTS (IEPs) will be given to the two students voted by the class via sociometric polling as the most valuable students in the class (12 POINTS). Two runner-up MVSs will receive 6 IEPs. If your contribution grade is 90+ you will receive 8 IEPs regardless of whether you are selected as an MVS or not. ASSESSMENT OF CONTRIBUTION PERFORMANCE Using Excel, I enter each learner’s daily attendance and contribution point totals. I determine this from my auditing count and comments I have recorded on the role sheet during class. I also track tardiness and with each late arrival to class, a learner loses 1 point from his/her daily contribution point total. I retain all role sheets should a learner want to review a day’s contribution or contest an absence or tardy. While learners sometimes challenge their performance for a given class, in a semester that has, on average, 28 class meeting days, I can present a clear, detailed longitudinal “picture” of the students’ attendance patterns and contribution levels throughout the semester. On a timely basis, I confront individual patterns of absenteeism or tardiness since both can have detrimental effects on a learner’s contribution and final grade in class. Also, I encourage learners to come by to discuss their performance whenever they choose. At about midterm, I email each learner his/her standing and indicate what he or she needs to do to improve performance. This feedback includes a profile of absences, tardiness and contribution points through the last class prior to the feedback. For learners who need or desire to improve their in-class contribution, I remain available for positive consultation. I determine the final contribution grade for each learner by totaling his/her point accumulations across the semester. Then I reflect on the impression the individual learner has created and combine that with the point total for the final grade. Consistently, those learners who make the best impression across the semester are also the learners who tend to have the highest final point total. LEARNER’S RESPONSE TO METHOD AND REALITIES OF USAGE Overall, the consistent, general response of learners to the contribution element of performance is quite good. Learners appreciate that the “subjectivity” that often frames (in their perception) participation or contribution grades is reduced significantly. They also report that they enjoy the energy and dynamism of dialogue and experiential activities as opposed to the many lecture classes they endure. Learners also indicate that they like being able to see their contribution grade framed in a tangible point system. Learner feedback suggests that most go through an adjustment period to the dialogic nature of class and to the fact that they are held accountable for their performance on Blackboard Discussion Forums. Since may learners take more than one upper-division or graduate class with me, their contribution “grades” tend to improve markedly in subsequent classes. The effectiveness of this method is supported in that I have never had a grade appeal based on either the contribution or final grade in any of my classes. Additionally, I rarely hear the lamentation that my grading Developments in Business Simulation and Experiential Learning, Volume 34, 2007 34 is too subjective, although I have had interesting dialogues with students who think their self-evaluation is more accurate than the tangible record of performance I retain. I have non-empirically tracked contribution patterns across a number of classes and there appears to a trend suggesting that as a semester progresses, more contribution input is evident for the majority of learners. This trend is most evident after mid-term feedback emails. Also, I have noted that in second and third classes, learners tend to increase their overall contribution and participation as compared to their first classes with me. These patterns and the anecdotal reports of learners suggest that the system works not only for learning enhancement, but also for helping learners to become more actively expressive. When learners take a second or third class with me, they are fully acclimated to being active partners in the learning engagement of the class. Since learners can earn IEPs (Incentive for Excellence Points) for A-level performance (to be discussed below), there is tangible incentive beyond their accrued contribution point total and grade for a high level of contribution. The primary challenge to contribution monitoring is the time involved in auditing and recording performance both in and beyond class. When I initiated this process, I sometimes had difficulty auditing individual performance and also facilitating class movement. Over time, this has proved to be less of a challenge. One reason for this is that learners know that they are not dependent on me to direct class focus. A second reason is that they are fully aware that I am auditing performance as our dialogue eventuates. I am able to utilize this system because my class sizes are small in relation to the class sizes at many institutions. This approach probably is unrealistic to use in a class of more than 25 learners and the largest class in which I have used in included 33 MBA learners. While I have tried to use it in classes up to 38 learners, it becomes somewhat unmanageable in that context. DISCUSSION FORUM CONTRIBUTION I utilize Blackboard Discussion Forums as a second graded individual contribution method in a number of classes. This performance dimension usually accounts for between 15-20% of the final grade for learners in classes where it is used. I began using this graded approach to discussion board contributions about five years ago. Prior to that I had used student journals to encourage reflection and expression in regards to learning topics and focus within my classes. I made the change in order to engender greater interactive dialogue between students regarding salient issues and ideas relevant to but ancillary from in-class focus. I adopted this approach so that learners would have a second class venue wherein they can: 1) reflect on relevant class issues and topics not necessarily covered in class; 2) respond to specific questions I posed under each discussion forum; and 3) respond to and comment on the postings of their peers or me. When I initiated this learning approach, I was actively involved in discussion forums. Over time, I have come to realize that my contributions were shaping the direction of dialogue too much, hence I no longer comment directly on discussion forums, but occasionally email individual learners with ideas or comments regarding their postings. Below is an excerpt from a recent syllabus that defines this performance expectation: SYLLABUS EXCERPT Discussion Group Contribution – On BLACKBOARD, there is a section devoted entirely to discussion boards and forum. I will be regularly adding forums wherein you should make your discussion forum postings. Prior to commenting on other’s posts within a forum, you must first respond to the forum questions from your own perspective. Across the semester, I will post approximately nine boards for you to address. Each will remain posted for approximately 14 days during which time you can make your postings. Given that this is a mandatory facet of class performance, you should regularly contribute to the discussion threads with ideas, insights, opinions, etc. that relate to the learning that occurs within class or within your own views of class topics and discussion forum questions or almost anything that is clearly relevant to the topics and coverage within this class. As with class contribution, I will audit the number of postings you make within a given forum and assign a quality rating of between 1 and 6 points per posting based on the criteria below. Hence, both quantity and quality of your postings will shape your final grade on this performance dimension. The more quality postings you have, the higher your grade. In previous classes where I have used a graded Discussion Board approach, students who have received Cs or below in class have simply and literally failed to address this aspect of performance. The discussion boards allow us to go beyond the confines of classroom time and space and consider additional issues, topics and questions for deliberation and reflection. I will read and audit everyone’s participation in all discussion forums. By the end of the semester, I will have a very clear, documented “picture” of your level and quality of discussion input. Keep in mind that you need to make certain (via CIT) that you have access to the INTRANET if you are planning to make your contributions from a distant site. Contact CIT at 7217 if you have a question. I will evaluate your semester long contribution to the discussion group on the following criteria: 1) Frequency and regularity of your posting (more is better than less); 2) Quality of your postings (high is better than low); 3) Reflectiveness and personalization of your postings; and 4) Relevancy of posting to material covered within the class. IEPs: If you make 90+ on the Discussion Contribution, you will receive 12 IEPs. Developments in Business Simulation and Experiential Learning, Volume 34, 2007 35 ASSESSMENT OF DISCUSSION BOARD CONTRIBUTION PERFORMANCE Assessment of learner performance is straightforward. I read each and every posted entry for every learner on each forum. For each posted entry, a learner receives one “frequency” point. So, if for instance, a learner makes seven posts on a particular forum, he/she will receive seven “frequency” points. I also evaluate the “quality” of each post based on the criteria identified in the syllabus excerpt above. A learner can receive between 1 and 6 points on the quality dimension for each post. For each forum, I add up both the frequency and quality points for each learner. After the last discussion forum is closed, I total up both frequency and quality points for each learner. This provides me with a very accurate longitudinal framework to assign a contribution grade in this domain. Learners who have chosen to be active in forum discussions do well and those who chose not to participate suffer the natural consequences of choosing not to participate. PARTICULAR BENEFITS OF THIS METHOD While some may question whether discussion forum contribution should be a graded aspect of learner performance, I assert that if I am going to ask learners to do something, I should responsibly provide them with legitimate evaluation of their efforts (or lack thereof). Three other benefits of the method stand out. First, as a result of reading all postings, I get a clear window into the thinking style, motivation and writing ability of every learner. Contributing regularly and meaningfully to discussions takes time, effort and reflection. The degree to which a learner is committed to his/her own achievement becomes readily apparent after several forums. When appropriate I provide positive or constructive feedback to learners on these factors via emails to them. Second, the discussion forums are accessible via Blackboard from any computer with internet access anytime of the day or night. Hence, learners cannot make excuses for not having contributed. Just as with class contribution, they know at the beginning of class that they are expected to contribute and whether they do or not remains completely their decision to make. Lastly, for learners who might be inclined to use their “introversion” as an excuse for not contributing to classroom dialogues, the discussion forums offer them a safe haven to communicate their perspectives and their reactions to others perspectives. I have found that those reticent to contribute in class are often the “stars” of the discussion forums. The major drawback of the method is that it is very labor-intensive. Reading and evaluating posts takes a good bit of time as does cataloging points. When 8-10 forums run within a semester a lot of time can be eaten up working within the method. However, uniform feedback from students is positive both in terms of the learning they gain about themselves and others. INCENTIVE FOR EXCELLENCE POINTS About a decade ago, I tried a new method for awarding extra credit points to students. Instead of the usual methods of handing out free points, I decided to link my allocation of extra points to excellent performance in any and all of the performance domains of my classes. Hence, if a student or student team earns A-level work (90+) on any facet of performance, he/she/they will receive a prescribed and published number of Incentive for Excellence Points (IEPs). In many ways, this system is similar to a “pay for performance” system in a real organization. Those learners who put forth the best effort with the best results will benefit themselves with the addition of IEPs to their final point total (as illustrated in the syllabi excerpts above). I contend that this method brings the entire performance of a class up as learners see that with excellent achievement they can directly and meaningfully improve their final grade in any class. All my classes are graded on a 1000 point relative grading scale. Learners are not in competition with one another. Rather all are attempting to earn as many of the 1000 points as they can within the mix of performance requirements. IEPs are added to the final point total for each learner earning IEPs. EXAMPLES OF IEP CREDIT Attendance: Attendance is a graded component of class performance. Learners receive a percentage grade for the number of classes they attend against the number of classes that are held. If a learner misses no classes, he/she will receive 15 IEPs. If a student misses but one class, he/she will receive 7 IEPs. Since instituting this system, I have had a 90+% attendance rate across my classes. I utilize a “no excused absences” policy so any absence is an absence. While that may seem controversial, I rarely have to listen to learners make up excuses for why they miss classes. Contribution: As noted above if a learner earns 90+ on either/both class contribution and discussion forum contribution, he/she will receive up to 10 IEPs. In terms of contribution, I also use a Most Valuable Student polling method (as identified in the syllabus excerpt above) to provide more IEPs to the students selected by their peers as having the most positive impacts on in-class learning. Projects, Team Cases and Exams: For projects, exams and cases, I usually use a team approach. Teams receive IEPs for each case, exam or project element (paper and presentation) for which they earn A-level work. Teams with the highest project grade, exam average or average on team cases are rewarded with additional IEPs for their overall achievement. Developments in Business Simulation and Experiential Learning, Volume 34, 2007 36 The IEP system has been well-received by my students and can make a significant contribution to their final grades in my demanding classes. I have never received a complaint about the system. It has been widely emulated by colleagues at my institution. CONCLUSION The learning approaches and accompanying grading schemes described in this piece are not suitable for all learning contexts nor for all educators. While they are labor- intensive endeavors for the educator, they have proven, in my experiences over a decade, to enhance student motivation, performance and learning. Sometimes, we as experiential educators, need to think outside of our own “boxes” and experiment with new ways of generating learner energy. These undertakings described above are my small contribution, at this point, to that undertaking. Table of Contents Volume 34, 2007 Experiential Teaching May Lead To Experiential Learning The Role Of Learning Versus Performance Orientations When Reacting To Negative Outcomes In Simulation Games: Further Insights An Analysis Of The Interaction Of Firm Demand And Industry Demand In Business Simulations Consistency Of Participant Simulation Performance Across Simulation Games Of Growing Complexity Assessing And Incentivizing Learner Contribution And Performance Excellence: Creating Win-Win Evaluation Options The Technological Impact Analysis: A Research-Based Exercise To Heighten Learners' Technological Sensitivity Beginnings: How We Start the Semester and Individual Classes : A Roundtable Discussion Simulation Performance and Its Effectiveness As A PBL Problem: A Follow-Up Study The Use of Multimedia Learning Tools to Facilitate Online Learning of Business Statistics Forecasting Accuracy And Learning: A Key To Measuring Business Game Performance Panel Discussion: Alternative Ways of Using the Internet for Business Simulations to Input Decisions, Process, and Present Financial and Economic Data Output Absel Research -- One Additional Perspective On Where We Are And Where We Have Come From Assessing participant learning in a business simulation From Case Presentation To Case Facilitation: How Assessment Changed The Capstone Course The Use Of Computer-Assisted, Interactive Role-Play Simulation In Hong Kong Corporate Positioning: A Business Game Perspective Demonstration Of A Computer-Assisted Global Business Simulation Applying .NET Remoting To A Business Simulation Assessing Emotional Intelligence: The EQ Matrix Exercise Outcomes And Observations Of An Extended Accounting Board Game The Use Of Learning Styles Questionnaire In Hong Kong Forming Teams For Classroom Projects Online Budgeting And Marketing Control With The Proforma Analysis Package The Effect Of Experiential Learning Experiences On Management Skills Acquisition Students' Perceptions on the Individual Managerial Performance Orderness: A New Definition Of Alignment An Exploratory Study Of The Efficacy Of Servsafe® Online Team Behavior And Team Success Results From A Board Game Simulation Telecommuting Internships - Do They Work? The WEE GAME: A Pre-Game Effects On Learning When Students Have Information About Games And Their Outcomes When Playing Them Assessment And Simulations: Measuring The Academic Learning Compacts Within Dangerous Business: An Interactive Ethics Case Activity The Paradise Islands Assessing And Applying fiAppropriatefl Conflict Management Styles Distance Learning In Communication: Blended Or On-Line? Developing An On-Line Advanced Communication Course World Café: Simulating Seminar Dialogues in a Large Class The Programming Game: An Exploratory Collaboration Between Business Simulation And Instructional Design The Canary Principle: An Alternative Model for Providing Real-Time Coaching in an On-Line Discussion Environment Assessing Character Development Using A Direct-Approach Business Ethics Exercise Persistence Of Decisions By Simulation Game Participants The Invalidity of Profit = f(Product Quality) PIMS Validation of Marketing Games A Study of The Applicability of The Perceptions of Organizational Politics Scale (POPS) For Use in The University Classroom The ROI Of Connected Projects: A Payroll Example Initiation of Research on Gaming Simulation in Japan Inappropriate Use of Citations and Corrupting the Body of Knowledge: Accepting Urban Legends as Truth An Exploration Of The Perceived Value Of Highly Socio-Inductive Learning How "Whole" Is Whole Person Learning? An Examination Of Spirituality In Experiential Learning Modeling Outsourcing and Strategy Alignment into a Business Game Successful Integration Of Webct Into A Small Business School The Thinking Steps Model In Game Theory: A Qualitative Approach In Following The Rules The Application Of Means-End Theory To Understanding The Value Of Simulation-Based Learning Armchair Travel: An active learning approach to increasing global awareness and participant self-efficacy Goldratt's Thinking Process: Is there a place for it in the Total Enterprise Simulation Assessing And Developing Student Skills Using a Group Exercise Computer Business Simulation Design: Novelty & Complexity Issues