IMPROVING THE EFFECTIVENESS OF PEER EVALUATIONS Developments in Business Simulation and Experiential Learning, Volume 30, 2003 IMPROVING THE EFFECTIVENESS OF PEER EVALUATIONS James Gentry University of Nebraska-Lincoln Barbara Howard SUNY-Geneseo Mary Jo Vaughan Mercer University Alan Cudworth Mitre Consulting Gretchen Vik San Diego State University ABSTRACT The evaluation of performance in experiential exercises and simulation games has been the topic of many ABSEL papers. Despite the frequent coverage, the evaluation of student performance, and especially group performance, remains one of the topics most discussed at ABSEL conferences. The authors propose a workshop devoted to the topic of peer evaluations. The workshop will take a three-prong approach: 1. an interactive exercise dealing with peer evaluations, 2. the results of research conducted by the authors at 4 different institutions, and 3. an open forum component that will allow the participants to share their expertise with peer evaluations. The evaluation of performance in experiential exercises and simulation games has been the topic of many ABSEL papers (an incomplete list includes Anderson and Lawton 1988; Burns 1990; Butler 1981; Butler and Parasuraman 1977; Chiesl 1998; Hsu 1998; Huston, d’Ourville, and Willis 1982; Mallinger 1988; Markulis et al. 1994; Richardson, Summey, and Raveed 1988; Scott 1974; Schibrowsky and Peltier 1992; Summers, Parker, and Boyd 1985; Teach 1992; Wheatley et al. 1991; Wheatley and Platt 1993; Wilson 1974; Wolfe and Byrne 1977; Wolfe and Roberts 1992; Yahr 1995). Despite the frequent coverage and the fact that the topic was raised at the first conference by Scott (1974) and Wilson (1974), the evaluation of student performance, and especially group performance, remains one of the topics most discussed at ABSEL conferences. Most instructors realize the fallacy of assuming that group performance is the result of equal contributions by all group members. The range of deviancies from the egalitarian ideal includes the underachiever free rider (Ingham et al. 1974; Joyce 1999; Latane, Williams, and Harkings 1979) tagging on to other group members’ coattails to the dominant personality (Johnson and Johnson 1984-5) who is unwilling to share responsibilities or, at times, even to listen to the opinions of others. Burns (1990) and Schibrowsky and Peltier (1992) make clear the need for peer evaluations; and Schibrowsky and Peltier (1992, p. 222) make the succinct statement, “The easiest way to evaluate the amount of participation by each student is to use confidential peer evaluations using some sort of summated ratings.” The authors propose a workshop devoted to the topic of peer evaluations. The objective of the session will be to develop a better understanding of the peer evaluation process. The workshop will take a three-prong approach: 1. an interactive exercise demonstrating some of the aspects of peer evaluations, 2. the results of research conducted by the authors at four different institutions, which looked at both group evaluations themselves and students’ attitudes toward the peer evaluation process, and 3. an open forum component that will allow the participants to share their expertise with peer evaluations. Through the medium of an interactive exercise the presenters will identify key issues that researchers in this field will recognize. These 109 Developments in Business Simulation and Experiential Learning, Volume 30, 2003 issues will be further examined by reference to recent research carried out by the presenters and by involving all participants in a structured discussion of peer evaluations. Chiesl, Newell E. (1998) "Integrating the Marketing Curriculum Using Collaborative Learning." Developments in Business Simulations and Experiential Exercises, 25, 168-174. While confidential peer ratings have become the standard means of assessing individual contributions to the group performances, several aspects surrounding their use continue to be debated. Are students fair and objective when they rate their peers? What do students think about having to rate their peers? Can the faculty member rely on them as an accurate evaluation method? Or should they be considered an assessment tool only? Do students respond differently depending on the impact their ratings have on the final grade? Does having the ability to “fire” a member of the group play a role in the process? Are there characteristics of the students that make them better (or worse) evaluators? This workshop will address the rationale for evaluating group performance in experiential activities and the problems associated with it. Cohen, Elizabeth g. (1994) “Restructuring the Classroom: Conditions for Productive Small Groups.” Review of Educational Research, 64 (No. 1), 1-35. Conway, R., D. Kemper, A. Sivan, and M. Wu (1993) "Peer Assessment of an Individuals' Contribution to a Group Project." Assessment & Evaluation in Higher Education, 8 (Spring), 30-38. Dommeyer, Curt J. (1986) “A Comparison of the Individual Proposal and the Team Project in the Marketing Research Course.” Journal of Marketing Education, 8 (Spring), 30-38. Feichtner, Susan Brown, and Elaine Actis Davis (1984) “Why Some Groups Fail: A Survey of Students’ Experiences with Learning Groups.” The Organizational Behavior Teaching Review, 9 (4), 58– 73. REFERENCES Freeman, Kimberly A. (1996) “Attitudes Toward Work in Project Groups as Predictors of Academic Performance.” Small Group Research, 27 (May), 265– 282. Anderson, Philip H. and Leigh Lawton (1988) "A Form and Processes for Nonconfidential Peer Evaluation." Developments in Business Simulations and Experiential Exercises, 15, 121-124. Fry, Stuart A. (1990) "Implementation and Evaluation of Peer Marking in Higher Education." Assessment and Evaluation in Higher Education, 15, 177-189. Alagna, Sheryle W. and Diane M. Reddy (1985) "Self and Peer Ratings and Evaluation of Group Processes in Mixed-Sex and Male Medical Training Groups." Journal of Applied Social Psychology, 15, 31-45. Gardner, Brenda s. and Sharon J. Korth (1998) “A Framework for Learning to Work in Teams.” Journal of Education for Business, Sept./Oct., 28-33. Bacon, Donald R., Kim A. Stewart, and Sue Stewart-Belle (1998) “Exploring Predictors of Student Team Project Performance.” Journal of Marketing Education, 20 (May), 63–71. Garvin, J.W., A.C. Butcher, L.A. Stefani, V. N. Tariq, H.R. Lewis, N.L. Blumson, R.N. Govier, and J.A. Hill (1995) “Group Projects for First-year University Students: An Evaluation.” Assessment and Evaluation in Higher Education, 20 (No. 3), 273-288. Batra, Madan M., Barbare E. Walvoord, and Krish S. Krishnan (1997) “Effective Pedagogy for Student- Teams Projects.” Journal of Marketing Education, 18 (Summer), 17-27. Glazer, Rashi, Joel h. Steckel, and Russell S. Winer (1987) “Group Process and Decision Performance in a Simulated Marketing Environment.” Journal of Business Research, 15, 545-557. Beatty, James R., Robert W. Haas, and Donald Sciglimpaglia (1996) “Using Peer Evaluations to Assess Individual Performances in Group Class Projects.” Journal of Marketing Education, 18 (Summer), 17-27. Hsu, Ti (1998) "Simulation within a Simulation: Job Layoffs and Emotional Reactions." Developments in Business Simulations and Experiential Exercises, 25, 284-291. Burns, Alvin C. (1990) "The Use of Live Case Studies in Business Education: Pros, Cons, and Guidelines." in James Gentry (Ed.), Guide to Business Gaming and Experiential Learning, East Brunswick, NJ: Nichols/GP Publishing, 201-215. Huston, C. Richard, Edmond L. d'Ouville, and T. Hillman Willis (1982) "A Pedagogical Approach to Business Gaming for the Commuting Student." Developments in Business Simulations and Experiential Exercises, 9, 153-155. Butler, John K. (1981) "Student Participation in Deciding Performance Criteria for Grading in an Organizational Behavior Course: An Exercise and a Case Study." Developments in Business Simulations and Experiential Exercises, 8, 231-233. Ingham, Alan G., George Levinger, James Graves, and Vaughn Peckham (1974) “The Ringelmann Effect: Studies of Group Size and Group Performance.” Journal of Experimental Social Psychology, 10, 371– 384. Butler, Richard J. and A. Parasuraman (1977) "Degree of Uniformity in Achievement Motivation Levels of Team Members: Its Effectiveness on Team Performance in a Simulation Game." Developments in Business Simulations and Experiential Exercises, 4, 118-123. Johnson, David W. and Roger T. Johnson (1984–85) “Structuring Groups for Cooperative Learning.” The Organizational Behavior Teaching Review, 9, 8–17. 110 Developments in Business Simulation and Experiential Learning, Volume 30, 2003 Overby, John D. and Festus Olorunniwo (1986) "Pilot Analysis of Self-Peer Evaluation in an Experiential Exercise Human Resources Management Case." Developments in Business Simulations and Experiential Exercises, 23, 131-135. Johnson, David W., Roger T. Johnson, and Karl A. Smith (1998) “Cooperative Learning Returns to College: What Evidence Is There That It Works?” Change, (July/August), 27–35. Joyce, William B. (1999) “On the Free-Rider Problem in Cooperative Learning.” Journal of Education for Business, (May/June), 271–274. Pescosolido, Anthony T. (2001) “Informal Leaders and the Development of Group Efficacy.” Small Group Research, 32, 74–93. Keller, Betty G. and Ronald C. Bishop (1985) "Self-Esteem as a Source of Raters' Bias in Peer Evaluation." Psychological Reports, 56, 995-1000. Pysarchik, Dawn Thorndike and Patricia Huddleston (1994) "Experiential Learning in Human Resources: A Performance Appraisal Application." Developments in Business Simulations and Experiential Exercises, 21, 90-91. Latané, Bibb, Kipling Williams, and Stephen Harkins (1979) “Many Hands Make Light the Work: The Causes and Consequences of Social Loafing.” Journal of Personality and Social Psychology, 37, 822–832. Reingold, Jennifer (1998) “Special Report: the Best B Schools.” Business Week, October 19, 87-108. Lerner, Linda D. (1995) “Making Student Groups Work.” Journal of Management Education, 19 (No. 1), 123- 125. Richardson, Neil A., John H. Summey, and Sion Raveed (1988) "The Problems of Motivating Students and Clients in Live-Case Projects." Developments in Business Simulations and Experiential Exercises, 7, 108. Longmore, Monica A., Dana Dunn, and Glen R. Jarboe (1996) “Learning by Doing: Group Projects in Research Methods Classes.” Teaching Sociology, 24 (January), 84-91. Saaverda, Richard and Seog Kwun (1991) “Self-versus Peer Ratings of Specific Emotional Traits: Evidence of Convergent and Discriminant Validity.” Journal of Personality and Social Psychology, 60, 927-940. Mallinger, Mark (1988) "Self-Managed Learning: An Experiential Course Design Using the QWL Paradigm." Developments in Business Simulations and Experiential Exercises, 15, 231-234. Salas, Eduardo, Drew Rozell, Brian Mullen, and James E. Driskell (1999) “The Effect of Team Building on Performance, an Integration.” Small Group Research, 30, 309–329. Mallinger, Mark. (1998) “Maintaining Control in the Classroom by Giving Up Control.” Journal of Management Education, 22, 472–483. Scott, Charles R. Jr. (1974) "Application of Theory and of Computerized Grading in Management Simulation." Developments in Business Simulations and Experiential Exercises, 1, 205-208. Marcoulides, George and Mark G. Simkin (1991) "Evaluating Student Papers: The Case for Peer Review." Journal of Education for Business, 67, 80-83. Markulis, Peter M., Daniel R. Strang, Jerry Gosenpud, and Walt Wheatley (1994) "Cooperative Learning or ‘Learning to Cooperate.’" Developments in Business Simulations and Experiential Exercises, 21, 113-117. Schibrowsky, John A. and James W. Peltier (1992) "The Impact of Academic Dishonesty on Business Simulation and Experiential Learning Activites." Developments in Business Simulations and Experiential Exercises, 19, 222. McCorkle, Denny E., James Reardon, Joe F. Alexander, Nathan D. Kling, Robert C. Harris, and R. Vishwanathan Iyer (1999) “Undergraduate Marketing Students, Group Projects, and Teamwork: The Good, the Bad, and the Ugly?” Journal of Marketing Education, 21 (2), 106–117. Schibrowsky, John A. and James W. Peltier (1995) "The Dark Side of Experiential Learning Activities." Journal of Marketing Education, 17 (Spring), 13-24. Shanka, Tekle and Julie Napoli (2001) "Learning through Group Projects: The Student Perspective." On Global Marketing Issues at the Time of the Millennium, Proceedings, World Marketing Congress of the Academy of Marketing Science, Cardiff, Wales. McKendy, Thomas (1990) “Legitimizing Peer Response: A Recycling Project for Placement Essays.” College Composition and Communication, 41, 89-91. Morgan, Meg (1994) “Women as Emergent Leaders in Student Collaborative Writing Groups.” Journal of Advanced Composition, 14 (1), 1–19. Sharan, Shlomo (1980) “Cooperative Learning in Small Groups: Recent Methods and Effects on Achievement, Attitudes and Ethnic Relations.” Review of Educational Research, 50 (No. 2), 241-271. Morgan, Meg and Mary Murray (1991) “Insight and Collaborative Writing.” Collaborative Writing in Industry: Investigations in Theory and Practice, Mary M. Lay and William M. Karis, Amityville, NY: Baywood, 64-81. Siciliano, Julie (1999) “A Template for Managing Teamwork in Courses across the Curriculum.” Journal of Education for Business, (May/June), 261–264. Summers, Irvin, Steve Parker, and Charles W. Boyd (1985) "Simulation and Experiential Practices of Faculty: A Data-Based Workshop." Developments in Business Simulations and Experiential Exercises, 12, 166-167. Morse, Ken (2002) "International Management Virtual Teamwork: A Simulation." Developments in Business Simulations and Experiential Exercises, 29, 143. 111 Developments in Business Simulation and Experiential Learning, Volume 30, 2003 112 Teach, Richard (1992) "Using Forecasting Accuracy as a Measure of Success in Business Simulation." Developments in Business Simulations and Experiential Exercises, 19, 103-107. Wheatley, Walter J., Raid V. Amin, Terry R. Armstrong, and Chantele T. Vander Linde (1991) "The Effect of Leadership and Cognitive Processing Styles upon Peer Performances Evaluations: Implications for the Utilization of Simulation in Business Pedagogy." Developments in Business Simulations and Experiential Exercises, 18, 1116-120. Wheatley, Walter J. and Richard Platt (1993) "Effectively Preparing Students for Careers in a Global Environment by Integrating Total Quality Management throughout the Business Curriculum." Developments in Business Simulations and Experiential Exercises, 20, 143. Wilson, Harold K. (1974) "Administration: The Key to a Successful Gaming Experience." Developments in Business Simulations and Experiential Exercises, 1, 174-181. Wolfe, Douglas E. and Eugene T. Byrne (1977) "A Comparison of Perceived Learning in Three Pedagogically Different Sections of a Required Business Policy Game." Developments in Business Simulations and Experiential Exercises, 3, 474-478. Wolfe, Joseph and C. Richard Roberts (1992) "Peer Group Indicators of the External Validity of Business Games: A Five Year Longitudinal Study." Developments in Business Simulations and Experiential Exercises, 19, 194-198. Yahr, Michael A. (1995) "Student and Teacher Perceptions of a Management Simulation Course." Developments in Business Simulations and Experiential Exercises, 22, 124-125. Table of Contents Volume 30, 2003 The Optimal Timing For Introducing Business Simulations Can Handicapped Students Access Your Class Web Site? The Competition Game: Decision Making In A Dynamic Environment Pan-Pacific Enterprises: Strategic Decision Making Simulation Study Of Stochastic Channel Redistribution The Impact Of Business War Games: Quantifying Training Effectiveness Experiential Learning: Introducing Faculty And Staff To A University Leadership Development Program The Feasibility Of The Balanced Scorecard For Business Games A Misuse Of Pims For The Validation Of Marketing Management Simulation Games Incorporating Technology Into The 21st Century Classroom: Are We Facilitating Academic Dishonesty? Improving The Effectiveness Of Peer Evaluations The Use Of A Simulation In An Integrated Mba Curriculum Student Portfolios In Business Education Student Portfolios In Business Education At Ashland University Using SAP ERP Technology To Integrate The Undergraduate Business Curriculum Board Games And Teaching Textile Marketing And Finance Blogging: A New Threat To Student Research? The Way We Talk! Take II Strategic Management: An Evaluation Of The Use Of Three Learning Methods In Hong Kong Adoption Of Discussion-Based Teaching And Assessment In Teaching Strategic Management In Hong Kong The Tobin Q As A Company Performance Indicator Using Representative Nominal Group Technique For Course Review And An Interactive Solicitation Of Ways To Enhance Absel's Image Making Teaching Matter: The Art And Science Of Teaching Business Communication Beyond Sex, Age, And Race: Exploring The Deeper Contents Of Diversity Teaching & Learning The Facilitation Process A Brief On Debriefing: What It Is And What It Isn't Changing Perceptions Of The Importance Of Leadership: The Contribution Of Individual Spirit Harmonics In Leadership Knowledge, Skills And Sustainable Values In Learning Organizations: Some Implications From The Multicultural Virtual Classroom Challenges Of Teaching Undergraduate Organizational Behavior In A Nontraditional Time Format Interactive Online Positioning With The Web-Based Product Positioning Map Graphics Package The Longitudinal Effects Of Entrepreneurship Training On Risk Tolerance: A Look At Similarities And Differences Between Male And Female Undergraduate Students Revisiting Strategy Learning In A Total Enterprise Simulation What Are Simulations For?: Learning Objectives As A Simulation Selection Device Gaming Agency Markets Cooperate For Profits Or Compete For Market? Study Of Oligopolistic Pricing With A Business Game The Design Of A Business Simulation Using A System-Dynamics-Based Approach Modeling The Product Development Function For An Entrepreneurial Firm Simulation Performance And Forecast Accuracy? Is That All? Business Manager Identification Of Competitors In Real World And Simulation Settings Monte Carlo Simulation Analysis On The Costs Reduction Argument Of Interest Rate Swaps Ebiz Game: A Scalable Online Business Simulation Game For Entrepreneurship Training A Model For Online Education Delivery And A Look At Online Delivery Effectiveness Incorporating "Company Reputation" into Total Enterprise Simulations The Genesis And Future Of The Absel "Classicos" Initiative