*Corresponding author - yyscarpe@ucalgary.ca Urch, G., & Carpenter, Y. (2019). Conversations and perspectives on peer feedback for problem solving. Papers on Postsecondary Learning and Teaching: Proceedings of the University of Calgary Conference on Postsecondary Learning and Teaching, 3, 81-86. CONVERSATIONS AND PERSPECTIVES ON PEER FEEDBACK FOR PROBLEM-SOLVING Geoffrey Urch and Yuen-ying Carpenter* University of Calgary Peer feedback has been suggested as an avenue to leverage students as partners in their own learning and assessment across many disciplines. However, successful implementation of peer feedback activities may prove challenging if students believe that feedback requires more objective expertise than they possess. Summarizing participant contributions and dialogue from a conversation café session at the 2018 University of Calgary Conference on Postsecondary Learning and Teaching, this paper explores and classifies the common themes in peer feedback in the context of literature on the subject. The most promising areas for future research and practitioner support in scientific problem-solving tasks are highlighted. Peer review forms a critical component of scientific publishing, and critique forms the backbone of professional dialogue across disciplines, ranging from engineering design to architecture and fine art. Students often enter university with experience giving peer feedback in contexts that they perceive as subjective (writing, art, oral presentations). However, students have had infrequent, if any, experience offering qualitative peer feedback on potentially more objective problem-solving tasks. When asked to provide comment on peers’ preliminary solutions to a given problem, students often defer to authoritative or expert views – “How can I give any useful feedback if I don’t know the right answers?” they ask. Engaging students as partners in their own and each other’s evaluation adds a new dimension to the classroom. For instructors interested in delving into peer feedback, the breadth of differing and sometimes conflicting models, and potential implementation strategies for peer feedback and peer assessment can likewise prove overwhelming. In the interest of surveying current perspectives and awareness of peer feedback approaches, the co-authors facilitated a conversation café at the 2018 University of Calgary Conference on Postsecondary Learning and Teaching. While the attendees participating in this conversation form only a small sampling of the postsecondary educator population, themes in this conversation highlight common threads and notable absences in the dialogue around this practice. We share herein a summary of the issues and perspectives raised in this conversation, along with the connection to the literature in this area, with the aim of igniting renewed dialogue around promising peer feedback approaches that might be less well-known to this community. As the terminology around peer feedback is as diverse in its use as the approaches themselves, we would like to emphasize that we will use the term peer feedback herein to broadly refer to any activity where students engage in review of and provide feedback on each other’s work. Urch & Carpenter (2019) 82 EMERGENT CONVERSATION THEMES Participants in the conversation café were invited to first share their incoming views of peer feedback based on selected prompts from the facilitators. Participants’ contributions were then used to direct the facilitated conversations which followed. After the conference, participants’ contributions and facilitator’s notes from the group discussion were transcribed and analyzed for emergent themes around peer feedback in science education. Why peer feedback? When broadly asked to share their motivations and goals for engaging students in peer feedback tasks, participants highlighted both potential positive outcomes for students and the connection between peer feedback and those outcomes (Table 1). Table 1 Recurring themes in conference participant responses when asked to brainstorm around the questions, “Why are you interested in peer feedback?” and “What are your goals?” Peer feedback ideally results in... ...because of... Increased performance & content mastery Mutual benefit Increased reflection Increased critical thinking Professional skills (critique and communication) Increased quantity of feedback Exposure to a range of alternatives Opportunities for comparison Participants were motivated to use peer feedback as a mechanism for broadly improving student performance within their courses, either through revision of a student’s initial work or by transfer of assignment learning gains toward future assessments. A majority of participants mentioned the mutually beneficial nature of peer feedback for students both receiving and providing feedback. Regardless of the participant’s discipline, the potential impact of peer feedback on students’ capacity for critical thinking and reflection was highlighted. A smaller portion of the group noted that peer review or critique was a crucial professional skill within their practice (e.g. in reviewing scientific journal articles) and was therefore an explicit learning outcome for their course or degree program. Participants sentiments are echoed by existing studies in the literature, including studies reporting improved performance on both immediate assignment-level outcomes (McGourty, Dominick, & Reilly, 1998; Reinholz, 2016) and course-level outcomes such as improved pass rates even (Reinholz, 2016). Likewise, Çevik (2015) reported that both assessors and assessees improved their problem-solving skills when engaged in peer feedback of their work on ill- defined or open-ended problems. However, in contrast to the assumptions and anecdotal observations of our participant pool, student assessors in this study demonstrated larger gains than their assessed peers in their ability to justify their solutions to later problems. Even in contexts where students were revising initially submitted work based on peer feedback, it was the students providing feedback who produced higher quality work upon their revised submission, not those who only received feedback (Li, Liu, & Steckelberg, 2010). Urch & Carpenter (2019) 83 For some participants, peer feedback was also a solution to the practical dilemma of providing ongoing feedback to a large pool of students within the constraints of limited available time from experts (i.e. instructors and teaching assistants). However, more participants emphasized that engaging in peer reviewing allowed students to explore a larger breadth of potential solutions and problem-solving approaches. Several studies support this rationale for the positive benefits of student peer review activities, noting that the opportunity to compare a range of solutions above and beyond an expert-generated solution guide was a key factor in achieving student improvement (ArchMiller, Fieberg, Walker, & Holm, 2017; Carnell, 2016). Given the significant potential benefits of engaging students in peer feedback activities, it follows that it would be important to elaborate on other factors which support successful implementation of such strategies. What makes for ideal peer feedback? Participants were therefore tasked with brainstorming what ‘good’ feedback should look like (to achieve their desired outcomes), and what barriers they perceived to student buy-in and success at providing this idealized feedback. Most notably, responses to these questions were nearly always paired; participants had an idea of the features of feedback they hoped students would provide. Participants also predicted specific challenges in each case. For example, participants’ descriptions of high quality feedback focused on measuring the impact of that feedback. Using words such as constructive, thoughtful, and motivating, participants spoke primarily of the need for feedback to be useful to the recipient, rather than the qualities which made it useful. At the same time, participants were concerned that students’ comparative lack of content expertise and growing communication skills would negatively impact feedback correctness and utility. Reassuringly, in the literature, when feedback quality was rated according to alignment with assessment criteria, specificity, presence of justification for the feedback, presence of suggestions for improvement, and clarity, only the presence or absence of justification was positively correlated with student performance on their revised written work (S. Gielen, Peeters, Dochy, Onghena, & Struyven, 2010). Even more surprisingly, the authors noted that even feedback which was incorrect and poorly aligned to the project outcomes yielded performance improvements if justification was provided by the reviewer. Roughly one-third of participant suggestions emphasized the need for appropriate tone and objectivity in peer feedback, reflecting participants’ focus on maintaining a safe classroom environment even when introducing students to non-anonymized peer critique. Indeed, Kaufman and Schunn (2011) noted that students who perceived the feedback they received as unfair or biased were more likely to have negative views of peer assessment activities, considering such tasks to have low value for their learning. Participants expressed significant concern this would result in low effort and less meaningful student participation. While student perceptions of equity and fairness can be improved if student work is assessed and graded by both a peer and by an instructor (Kaufman & Schunn, 2011), previously discussed constraints on instructor-time limits the practicality of this approach. Instead, instructors may consider alternate strategies previously shown to alleviate such concerns among students, such as activity framing which includes the rationale for peer feedback (Thomas, Martin, & Pleasants, 2011), use of detailed and highly structured rubrics (M. Gielen & De Wever, 2012; Panadero, Romero, & Strijbos, 2013), and additional student training and practice at giving feedback (Reinholz, 2016; van Zundert, Sluijsmans, & van Merriënboer, 2010). Urch & Carpenter (2019) 84 Although participants agreed that student buy-in was a common concern for them, their most significant collective concern was the potential for students to be misled by their peers. Participants worried that students might receive substantially different, even conflicting, feedback from their peers, leading to future misconceptions, or anxiety when there is a later mismatch with the instructor’s evaluation. Recent work in peer-assessed grading of written work shows acceptable margins for agreement between student and teacher assessments (De Wever, Van Keer, Schellens, & Valcke, 2011). Since students’ may rate their peers more positively than an expert evaluator, particularly if the peer is also a friend, use of structured grading rubrics minimized differences between instructor and student ratings (Panadero et al., 2013). Alternately, comparative judgments, where students directly compare their peers’ solutions to each other or to exemplars, resulted in increased reliability over students’ absolute judgements and are particularly useful in more open-ended activities when the breadth of possible student solutions is unknown or too broad for an effective rubric to be designed (Jones & Wheadon, 2015; Potter et al., 2017). CONVERSATION GAPS: EVALUATIVE VS REFLECTIVE FEEDBACK While participants were comfortable discussing a breadth of features and challenges in peer feedback for assessment, the use of peer feedback in an ungraded, problem-solving context was notably absent from the conversation. Significant portions of the conversation focused on ensuring reliability of student-assigned grades, and several participants expressed surprise when facilitators suggested avoiding the problem entirely by using more reflective and qualitative peer feedback in the absence of a student grading task. Notably, some instructors in the group commented that they chose graded peer feedback explicitly to offer regular formative assessment that would otherwise be impossible given their marking load. However, even the broader literature on peer feedback offers comparatively few examples of peer feedback activities which are not assessment-oriented (Carnell, 2016; Hamer, Kell, & Spence, 2007; Reinholz, 2015; Reinholz & Dounas-Frazer, 2016), especially for non-essay-based tasks. While the literature on, and therefore exposure to, ungraded peer feedback is comparatively limited, we would argue that the potential value of these activities is significant enough to warrant increased attention. To demonstrate the areas better served by a formative, ungraded peer feedback model, we will use the IDEA framework for the benefits of peer feedback (Reinholz, 2015): Iteration, Discussion, Explanation, and exposure to Alternatives. While participants explicitly acknowledged the benefits of exposing students to a range of diverse approaches to problem-solving (Alternatives), their focus as a group on graded, written peer feedback meant the other three aspects of this model were often outside the scope of their existing experience. As such, opening the door to these aspects offers significant opportunities for growth in their peer feedback activity design and facilitation practices. For example, unless students were submitting essay-based work or similar, participants did not typically ask students to revise and resubmit their problem-solving work. Yet, in several studies in introductory undergraduate math and physics, students who engaged in structured ungraded peer feedback activities (referred to as peer-assisted reflection) developed an Iterative approach to their problem-solving work, an approach which more closely resembles authentic scientific practice (Reinholz, 2015; Reinholz & Dounas-Frazer, 2016). Likewise, students in these studies became more focused on Explaining and justifying their approach to any given task; these justifications not only offer potential benefits to the recipient (S. Gielen et al., 2010), but the need to explain their assessment reveals areas of concern in a student’s knowledge on a topic (Lombrozo, 2006) Urch & Carpenter (2019) 85 and can help students to reflect on their work (Reinholz, 2016). Finally, receiving only written feedback, students miss out on the potential benefits of Discussing their peer’s work and justifications as a dialogue, which students often commented was a particularly positive aspect of the feedback process (Carnell, 2016 and references therein). MOVING FORWARD WITH PEER FEEDBACK In recognizing the potential to expand to a broader, more reflective framework for the key aspects of peer feedback like IDEA and away from a strict adherence to evaluative, criterion- based peer grading, we hope to highlight opportunities for instructors and students to engage with each other in new learning partnerships. Numerous opportunities likewise exist in the literature to study and develop new practical frameworks that better define the impacts of different models and scaffolding in supporting students’ growing practice as reflective and critical reviewers in distinct scientific disciplines, each with unique conventions. Returning to the goals of our participants and others interested in delving into peer feedback, the question remains open: How can instructors best support students, still relative novices in the discipline, in engaging with peer feedback in problem-solving? REFERENCES ArchMiller, A., Fieberg, J., Walker, J. D., & Holm, N. (2017). Group peer assessment for summative evaluation in a graduate-level statistics course for ecologists. Assessment & Evaluation in Higher Education, 42(8), 1208–1220. https://doi.org/10.1080/02602938.2016.1243219 Carnell, B. (2016). Aiming for autonomy: formative peer assessment in a final-year undergraduate course. Assessment & Evaluation in Higher Education, 41(8), 1269–1283. https://doi.org/10.1080/02602938.2015.1077196 Çevik, Y. D. (2015). Assessor or assessee? Investigating the differential effects of online peer assessment roles in the development of students’ problem-solving skills. Computers in Human Behavior, 52, 250–258. https://doi.org/10.1016/j.chb.2015.05.056 De Wever, B., Van Keer, H., Schellens, T., & Valcke, M. (2011). Assessing collaboration in a wiki: The reliability of university students’ peer assessment. The Internet and Higher Education, 14(4), 201–206. https://doi.org/10.1016/j.iheduc.2011.07.003 Gielen, M., & De Wever, B. (2012). Peer assessment in a wiki: Product improvement, students’ learning and perception regarding peer feedback. Procedia - Social and Behavioral Sciences, 69, 585–594. https://doi.org/10.1016/j.sbspro.2012.11.450 Gielen, S., Peeters, E., Dochy, F., Onghena, P., & Struyven, K. (2010). Improving the effectiveness of peer feedback for learning. Learning and Instruction, 20(4), 304–315. https://doi.org/10.1016/j.learninstruc.2009.08.007 Hamer, J., Kell, C., & Spence, F. (2007). Peer assessment using Aropä. Proceedings of the Ninth Australasian Conference on Computing Education, 66, 43–54. Retrieved from http://dl.acm.org/citation.cfm?id=1273672.1273678 Jones, I., & Wheadon, C. (2015). Peer assessment using comparative and absolute judgement. Studies in Educational Evaluation, 47, 93–101. https://doi.org/10.1016/j.stueduc.2015.09.004 Urch & Carpenter (2019) 86 Kaufman, J. H., & Schunn, C. D. (2011). Students’ perceptions about peer assessment for writing: their origin and impact on revision work. Instructional Science, 39(3), 387–406. https://doi.org/10.1007/s11251-010-9133-6 Li, L., Liu, X., & Steckelberg, A. L. (2010). Assessor or assessee: How student learning improves by giving and receiving peer feedback. British Journal of Educational Technology: Journal of the Council for Educational Technology, 41(3), 525–536. https://doi.org/10.1111/j.1467-8535.2009.00968.x Lombrozo, T. (2006). The structure and function of explanations. Trends in Cognitive Sciences, 10(10), 464–470. https://doi.org/10.1016/j.tics.2006.08.004 McGourty, J., Dominick, P., & Reilly, R. R. (1998). Incorporating student peer review and feedback into the assessment process. In Frontiers in Education Conference, 1998. FIE ’98. 28th Annual (Vol. 1, pp. 14–18). https://doi.org/10.1109/FIE.1998.736790 Panadero, E., Romero, M., & Strijbos, J.-W. (2013). The impact of a rubric and friendship on peer assessment: Effects on construct validity, performance, and perceptions of fairness and comfort. Studies in Educational Evaluation, 39(4), 195–203. https://doi.org/10.1016/j.stueduc.2013.10.005 Potter, T., Englund, L., Charbonneau, J., MacLean, M. T., Newell, J., & Roll, I. (2017). ComPAIR: A new online tool using adaptive comparative judgement to support learning with peer feedback. Teaching & Learning Inquiry, 5(2), 89–113. https://doi.org/10.20343/teachlearninqu.5.2.8 Reinholz, D. L. (2015). Peer-assisted reflection: A design-based intervention for improving success in calculus. International Journal of Research in Undergraduate Mathematics Education, 1(2), 234–267. https://doi.org/10.1007/s40753-015-0005-y Reinholz, D. L. (2016). The assessment cycle: a model for learning through peer assessment. Assessment & Evaluation in Higher Education, 41(2), 301–315. https://doi.org/10.1080/02602938.2015.1008982 Reinholz, D. L., & Dounas-Frazer, D. R. (2016). Using peer feedback to promote reflection on open-ended problems. Physics Teacher, 54(6), 364–368. https://doi.org/10.1119/1.4961181 Thomas, G., Martin, D., & Pleasants, K. (2011). Using self- and peer-assessment to enhance students’ future-learning in higher education. Journal of University Teaching & Learning Practice, 8(1), 1–17. http://ro.uow.edu.au/jutlp/vol8/iss1/5/ van Zundert, M., Sluijsmans, D., & van Merriënboer, J. (2010). Effective peer assessment processes: Research findings and future directions. Learning and Instruction, 20(4), 270– 279. https://doi.org/10.1016/j.learninstruc.2009.08.004