Case Studies Compass: Journal of Learning and Teaching, Vol 8, No 12, 2016 Southampton Feedback Champions: Students and staff improving feedback together, University of Southampton Laurence Georgin, Kristina Stuart, Rebecca Wainwright, Rameesha Anwar and Abisola Tina Hammed University of Southampton Introduction This case study is about a project undertaken in 2014-2015 at the University of Southampton by Laurence Georgin (Senior Academic Coordinator and Manager of the project) and twenty- seven student ‘champions’, who took part in the project and who were selected from all eight University Faculties. The Southampton Feedback Champions project originated from a previous European project, SPEAQ (Sharing Practice in Enhancing and Assuring Quality - https://speaqproject.wordpress.com/), which aimed to develop initiatives that improve quality, practice and culture within higher education institutions. Each institution involved in SPEAQ had to identify an in-house issue and work with its staff and students to resolve it, using a bottom-up approach. Southampton staff and students identified feedback as a key issue, reflecting national and even international students’ dissatisfaction with feedback (Evans, 2013). As part of SPEAQ, the website (http://blog.soton.ac.uk/gmoof/) which was created aimed to gather useful resources to help staff and students improve feedback. These resources included interviews with key staff who had been recognised (through the Students’ Union Excellence in Teaching Award for Feedback or the National Teaching Award for Most Innovative Teacher of the Year) for delivering high-quality feedback. The project was supported by the University’s Pro Vice-Chancellor for education. The initial project received positive feedback from the European Commission, but it was clear that the topic needed more attention - a bigger project would enable the team to gather further resources. A bid was successfully put forward to the University of Southampton Education Enhancement and the new ‘Southampton Feedback Champions’ was born. This project followed the successful model of the i-Champions (Harvey, 2015) and involved students working as agents of change (“Students as agents of change,” Jisc, 2011). Indeed, with “higher student fees […] changing the relationship between institutions and students”, institutions need to pay more attention to students’ expectations and respond to them in creative ways (“Students as Agents of Change,” Jisc, 2011), Entering into a partnership with students therefore seems a productive way to involve students in the delivery of their education so that it remains as relevant as possible to their needs as future graduates looking for jobs in an ever-more-challenging jobs market. Methodology The new project built on SPEAQ and aimed to explore the creation, collection and application of feedback in a full range of contexts and through a variety of tools and mechanisms with the potential to enhance good feedback practices. Students were employed as Southampton Feedback Champions (SFC) to collect and develop further examples of good feedback practice as identified by Southampton students and staff. SFCs https://speaqproject.wordpress.com/ http://blog.soton.ac.uk/gmoof/ Case Studies Compass: Journal of Learning and Teaching, Vol 8, No 12, 2016 were at the heart of the project and took on roles as researchers, project managers, communications officers and conference organisers and speakers. They worked in collaboration with Laurence Georgin and two senior advisers, who shared (via group training, one-to-one training and mentoring) their expertise in leadership, project management, professional development and resources design. These skills are particularly important for students to develop, as they are not always covered within the curriculum. The intended outcomes of the project were: • a better understanding of the feedback-related issues; • identification of a range of solutions related to these issues; • collection of best practices from all disciplines; • additional feedback-related resources available to staff and students; • increased collaboration between students and staff on learning and teaching issues. During the activities that took place, project participants: • carried out qualitative research with over 100 staff and students; • produced a report, based on the research outcomes, which identifies the causes of feedback-related issues and offers a variety of solutions for students and staff; • collected exemplars of good practice, as identified by academics and students, including practical examples of tools and technologies which can be used to support more effective feedback practices; • developed a website showcasing the project’s findings, resources and the SFCs’ reflective blog; • developed an interactive tool to educate students about what feedback is, what their role in it is and why they should care about it; • established a Twitter account and a Facebook page promoting the project outcomes; • organised a student-led conference on sharing good feedback practice. Research: collecting examples of good practice The SFC research was conducted using an interpretative approach to data collection and analysis. Such a method was chosen because project goals required a less rigid approach to interpreting meaning and understanding. Data was collected through semi-structured interviews and used a non-probability sampling. To ensure representativeness, each faculty was represented by a team made up of: • one postgraduate (PG) student who played a supervisory role; • two undergraduate (UG) research investigators who conducted the interviews. This composition of teams allowed the research-experienced PG students to introduce research practice to the UGs and mentor them in the best practices of qualitative interviewing. Faculty field teams were responsible for a minimum of twelve interviews, with each UG research investigator responsible for interviewing: • two members of the teaching staff; • two PG students; • two UG students. Case Studies Compass: Journal of Learning and Teaching, Vol 8, No 12, 2016 The respondent population was reached largely through convenience and random sampling. In an effort to encourage participation, incentives were given to students. PGs ensured the representativeness by making certain that the UGs identified people from different academic disciplines, so that varied perspectives were captured for analysis (Flick, 2009). In preparing students for the field, Laurence Georgin organised two workshops: a leadership workshop for PG students and a data collection/ethics workshop for UG students. Two semi- structured interview guides (Staff and Students) of five essential questions were designed by students and LLAS staff and used by student researchers in the field (Flick, 2009). The open-ended construction of these questions was designed to elicit rich data on respondent opinion concerning matters of feedback procedure and delivery (Silverman, 2001). Moreover, this open-ended construction in most cases led to follow-up questions which captured a wealth of data. Interviews were tape-recorded with respondents’ permission and subsequently transcribed (Flick, 2009). The raw data was uploaded to a spreadsheet, with separate worksheets for student and staff data. Spreadsheet® allowed for easy manipulation of the raw data and thematic analysis was then done in three phases: 1. by respondent group: student or staff; 2. by faculty; 3. per individual question. In the first phase of analysis, the data was analysed by drawing out codes related to each faculty in an effort not to lose any nuanced information from an initial general analysis. For example, students within the medical faculties had required different delivery of feedback from that required by social scientists. Following the identification of codes by faculty and question, the data analysis team analysed the results for manifest themes which painted a picture of the efficacy of existing policies or highlighted the disconnect between student and staff need (Braun and Clarke, 2006). Secondly, using the identification of the themes, the analysis team compared faculty-specific staff and student responses for nuanced information on faculty feedback practice and policy. Thirdly, staff and student themes generally were analysed separately to monitor trends across campus and all results reported (Flick, 2009). 1 List the different ways in which you give feedback 2 What do you think characterises good feedback? And why? 3 What do you expect students to do with the feedback you give them? 4 Do you have an example of a good feedback technique you would like to share? (either your own or others’) 5 Do you have any suggestions on how feedback practice can be improved? Staff questions 1 List the different ways in which you receive feedback 2 What do you think characterises good feedback? And why? 3 What do you do with the feedback? 4 Do you have an example of a good feedback technique you came across? And who used it? 5 Do you have any suggestions on how feedback practice can be improved? Student questions Case Studies Compass: Journal of Learning and Teaching, Vol 8, No 12, 2016 The analysis was carried out by a small group of students (PGs and UGs) and was translated into an initial report, which highlighted the following points:  Need for more timely feedback: feedback on one assignment needs to be delivered before submission of the next. Once the module has been ‘passed’, students perceive feedback comments as worthless for other modules, which might explain why they do not always collect their transcripts.  Fear of engaging with feedback: lack of staff training on how to provide feedback, together with a jaded attitude, often translates into students’ not feeling encouraged to seek feedback, not feeling at ease to discuss it and fearing humiliation.  Lack of feedback ‘education’ for students: students are often unaware of the different ways in which feedback is delivered to them, viewing it as merely the written comments on their cover sheet. They also rarely view feedback as a two-way process in which they have a role to play.  Lack of staff training: it is felt that there is a lack of training for staff, especially new staff, PhD students acting as markers and staff coming from industry. This may result in non-effective feedback being delivered to students and potentially a lack of consistency in marking.  Lack of consistency in the delivery of feedback: feedback delivery is very faculty- specific, with some faculties having specified formal practices based on students’ needs and module construction. There is a need for a University-wide feedback culture, which shares good practice and promotes consistency in feedback delivery. The report, supported by references in the field of feedback and assessment (Gibbs, 2015; Price et al., 2008; Sambell, 2011), was then used as a basis for the project’s conclusions and recommendations. These findings may be accessed at http://blog.soton.ac.uk/feedbackchampions/what-is-it/ and are summarised below: Fostering a supportive feedback culture The project bel i eves that i n order to i mprove feedback at the Uni vers i ty of Southampton, we need to fos ter a s upporti ve feedback cul ture, i n whi ch s taff and s tudents recogni s e what effecti ve feedback i s , they feel s afe and comfortabl e to di s cus s i t and they engage equal l y i n the proces s i n order to maxi mi s e thei r potenti al . Thi s cul ture i s bas ed on three mai n pri nci pl es : 1. Education for all Educate s tudents and s taff about effecti ve feedback, the di fferent ways i n whi ch they recei ve i t and thei r rol e wi thi n i t 2. Safe communication Promote a s afe and comfortabl e envi ronment whi ch nurtures s tudents ’ agency and encourages them to engage wi th s taff i nto a feedback di al ogue 3. Active engagement Encourage s tudents to take control of thei r l earni ng and s taff to enabl e them to do s o, s o that both can engage i n the feedback proces s and make i t more effecti ve Encouraging Assessment for Learning (AfL) As wel l as recommendi ng to fos ter a s upporti ve feedback cul ture to i mprove feedback at Uni vers i ty, the project encourages the Uni vers i ty to adopt an as s es s ment approach bas ed on As s es s ment for Learni ng, where feedback i s key and whi ch vi ews as s es s ment as a way to ‘hel p s tudents i mprove thei r l earni ng’ and enabl e them to become i ndependent l earners for l i fe, and not jus t as a meas uri ng tool whi ch was often referred to i n a negati ve way i n the i ntervi ews carri ed out as part of the project. Ongoing Evaluation Fi nal l y, for thi s feedback cul ture to remai n acti ve and adapted to the needs of i ts members , i t needs to be revi ewed and eval uated regul arl y. Thi s can be done vi a mi d-term or end-of-term eval uati on forms , s tudent-s taff commi ttees or the Nati onal Student Survey (NSS), whi ch are al l ways to refl ect on teachi ng practi ces and engage s tudents i n the proces s . Staff and s tudents al i ke are not maki ng the mos t of thes e opportuni ti es and s houl d be encouraged to do s o. http://blog.soton.ac.uk/feedbackchampions/what-is-it/ Case Studies Compass: Journal of Learning and Teaching, Vol 8, No 12, 2016 Dissemination: translating the report into different messages for different audiences In order to disseminate the project’s findings, the SFCs engaged in a wide range of activities in which the project’s key messages were adapted to the various audiences it addressed. These are described below. Website and social media The website (http://blog.soton.ac.uk/feedbackchampions/) was created by a student intern as a tool to share good feedback practice across the University and promote a holistic view of feedback as a continuous dialogue between all members of the University community. It was complemented by a social media presence on YouTube, Facebook and Twitter. Including as it does a wide range of best feedback practice, tools and resources for staff and students, it constitutes a central place for all teaching staff, students and professional services to engage with feedback, share good practice and keep the feedback dialogue open. As it proved challenging to find a suitable format for both staff and students, the website underwent various transitions. Staff and student events The SUSU stall The stall allowed for a line of direct conversation with students, as SFCs asked them for their opinion on what makes for excellent feedback and how they could pledge to improve their role in the feedback process. Students were almost all shy initially, but the Champions’ coaxing, with examples of poor or useful feedback they themselves had received, soon produced many strong opinions about exactly what constitutes fair feedback. The Biological Sciences drop-in lunch event SFCs also helped to co-host staff events within departments, such as the Biological Sciences drop-in lunch event, which attracted twenty-one members of staff. The faculty was presented with the project findings: since the event was both stimulating and illustrative, points of discussion were raised by many. Some senior lecturers were greatly interested by the project’s findings. The National Student Survey feedback scores for Biological Sciences had been somewhat disappointing in the previous years, leading to an overall decline in the department’s national placing, despite the excellent facilities and teaching quality it offers. With the tweaking of the feedback process over the years and the SFCs’ suggestions on potential ways to improve current methods, there ensued a lively debate about the future direction the department should take in improving feedback methods. As direct result, the Director of Programmes congratulated the members of the project team for the success of the event and informed them of a reinforced feedback policy within the Biological Sciences department. Additionally, lecturers within the department later said that the event had been highly stimulating and popular amongst staff, with the final number in attendance at the event being much higher than average. http://blog.soton.ac.uk/feedbackchampions/ Case Studies Compass: Journal of Learning and Teaching, Vol 8, No 12, 2016 Conference The conference was well attended, with around seventy participants, including staff and students, even though there had been a worry that students would not be interested in spending a day talking about feedback. Keynote speakers included key names in the field of assessment and feedback, such as Tansy Jessops from the University of Winchester, as well as senior education managers at the University and, of course, the Feedback Champions, who planned the day, delivered a talk and managed the afternoon group discussions with all the delegates. As students attending the conference, SFCs were pleasantly surprised and quite impressed. Furthermore, as part of the organising team, they were really proud to be there because it had been laid out in a professional manner and many staff members were present; it was pleasantly reassuring to see that the staff cared about improving feedback practices, despite what students might sometimes think. SFCs were also very proud to have been members of the team that made the conference happen. In the afternoon group discussions, five teams of two feedback champions worked with groups of staff and students on each table and talked honestly about feedback. It might have been the first time that many staff members really understood what sometimes infuriates students about feedback. For students, it was definitely the first time they realised that staff too had problems with feedback. The staff seemed interested and openly pursued ideas about finding new and more interactive ways of communicating with students. It was enlightening and it gave a great platform for honest communication between the two bodies. As a result, the conference was a success. One student participant commented: “The conference was amazing. I’m a student and it actually helped me get more insight of what to expect from my feedback!” Video To further publicise the SFCs’ project, a video was compiled by three of the student champions. It was created in collaboration with the University’s Institute for Learning Innovation and Development (ILIaD). The aim was to create a vibrant visual resource to increase awareness of the project amongst University staff and students, as well as amongst a wider audience. Before making the video, the three students worked together to come up with an effective script that would, within the set time limit, portray the numerous aspects of the project; they brainstormed ideas on keeping the video dynamic to sustain viewers’ interest. This proved to be a challenge, as all three had limited experience in script-writing or video-making, but, with a little guidance from media staff in ILIaD, the task was soon accomplished. After introducing the project and providing a brief background to the Southampton Feedback Champions initiative, the team focused on defining what good feedback is and highlighted the key findings from the data collection process, in the hope of making a positive change to feedback exchange within the university. They also included future events and outlined resources available for both staff and students to enrich their educational experience. The video can be found at https://www.youtube.com/watch?v=PWln6alkBjE. https://www.youtube.com/watch?v=PWln6alkBjE Case Studies Compass: Journal of Learning and Teaching, Vol 8, No 12, 2016 For SFCs, working on this video project, being able to voice students’ reflections on the feedback process and communicating those to members of staff, the very people responsible for providing the feedback, was an invaluable experience. It enabled them to work as change agents (Dunne et al, 2011; Healey, 2012), allowing them to voice opinions and views, not only their own, but those too of fellow students, in the hope of bringing about a constructive change in the feedback-giving and -receiving culture within the university. Interactive tool for students using Articulate Storyline Over the summer, and again in collaboration with a Multimedia Developer from ILIaD, a Southampton EXCEL Placement intern produced the final project resource. This new interactive tool is designed for students to get to grips with feedback at university and is called ‘FEEDBACK FOR SUCCESS – Everything you need to know about feedback’. The tool is available at http://www.edshare.soton.ac.uk/14842/1/story.html and includes the following sections: • Why students should bother about feedback • The different types of feedback • Understanding difficult feedback • Using feedback to improve your grades • Academic appeals explained • Using peer feedback • Further resources The tool is intended to inform students about feedback in an easily accessible format and in one place, giving them the ability to improve their learning experience. The future: the challenge of embedding good practice To conclude, the outcomes stated at the beginning of this paper have been achieved. The project has produced a better understanding of the reasons why feedback was not always working for our students and we have gathered many examples of good practice aiming to resolve the issue. However, embedding good practice across the whole institution will be challenging. The bottom-up approach, on which the project was based, was a fantastic way to identify good practice across the institution, but it has its limitations and needs to receive an institutional push to support the culture change it suggests if it is to go beyond being merely a repository of University-wide good practice initiated by passionate educators. Thankfully, the Pro Vice-Chancellor for education has tasked a group of senior people to start a reflection on assessment, in order to identify ways to transform assessment and, particularly, its frequency, as a means of improving feedback as a result. Additionally, a new professor specialising in assessment and feedback was recently appointed to the School of Education. They are carrying on the conversation left open by the SFC project and are organising a series of discussions about feedback and assessment, one of which will showcase a small-scale trial carried out as a result of the SFC project. The trial was done in collaboration with a lecturer and aimed at implementing peer feedback and self-assessment as part of an existing module. The theme was chosen specifically as it had been highlighted and recommended as good practice by the SFC project. It was a success with students who took part and they gave such extremely positive feedback as: http://www.edshare.soton.ac.uk/14842/1/story.html Case Studies Compass: Journal of Learning and Teaching, Vol 8, No 12, 2016 “Peer feedback was really useful! I wish we did it more in other modules.” “Having to comment on other students’ work made us really integrate the assessment criteria which was useful for our own work.” “we received guidance on how to give feedback to peers so it was not as daunting as it could have been.” “Self-assessment (appraisal-type marking) was useful as you know what you’re going to be marked on so you know what to focus on. Of course it helps to have a look at the criteria in advance because if you comment on them at the last minute, you realise everything you haven’t done but by that time it’s too late.” These comments show that the project has started having a direct impact on teaching practice and student satisfaction and that things are beginning to change, with concrete actions being taken in order to improve feedback across the University, even if the process may be slow. However, as Trowler et al (2003) suggest, ‘cultural change takes time’ and, although the SFC project has not achieved a complete cultural change, it has moved in the right direction and sent ripples across the institution, which might in time result in the culture change which was hoped for at the start of the initiative. Nonetheless, the project gave a fantastic opportunity to many students to be involved in institutional change and to have a direct impact on their university’s practices. For many of them, it was the first time that they could go beyond voicing their opinion and become agents of change, while simultaneously discovering what happens ‘behind the scenes’ of educational practices. Through their experience as champions, many of them felt more empowered. They also recognised that the experience had contributed to making them more employable, something which, in today’s increasingly competitive jobs market, is undeniably an advantage for students. Finally, the project was recently highlighted by the Pro Vice- Chancellor for education in the Times Higher Education (Havergal, 2015), as an example of good practice within the University. This is very encouraging, as it shows that the work carried out by the SFCs has been valued at the highest level of their institution. Reference list Braun, V. and Clarke, V. (2006) ‘Using thematic analysis in psychology.’ Qualitative Research in Psychology, 3(2), 77-101. Dunne, E., Zandstra, R., Brown, T. and Nurser, T. (2011) Students as change agents: New ways of engaging with learning and teaching in Higher Education. Available at: http://escalate.ac.uk/downloads/8242.pdf (Accessed: 22 April 2016). Evans, C. (2013) ‘Making sense of assessment feedback in higher education.’ Review of Educational Research, 83, 70-120. Flick, U. (2009) An Introduction to Qualitative Research. Los Angeles, CA: SAGE Publications. http://escalate.ac.uk/downloads/8242.pdf Case Studies Compass: Journal of Learning and Teaching, Vol 8, No 12, 2016 Gibbs, G. (2015) Making feedback on assignments effective: principles and guidance for tutors. Available at: http://www.testa.ac.uk/index.php/resources/best-practice- guides/category/7-best-practice-guides# (Accessed: 22 April 2016). Harvey, F. (2015) iChamps at the University of Southampton. Available at: https://digitalstudent.jiscinvolve.org/wp/files/2015/02/DS38-iChamps-at-the-University-of- Southampton.pdf (Accessed: 30 October 2015). Havergal, C. (2015) ‘Should Students Be Partners in Curriculum Design?’ Times Higher Education (THE), 17 December 2015. Available at: https://www.timeshighereducation.com/features/should-students-be-partners-in-curriculum- design. (Accessed: 7 February 2016). Healey, M. (2012) ‘Students as change agents.’ In: International Society for the Scholarship of Teaching and Learning Conference. Available at: http://www.mickhealey.co.uk/wp- content/uploads/2012/10/Students-as-Change-Agents-Handout.pdf (Accessed: 30 October 2015). Jisc. (2011) Students as agents of change. Available at: https://www.jisc.ac.uk/news/students-as-agents-of-change-29-jun-2011 (Accessed: 30 October 2015). Price, M., O’Donovan, B., Rust, C. and Carroll, J. (2008) ‘Assessment Standards: A Manifesto for Change.’ BROOKES eJOURNAL OF LEARNING AND TEACHING. Available at: http://bejlt.brookes.ac.uk/paper/assessment_standards_a_manifesto_for_change-2/ (Accessed: 16 July 2015). Sambell, K. (2011) Rethinking feedback in higher education: an assessment for learning perspective. Bristol: HEA Subject Centre for Education, University of Bristol. Trowler, P., Saunders, M. and Knight, P. (2003) ‘Change Thinking, Change Practices: A Guide to Change for Heads of Department, Programme Leaders and Other Change Agents in Higher Education.’ Available at: https://www.heacademy.ac.uk/sites/default/files/id262_change_thinking_change_practices.p df (Accessed: 9 February 2016). http://www.testa.ac.uk/index.php/resources/best-practice-guides/category/7-best-practice-guides http://www.testa.ac.uk/index.php/resources/best-practice-guides/category/7-best-practice-guides https://digitalstudent.jiscinvolve.org/wp/files/2015/02/DS38-iChamps-at-the-University-of-Southampton.pdf https://digitalstudent.jiscinvolve.org/wp/files/2015/02/DS38-iChamps-at-the-University-of-Southampton.pdf https://www.timeshighereducation.com/features/should-students-be-partners-in-curriculum-design https://www.timeshighereducation.com/features/should-students-be-partners-in-curriculum-design http://www.mickhealey.co.uk/wp-content/uploads/2012/10/Students-as-Change-Agents-Handout.pdf http://www.mickhealey.co.uk/wp-content/uploads/2012/10/Students-as-Change-Agents-Handout.pdf https://www.jisc.ac.uk/news/students-as-agents-of-change-29-jun-2011 http://bejlt.brookes.ac.uk/paper/assessment_standards_a_manifesto_for_change-2/ https://www.heacademy.ac.uk/sites/default/files/id262_change_thinking_change_practices.pdf https://www.heacademy.ac.uk/sites/default/files/id262_change_thinking_change_practices.pdf