Literature Review Journal of Learning Development in Higher Education ISSN: 1759-667X Issue 5: March 2013 ‘Re-formative’: Assessing the Value of Digitally Recording Formative Feedback Elizabeth Gaston University of Leeds, UK Abstract This project investigates the value of digitally recording weekly formative feedback tutorials and then e-mailing the recording to the student. It aims to address differences in perception between the students’ and the lecturers’ interpretation of feedback, and improve the students’ understanding of formative feedback delivered during weekly tutorials. Keywords: formative feedback; digital recording; student tutorials; art and design; added value. Introduction This project investigates the value of digitally recording weekly formative feedback tutorials and then e-mailing the recording to the student. Formative feedback is ‘information communicated to the learner that is intended to modify his or her thinking or behaviour to improve learning…if delivered correctly’ (Shute, 2008, p.154). Detailed and developmental formative feedback is one of the most useful things we do for our students (Brown, 2007). There is much evidence to support the use of providing audio summative feedback after assessment (Rotherham, 2009; Chiang, 2010), however, there has been less investigation into the impact of providing weekly formative feedback in this format. Gaston ‘Re-formative’: Assessing the Value of Digitally Recording Formative Feedback Journal of Learning Development in Higher Education, Issue 5: March 2013 2 Rationale The project involved level two students on the knitted fabric design pathway of the BA (Hons.) Textile Design programme at the University of Leeds. A feature of textile design programmes nationally is active learning through project based enquiry (Kennedy and Welch, 2008), supported by verbal tutor feedback delivered during individual and group tutorials. This nationally accepted model is labour-intensive, however, Devas found that small regular formative feedback is more useful than summative feedback reports (Devas, 2001). Textile design students at the University of Leeds receive weekly verbal feedback on their practical work. Despite this investment of time, many students complain that they do not receive enough feedback on their work as evidenced by the National Student Survey (Bright, 2010; Gladstone, 2010). Brooks found that there was a discrepancy between the students’ and the lecturers’ interpretation of feedback (Brooks, 2008), and students fail to recognise feedback that is not written (Bright, 2010). It has been noted (informally) that most students do not record the feedback they are given. Tutors feel that they often repeat the information given during weekly tutorials several times before it is acted upon, as students can only recall a small proportion of the tutorial content. It was hoped that recording tutorials and e-mailing the recording to students with the subject ‘feedback’ would signpost the support that they already receive and eliminate the need for either tutor or student to take notes, so that both could concentrate fully on the conversation. A negative aspect of classroom tutorials is the anxiety that they can cause to students which may reduce their effectiveness (Blair, 2007). One-to-one tutorials can cause less emotional vulnerability; however, normative feedback is important as students need to learn critical awareness of their own and their peers’ work (Shute, 2008). Feedback delivered during group tutorials helps students to develop the same evaluative skills as their teachers and peers (Juwah et al., 2004). Digital audio feedback allows control over how, when and where re-listening to feedback takes place, which enables students to receive feedback in an emotionally ready state (Hepplestone et al., 2010). This could increase the effectiveness of the feedback and negate the negative aspects of group tutorials caused through stress. Positive feedback can reduce anxiety about a student’s own performance and reduce the cognitive load of the student (Shute, 2008); however, verbal feedback is not always Gaston ‘Re-formative’: Assessing the Value of Digitally Recording Formative Feedback Journal of Learning Development in Higher Education, Issue 5: March 2013 3 retained unless it confirms the students’ views on their performance (Brown, 2007). It was anticipated that allowing the students to review their feedback in detail would reduce the effect of glossing over any negative comments, and produce a more honest self- assessment of achievements. Complexity of feedback is inversely related to learning efficiency (Juwah et al., 2004; Shute, 2008). As audio feedback can be more detailed than written feedback (Merry and Orsmond, 2007; Chiang, 2010) it is important to specify a limited number of aspects for students to work towards and not every imperfection (Hugh-Jones et al., 2008; Beverley, 2010). Allowing students to listen to their feedback again could overcome the problem of feedback overload. Methodology In the first class of the semester, the aims and methodology of the feedback project was explained to the students. They were then given a short questionnaire to determine their current views on feedback and tutorials. The questionnaire was designed mainly using scaled responses to allow ease of answering, but also to give wider, meaningful variance. Weekly group and individual tutorials were recorded on a digital Dictaphone in MP3 format so that they could be e-mailed easily to students and listened to on a variety of devices (Chiang, 2010). The recordings were made on the highest quality setting to counteract background noise; this however resulted in large file sizes. Each student’s feedback was recorded as a separate file and then e-mailed to the student as an attachment. During the trial, students were not reminded to listen to their feedback and were allowed to make their own decisions on the use of the recorded information, as they had already received their feedback during the classroom tutorial. At the end of the trial, a second questionnaire was delivered that concentrated on whether the recording was used and if it was useful. Gaston ‘Re-formative’: Assessing the Value of Digitally Recording Formative Feedback Journal of Learning Development in Higher Education, Issue 5: March 2013 4 Results and analysis Survey one Contrary to information gathered during module reviews, all students recognised their traditional tutorials as feedback and 86% of students agreed that they received enough feedback. Contrary to tutor experience, all of the students claimed to make notes during their weekly feedback tutorials; however, 57% of students indicated that they did not always act on their feedback. This could signify that the student did not remember every aspect of the feedback given to them or the feedback given was not useful. Survey two Pleasingly all students responded very positively about the effects of listening to their feedback again. However, the lack of any negative feedback could have been due to students wanting to give the right answers in the project evaluation questionnaire. All of the students found it very helpful to listen to their feedback again and stated that it improved their understanding of the feedback. They all stated that when listening to their feedback again, they had noticed ideas that they had missed during their class tutorial and that they had changed aspects of their project due to listening to feedback again. One student said: I find the recordings really useful, as it is a great way to listen to feedback, and remember what I had expressed and wanted to achieve from the last lesson. Another stated: It is so useful and helped me enormously when trying to recall the stages that I took in my work. All students stated that they had listened to some of the recordings more than once. Despite initial reservations about being recorded, which were allayed by reassurances that no one else would hear the recording, all students wanted to continue to receive a digital copy of their class tutorial feedback. Student comments include: Gaston ‘Re-formative’: Assessing the Value of Digitally Recording Formative Feedback Journal of Learning Development in Higher Education, Issue 5: March 2013 5 I felt more engaged with this module and found the feedback really helpful. …found it much easier getting feedback in this way, rather than trying to make written notes during feedback in class. In a comparison of the results from both surveys a change in behaviour was indicated. Whilst only 29% of students always made notes during traditional tutorials and revisited them at a later date, 83% of students always listened to their feedback again, showing a marked improvement in the number of students who review their feedback after the tutorial. The second major positive impact of the project was whilst only 43% of students always acted on the feedback given in their tutorial during traditional tutorials, 100% of students stated that re-listening to their feedback improved understanding of feedback and improved the detail recalled from the tutorial, which resulted in their project developing positively. One student said: I sometimes feel slightly lost or stuck with an idea and find listening to the recording again re-focuses me and leads me onto another idea. All students found that the technology used was appropriate and that it was either very easy or quite easy to use. The recording was audible in all cases and no students complained of background noise. The main technical problem stated was that file sizes were large. Colleagues have commented informally that the students involved in the trial have discussed the trial with them and have been very positive about the benefits. Initial tutor reservations regarding added workload were assuaged through the production of a practical guide to best practice in sending the recordings using the university’s administration system, and practical training demonstrating ease and speed. Gaston ‘Re-formative’: Assessing the Value of Digitally Recording Formative Feedback Journal of Learning Development in Higher Education, Issue 5: March 2013 6 Further work To improve access to recordings it would be beneficial for students to arrive at a file size that could be accessed from a mobile phone. The Sounds Good project found that when students had to take more than one step to access files, they didn’t bother (Rotherham, 2009). However, there needs to be a balance between file size and recording quality (Rotherham, 2009). Additional benefits Students have difficulty articulating the concepts and influences of their work in writing, however, during their tutorials many students speak fluently and intelligently about their work. The recordings allowed students to use this commentary to form the basis of their critical journal. Conclusion The success of the project exceeded expectations. Freeing students from the responsibility of recording tutorials and facilitating re-listening to tutorials has been shown to enrich feedback and improve effectiveness. It has also been shown to improve student engagement with their learning. The use of audio recording of tutorials enhances student experience and offers added value to the students with little investment of either financial costs or staff time. Students also value extra staff efforts. The final student comment sums up the project…‘Just fab!!!’. Gaston ‘Re-formative’: Assessing the Value of Digitally Recording Formative Feedback Journal of Learning Development in Higher Education, Issue 5: March 2013 7 References Beverley, K. (2010) School of Design feedback reminders. Internal memorandum. Leeds: University of Leeds. Blair, B. (2007) ‘Perception/interpretation/impact’, Networks (Art Design Media Subject Centre), 1, pp. 10-13. Available at: http://www.adm.heacademy.ac.uk/resources/publications/networks-magazine (Accessed: 14 March 2013). Bright, K. (2010) Providing individual written feedback on formative and summative assessments. The Centre for Legal Education. Available at: www.ukcle.ac.uk/resources/assessment-and-feedback/effectivefeedback (Accessed: 14 March 2013). Brooks, K. (2008) ‘Could do better? Students’ critique of written feedback’, Networks (Art Design Media Subject Centre), 5, p.28 Available at: http://www.adm.heacademy.ac.uk/resources/features/could-do-better-students- critique-of-written-feedback (Accessed: 14 March 2013). Brown, S. (2007) Feedback and feed-forward. Centre for Bioscience, The Higher Education Academy. Available at: http://www.bioscience.heacademy.ac.uk/ftp/newsletters/bulletin22.pdf (Accessed: 24 January 2013). Chiang, I. (2010) Optimising audio feedback assisted learning for student and staff experience (AFAL final report). Available at: http://www.jisc.ac.uk/whatwedo/programmes/elearning/ltig/afal.aspx (Accessed: 14 March 2013). Devas, A. (2001) Using formative assessment in the delivery of a level one module for a media arts degree. Available at: http://www.adm.heacademy.ac.uk/resources/case- studies/using-formative-assessment-in-the-delivery-of-a-level-one-module-for-a- media-arts-degree (Accessed: 14 March 2013). http://www.adm.heacademy.ac.uk/resources/publications/networks-magazine http://www.ukcle.ac.uk/resources/assessment-and-feedback/effectivefeedback http://www.adm.heacademy.ac.uk/resources/features/could-do-better-students-critique-of-written-feedback http://www.adm.heacademy.ac.uk/resources/features/could-do-better-students-critique-of-written-feedback http://www.bioscience.heacademy.ac.uk/ftp/newsletters/bulletin22.pdf http://www.jisc.ac.uk/whatwedo/programmes/elearning/ltig/afal.aspx http://www.adm.heacademy.ac.uk/resources/case-studies/using-formative-assessment-in-the-delivery-of-a-level-one-module-for-a-media-arts-degree http://www.adm.heacademy.ac.uk/resources/case-studies/using-formative-assessment-in-the-delivery-of-a-level-one-module-for-a-media-arts-degree http://www.adm.heacademy.ac.uk/resources/case-studies/using-formative-assessment-in-the-delivery-of-a-level-one-module-for-a-media-arts-degree Gaston ‘Re-formative’: Assessing the Value of Digitally Recording Formative Feedback Journal of Learning Development in Higher Education, Issue 5: March 2013 8 Gladstone, M. (2010) Student expectations on academic feedback. Internal document. Leeds: Leeds University Union. Hepplestone, S., Parkin, H., Irwin, B., Holden G., Thorpe, L., and Burn, C. (2010) A best practice guide for academic staff, Sheffield Hallam University. Sheffield: Sheffield Hallam University. Available at: http://www.enhancingfeedback.ed.ac.uk/documents/guide+for+academic+staff+FIN AL.pdf (Accessed: 14 March 2013). Hugh-Jones, S. and Waterman, M. (2008) ‘Assessment and feedback priorities: focus on effective feedback’, Making Assessment Relationships Known (M.A.R.K.) Project. Leeds: University of Leeds. Juwah, C., Macfarlane-Dick, D., Matthew, B.,Nicol, D., Ross, D., and Smith, B., (2004) Enhancing student learning through effective formative feedback. The Higher Education Academy. Available at: http://www.heacademy.ac.uk/assets/documents/resources/database/id353_senlef_ guide.pdf (Accessed: 14 March 2013). Kennedy, G and Welch, E. (2008) Subject benchmark statement: art and design. Gloucester: Quality Assurance Agency for Higher Education. Available at: http://www.qaa.ac.uk/Publications/InformationAndGuidance/Documents/ADHA08.p df (Accessed: 14 March 2013). Merry, S. and Orsmond, P. (2007) Feedback via MP3 audio files. Centre for Bioscience, The Higher Education Academy. Available at: http://www.bioscience.heacademy.ac.uk/ftp/newsletters/bulletin22.pdf (Accessed: 14 March 2013). Rotherham, B. (2009) Sounds good: quicker, better assessment using audio feedback final report. Leeds: Leeds Metropolitan University/JISC. Shute, V. (2008) ‘Focus on formative feedback’, Review of Educational Research, 78(1), pp. 153-189. http://www.enhancingfeedback.ed.ac.uk/documents/guide+for+academic+staff+FINAL.pdf http://www.enhancingfeedback.ed.ac.uk/documents/guide+for+academic+staff+FINAL.pdf http://www.heacademy.ac.uk/assets/documents/resources/database/id353_senlef_guide.pdf http://www.heacademy.ac.uk/assets/documents/resources/database/id353_senlef_guide.pdf http://www.qaa.ac.uk/Publications/InformationAndGuidance/Documents/ADHA08.pdf http://www.qaa.ac.uk/Publications/InformationAndGuidance/Documents/ADHA08.pdf http://www.bioscience.heacademy.ac.uk/ftp/newsletters/bulletin22.pdf Gaston ‘Re-formative’: Assessing the Value of Digitally Recording Formative Feedback Journal of Learning Development in Higher Education, Issue 5: March 2013 9 Author details Elizabeth Gaston is a Senior Teaching Fellow in the School of Design at the University of Leeds, specialising in structured textile design.