Opinion Pieces Compass: The Journal of Learning and Teaching at the University of Greenwich, Issue 7, 2013 1 Look, Listen & Learn! Do students actually look at and/or listen to online feedback? Dr Ray Stoneham and Matt Prichard, School of Computing & Mathematical Sciences, University of Greenwich The National Student Survey and other student surveys regularly highlight student dissatisfaction with the nature and timescale of the provision of feedback (NUS 2010; National Student Forum, Annual Report 2009; Thompson & Meredith 2012). To address this issue the Managed Learning Environment at the School of Computing and Mathematical Sciences at the University of Greenwich (Stoneham 2012) has facilities for lecturers to provide online feedback through a range of media including text, annotated documents, audio and screencasts. Many lecturers have taken advantage of this to give timely in-depth feedback to students, and this has involved considerable effort and time on their part. A crucial question is whether students actually engage with this process by reading, viewing or listening to the feedback, and whether this makes a difference to their grades. This opinion piece presents evidence of the level of engagement as measured by whether students actually access the feedback, how quickly they do it, how often, and whether the level of engagement correlates with the grades they achieved. It does not address the issue of whether they actually read, view or listen to it after they access it, which we plan to investigate in future research. Previous studies in this area have established through interviews that alternative methods of feedback can be very effective, but most studies have been quite small scale and focused on just audio (Nortcliffe & Middleton 2007; Lunt & Curran 2010; King, McGugan & Bunyan 2008), although video and screencasts (Stannard 2007; Brick & Holmes (2008) are becoming more widely used. The system used for this research records when the student accesses their feedback (if at all), and how often. It was introduced part way through the 2011/12 academic year and the quantity of evidence was substantial. 3554 feedback files were uploaded by 37 staff in the academic year using 5 file types, namely Word, PDF, MP3, Shockwave (screencast generated by the free Jing recording software, Jing 2012) and ZIP (of MP3 and Word files together). Of these around 2000 feedback files were released to students after the monitoring system was in place. An in-house storage system for the feedback was chosen. Feedback is personal data as defined by the Data Protection Act (Data Protection Act 1998). Cloud-based storage (e.g. JING, DropBox, Google Docs, Vimeo) has issues due to uncertainty of where it is physically stored, who has access to it, its security, and the period over which it is stored. There is also no guarantee of long-term access to the feedback. In-house storage, on the other hand, is not expensive, is stored in one location and is easy to manage and use for monitoring purposes. It also allows the production of an integrated profile of each student across all their courses, indicating the level of engagement with their programme of study, and helping to identify students at risk of failing. Opinion Pieces Compass: The Journal of Learning and Teaching at the University of Greenwich, Issue 7, 2013 2 The results of this study can be summarised as follows. Roughly a quarter of students failed to access their feedback by the end of the assessment period. The average was 1.9 times, with one student accessing it 18 times, 4% accessing it more than 5 times and 45% accessing it more than once. The quickest access was just over 5 minutes after it was released online, with the average being 27 hours and with 66% of students accessing it within a day of it being released. The average grade for students who did not access the feedback was 58.6%, whereas students who did access their feedback gained an average grade of 66.6%, eight percentage points higher. A surprising result was that MP3 files were significantly less likely to be accessed than other file types (40% compared to around 70% for the other types). This may be because audio feedback needs to be accessed serially whereas the other feedback types are easily scanned and navigated through, or it may be that students prefer visual feedback. This is something the authors plan to investigate over the next academic year by interviewing students and correlating the results with the data from monitoring their accesses to the feedback. The findings of our research are 1. students do generally access online feedback 2. students access it as soon as it is released in most cases 3. MP3 files are not popular with students as a feedback medium 4. Students with better grades are more likely to access their feedback. We also conclude that in-house storage is better than cloud-based storage, not least because it facilitates research of the nature presented in this paper. Most importantly, more research is needed to establish the best methods to provide feedback to students to make the most effective use of valuable staff time to provide the appropriate feedback that students are currently missing, as evidenced by student satisfaction surveys. References Brick, B. and J. Holmes (2008) ‘Using Screen Capture Software for Student Feedback' in Klinshuk, D., G.,Sampson, J.M. Spector, P. Isaias and D. Ifenthaler (eds.) Cognition and Exploratory Leaning in Digital Age: Proceedings of the IADIS CELDA 2008 Conference, Freiburg, Germany: pp. 339-342. Available from http://www.iadis.net/dl/final_uploads/200818C046.pdf (Accessed 13 September 2012). Data Protection Act (1998). Availble from: http://www.legislation.gov.uk/ukpga/1998/29/contents (Accessed 13 September 2012). King, D., McGugan, S. & Bunyan, N. (2008): ‘Does it make a difference? Replacing text with audio feedback.’ Practice and Evidence of Scholarship of Teaching and Learning in Higher Education, 3(2),145-163 Jing (2012). Available at: http://www.techsmith.com/jing.html (Accessed 1 July 2012). http://www.legislation.gov.uk/ukpga/1998/29/contents http://www.techsmith.com/jing.html Opinion Pieces Compass: The Journal of Learning and Teaching at the University of Greenwich, Issue 7, 2013 3 Lunt, T.; & Curran, J. (2010): ‘Are you listening please? The advantages of electronic audio feedback compared to written feedback’, Assessment & Evaluation in Higher Education, 35(7), 759-769. National Student Forum, Annual Report (2009). Available at: http://www.bis.gov.uk/assets/biscore/higher-education/docs/n/09-p83-national-studentforum- annual-report-09 (Accessed 18 April 2012]. Nortcliffe, A. and Middleton, A. (2007) ‘Audio Feedback for the iPod Generation’ International Conference on Engineering Education – ICEE, Portugal. NUS (2010). Charter on Feedback and Assessment. Available at http://www.nusconnect.org.uk/asset/news/6010/FeedbackCharter-toview.pdf (Accessed 13 Sept 2012). Stannard, R. (2007), ‘Using Screen Capture Software in Student Feedback: A case study from the HEA English Subject Centre’. Available at: www.english.heacademy.ac.uk/explore/publications/casestudies/technology/camtasia.php (Last accessed 26 March 2012). Stoneham, R. (2012): ‘Managed Learning Environments in Universities: Are they Achievable?’, Compass, the Teaching and Learning Journal of the University of Greenwich, 6, 45-54. Thompson, R. and Lee, L. (2012) ‘Talking with Students through Screencasting: Experimentations with Video Feedback to Improve Student Learning”, The Journal of Interactive Technology and Pedagogy, 1, Available at: http://jitp.commons.gc.cuny.edu/2012/talking-with-students-through-screencasting- experimentations-with-video-feedback-to-improve-student-learning/ (Accessed on 13 September 2012). Author Biographies Dr Ray Stoneham is a principal lecturer at the University of Greenwich and a specialist in data-driven web sites, with a particular interest in intranets and e-learning. He developed and manages several School intranets and the website used by external examiners of the university. Matt Prichard is a senior lecturer in the Creative Digital Technologies Department and course coordinator for web technologies. His research and scholarly interests include, e- Learning/e-Assessment, web technologies, HTML 5, kinesthetic learning, digital audio production and scripting for novices.