Developing the Feedback Process: Promoting Teacher and Student Dialogue via the Use of Screen Capture and Audio Case studies Developing the Feedback Process: Promoting Teacher and Student Dialogue via the Use of Screen Capture and Audio Louise Carr and Howard Lee Hadlow College Introduction Hadlow College is part of the Mixed Economy Group (MEG): approximately 40 Colleges, which are seeking to provide effective support to higher education (HE) students within further education (FE) institutions. As part of this commitment we have been studying how to improve the quality of our HE formative feedback for written assignments and posters. We are conscious of existing research within higher education institutions (HEIs) to overhaul formative assessment and feedback practices: e.g. the Re-engineering Assessment Practices (REAP) project at the Universities of Glasgow and Glasgow Caledonian (see Nicol and Macfarlane- Dick, 2006). In that research seven principles of good feedback practice had been identified: (i) clarify what good performance is; (ii) facilitate self-assessment; (iii) deliver high quality feedback information; (iv) encourage teacher and peer dialogue; (v) encourage positive motivation and self-esteem; (vi) provide opportunities to close the gap, and; (vii) use feedback to improve teaching. We wanted to find a tool that would allow the use of screen capture and audio to increase feedback options for HE students at Hadlow College. Chiang (2010) has reported, by comparing feedback methods, that audio-only is least popular whilst screen capture with audio commentary is preferred for written assignments. We therefore decided to investigate the potential of Camtasia software, which has been reported positively as a screen capture/audio feedback tool by other researchers (Silva, 2012). We are reporting here on the results of a pilot study we undertook in cooperation with four expert lead lecturers across faculties at Hadlow College. Methods Hadlow College was the education centre for this study – a Grade 1 and Beacon (Ofsted, 2012) College teaching 1,159 further education (aged 16-18) and 831 higher education (degree) students in the 2011- 12 academic year. These students were studying from Level 1 (Land Based Studies) to Levels 4, 5, 6 (BSc honours) and across a wide range of curriculum areas including Floristry, Countryside Management, Agriculture, Horticulture, Animal Management, Equine, Fisheries and Sports Studies. The main purpose of the project was to investigate the creation of short videos comprising screen capture/ Compass: The Journal of Learning and Teaching at the University of Greenwich, Issue 6, 2012 audio as an additional feedback option for HE students. The intentions were to: ●● Increase accessibility; ●● Develop screen/audio based delivery/feedback into a more dynamic two way process, and; ●● Consider possible future developments. Six members of staff who taught across the HE range at Hadlow College were asked to use Camtasia as a means of formative assessment, each for a chosen cohort of students. Staff were specifically asked not to include a grade in their Camtasia responses, with the intention of encouraging students to give greater attention to the details of this feedback. Staff were supported in the development of skills needed to suc- cessfully run Camtasia and, after completion, asked to give feedback on their experiences. The process in- volved students e-submitting a range of formats: essay drafts, final essay presentations, posters, and text as part of a 1:1 support dialogue; and receiving individual, screen capture and combined audio responses as a short (approximately three minute) video. The recipient students were likewise asked to provide re- sponses to this form of e-feedback. The recipient student groups were studying: BSc honours International Agriculture (Level 6); BSc honours Animal Management (Level 6), Postgraduate Certificate in Education, Access to HE (Land Based Studies) and Additional Learning Support for HE. We then investigated the experience of this process for both staff and students, via a questionnaire supported by ‘Survey Monkey’ (Survey Monkey, 2012). Different questionnaires were used for staff and students and are shown in Tables 1 and 2, respectively. Staff made additional text comments, which are also summarised below. This project is ongoing, so results shown here are preliminary. Additionally, semi- structured interview methods are being used to help understand the experiences of staff/students and will be reported soon. Results Staff feedback for paper-based assignments normally comprises: (i) comments written directly onto a hard copy of the assignment, with additional supportive points on a front sheet; or (ii) comments written or typed on separate attached feedback sheets, e.g. for posters, when it is not appropriate to write directly on the submission. Staff that were developing Camtasia feedback in this study were all able to: (i) display each assignment or poster electronically on screen; (ii) highlight specific parts of the text or figures/ tables as the cursor scrolled downwards; (iii) simultaneously give matching audio comments focussing on the point(s) as they were covered by the cursor, and; (iv) indicate how the student might achieve improvements to secure a higher future assessment grade. In essence each student was able to download a short but dynamic video-format which emphasised: (i) any factual and/or typographical errors; (ii) any advice about how to present data more effectively; (iii) potential improvements in the critical analysis of key issues; and (iii) how external relevant scientific literature could best be used. The questionnaires sought responses from staff: the experience of developing the Camtasia videos; and students: how their personal feedback video had affected their learning environment. Participating staff were in the Faculties of Access to Higher Education, Agriculture, Animal Management and Additional Learning Support and all had prepared personal Camtasia videos for 1:1 feedback. Three staff had prepared feedback for student written assignments and one for a poster. In terms of facility of 38 Compass: The Journal of Learning and Teaching at the University of Greenwich, Issue 6, 2012 use, two staff had found the process quite easy, one had struggled but succeeded and one had found the process too difficult to complete. The duration of Camtasia videos had varied from about 5 to 7 minutes for all staff. Student responses received by staff had so far been limited by the early stage of this project. Some staff were especially concerned that Camtasia might increase the turnaround time for assignments; e.g.: “I discussed the option of having feedback in this format with my students; generally the common issue was with having to wait for feedback.” Staff experiences when using the software were rather mixed but there were some helpful text comments: “The Windows software was far too slow to use to capture feedback, the video was stutter- ing to use and then playback was equally slow.” “The Mac software version was very smooth and quick to use.” Such technical challenges are currently being investigated – especially the need to increase response times for Windows. However, the greatest issue for staff was the pedagogic challenge. Text comments included: “Screen capture and audio combined was a learning curve - I had to think carefully about what I was hoping to achieve in terms of student support.” “A lot of preparation compared to normal written method, as I had to think about exactly what I was going to say before recording. This is different to how I might have given verbal feedback in a normal feedback situation- e.g. there would be interaction and I might explore reasons for making particular statements.” Comments about further applications of Camtasia and detailed staff text responses included the use of Camtasia to assist in distance/blended learning. Several comments also covered the adoption of Camtasia to help with ‘how to’ procedures for new students: “I intend to use the software for induction in 2012/13 to show use of the Greenwich Portal and Moodle. I find that this may be a more appropriate use of the software. I also intend to use Camtasia to verbally explain assignments and then upload to Moodle.” “Possibly used to demonstrate a particular IT function e.g. contents pages in Word or how to insert references.” Compared with those from staff, student responses via the questionnaire were generally more positive: students clearly felt more engaged in the assessment process. They also thought that Camtasia feedback had been relatively more helpful for their personal study skills development and a likely greater contributor to more positive performances in subsequent assignments. They were relatively comfortable with the on screen/audio format and most found that viewing the video once or twice was usually sufficient to obtain all the guidance they needed. No additional text comments were provided by students. 39 Compass: The Journal of Learning and Teaching at the University of Greenwich, Issue 6, 2012 Discussion Feedback is now well established as key to effective student learning: e.g. see good reviews by (Clynes and Raftery, 2008) and (Hoban and Hastings, 2006). Video feedback has been investigated since 2000, when (Embregts, 2000) considered its use for self-management by special needs learners but more recently in the same context by (van Vonderen, Didden and Beeking, 2012). A study of video feedback for HE students at the University of Reading (Crook et al., 2012) found that 80% liked this mode and found it easier to understand, so tended to pay better attention to comments. Likewise, staff in the same study felt that video feedback helped them emphasise the most important aspects for student improvement: comments which tended to get lost in the detail of written responses. Video feedback has been favourably reviewed by many other studies such as (Baartman et al., 2007) (Nicol and Macfarlane-Dick, 2006) (Schartel, 2012) and (Tillema, Leenknecht and Segers, 2011). The study reported at Hadlow College received similarly positive responses from students, though for staff it is still too early in the project to report more than the cognitive challenges of adapting to this format. The key experiences that we can relate are: ●● Technical challenges: like any software package, Camtasia requires tuition and support until staff are confident. We found that there were difficulties in using computer suites for training, since all computers needed prior uploads of Camtasia due to licensing issues: so, improving the flexibility of staff access would be helpful. There are also specific technical requirements to be addressed – e.g. Explorer 8 is needed to allow Camtasia files to be opened and downloaded. Headphones are vital, as is access to quietness for recording. Finally, there is the need to manage quite large files once the feedback has been recorded, and to ensure they can easily be accessed by students. ●● Other research using Camtasia has identified some technical issues, with successful downloading being seen as critical: some students can have initial problems with downloading and may need information and computer technology (ICT) support (Silva, 2012), however many quickly become familiar with downloading Camtasia feedback onto a range of mobile devices for viewing when convenient (Reuter, 2007). The former study examined student use of Camtasia feedback and established that most viewed it within three days and enjoyed the more personable support given: ●● “Students perceived that the video commentary modality afforded a degree of clarity and representation that was not evident or as effective in written modalities. (Silva, 2012) (p.10); ●● Allow more time than planned: the development of personalised tuition via screen capture/audio is initially time consuming. The additional issue to then consider is the extra time that such feedback might require from lecturers and how this can be managed effectively. Other research on video feedback (but not specifically Camtasia) has highlighted, from staff feedback, the need for time management if this mode is to be successful (Crook et al., 2012); ●● Privacy: it is not appropriate to post Camtasia feedback files on Moodle, since there is open access within student cohorts: guaranteeing privacy between the tutor and student is vital. The process of ensuring that each student has access to only their feedback is quite complex, involving a series of steps and involving ICT support, and needs to be simplified. Privacy can also be seen as an advantage in terms of video feedback for the recipient student and this has been noted by others (Schartel, 2012) (Clynes and Raftery, 2008); 40 Compass: The Journal of Learning and Teaching at the University of Greenwich, Issue 6, 2012 41 ●● Creating a suitable protocol for the internal verification and second marking requirements of assessment that has been screen capture/audio marked; ●● Cooperation: this was felt to be the most critical for success. Teaching staff are always very busy and we were fortunate to get their willingness to invest time and effort to make the process work. The support of ICT staff was also an essential component. However this is only part of the need for a much more dynamic learning partnership between the teacher and student. Improved cognition should be the start, as Silva reports for comments by Mayer: “The synchronicity of the audio and visual output establishes temporal contiguity, reduces the effect of cognitive overload, and increases the opportunity for essential and generative processing.” (Mayer reported in Silva, 2012, p.11). Thus, this form of feedback is much more personalised (O’Day, 2008), and encourages self-regulated learning and self-assessment (Kostons, van Gog and Paas, 2012), all of which are concluded to reinforce learning. We therefore argue that Camtasia has potential to allow the learner to take greater control of their studies via self-assessment, reflection and future-planning, as part of a more dynamic staff/student approach, which is also supported by similar conclusions from a survey of medical students (Rogers et al., 2012). Specifically we are impressed by the seven principles listed by (Nicol and Macfarlane-Dick, 2006) – see introduction of this paper. We are focussing especially on principles 1, 2 and 3. Principle 1 (clarify what good performance is) is a key part of our assessment procedures, with students requested to self-assess before marking by the tutor. Our feedback is then designed to help the student understand how any misconceptions on their part about performance can be positively forward-cast into future work. Principle 2 (facilitate self-assessment) we believe to be improved by Camtasia, and we are currently studying this. Principle 3 (deliver high quality feedback information) is also a priority for us: Camtasia is seen as an additional tool to assist in positive, personalised and supportive feedback for the learner. Overall, our conclusions are that Camtasia has clear potential as a powerful feedback tool that can support student development. More effective, individualised feedback to each student means clearer learning and study goals and thus better opportunities for achievement. However, there are clear challenges: ●● Roll out to tutors needs to be thought-through, and timed within academic year/programmes of study; ●● We need to build this initiative into staff training and allocate sufficient training time; ●● It will be vital to work with students on the best ways to maximise the benefits including how they become active users of their feedback. Further innovative development will come from staff taking ownership of the software and feeling comfortable with the technology (possibly through professional training). Students need to be shown how to become active participants in the feedback process but teachers need to give them the information in a way they can access it. 41 Compass: The Journal of Learning and Teaching at the University of Greenwich, Issue 6, 2012 What of the future? For us the Camtasia investigations are ongoing but we are also interested in helping staff develop their plans to use Camtasia for instructional, ‘how-to’ videos, as reported in our staff feedback. This has been positively reviewed elsewhere (Henrickson, 2008): Camtasia is recommended for the generation of ‘video demos,’ to help students quickly learn various computer tools. We are also interested in using Camtasia as a bridge for research into other ways of supporting teaching and learning – e.g. facilitating the development of personalised study programmes, especially for new blended deliveries; helping students with specific learning issues (such as autism) where face-to-face feedback is less effective; and exploring how staff can provide electronic feedback as part of a re-structured and more efficient time management regime. References Baartman, L.K., Bastiaens, T.J., Kirschner, P.A. and van der Vleuten, C.P. (2007). ‘Evaluating Assessment Quality in Competence-Based Education’, Educational Research Review, vol. 2, pp. 114-129. Chiang, I.C. (2010). ‘Evaluation of three feedback methods’, Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications, Chesapeake:Canada. Clynes, M.P. and Raftery, S.E. (2008). ‘Feedback: An Essential Element of Student Learning in Clinical Practice’, Nurse Education in Practice, vol. 8, pp. 405-411. Crook, A., Mauchline, A., Maw, S., Lawson, C., Drinkwater, R., Lundqvist, K., Orsmond, P., Gomez, S. and Park, J. (2012). ‘The Use of Video Technology for Providing Feedback to Students: Can it Enhance the Feedback Experience for Staff and Students?’, Computers & Education, vol. 58, pp. 386-396. Embregts, P.J. (2000). ‘Effectiveness of Video Feedback and Self-Management on Inappropriate Social Behavior of Youth with Mild Mental Retardation’, Research in Developmental Disabilities, vol. 21, pp. 409-423. Henrickson, K. (2008). ‘Multimedia Presentations: Camtasia Studio 5.1’, The American Biology Teacher, vol. 70, no. 8, pp. 506-506. Hoban, G. and Hastings, G. (2006). ‘Developing Different Forms of Student Feedback to Promote Teacher Reflection: A 10-Year Collaboration’, Teaching and Teacher Education, vol. 22, pp. 1006-1019. Kostons, D., van Gog, T. and Paas, F. (2012). ‘Training Self-Assessment and Task-Selection Skills: A Cognitive Approach to Improving Self-Regulated Learning’, Learning and Instruction, vol. 22, pp. 121-132. Nicol, D.J. and Macfarlane-Dick, D. (2006). ‘Formative Assessment and Selfregulated Learning: A Model and Seven Principles of Good Feedback Practice’, Studies in Higher Education, vol. 31, no. 2, pp. 199-218. O’Day, D. (2008). ‘Using Animations to Teach Biology: Past & Future Research on the Attributes that Underlie Pedagogically Sound Animations’, The American Biology Teacher, vol. 70, no. 5, pp. 274-278. Ofsted (2012). [Online]. Available at: http://www.ofsted.gov.uk/. [Accessed 12 May 2012]. 42 http://www.ofsted.gov.uk/ Compass: The Journal of Learning and Teaching at the University of Greenwich, Issue 6, 2012 Reuter, J. (2007). ‘AV and Software Reviews. Creating and Publishing custom-made multimedia screen presentations: Camtasia Studio 4.0 ‘, Classroom Technology Reviews, vol. 69, no. 2, pp. 117-118. Rogers, D.A., Boehler, M.L., Schwind, C.J., Meier, A.H., Wall, J.C. and Brenner, M.J. (2012). ‘Engaging medical students in the feedback process’, The American Journal of Surgery, vol. 203, pp. 21-25. Schartel, S.A. (2012). ‘Giving feedback – An integral part of education’, Best Practice & Research Clinical Anaesthesiology, vol. 26, pp. 77-87. Silva, M.L. (2012). ‘Camtasia in the Classroom: Student Attitudes and Preferences for Video Commentary or Microsoft Word Comments During the Revision Process’, Computers and Composition, vol. 29, pp. 1-22. Survey Monkey (2012). [Online]. Available at: http://www.surveymonkey.com/mp/lp/ examplesurveys/?utm_network=g&utm_term=survey+monkey&utm_campaign=e&gclid=CLKA_ I6m968CFWwntAodJzWIEA&pmt=e&utm_source=adwords&mkwid=s5s2LcwRS&utm_medium=ppc&sour ce=SearchNetwork&cmpid=brand&pcrid=21555879716&kw=survey+mon [Accessed 18 May 2012]. Tillema, H., Leenknecht, M. and Segers, M. (2011). ‘Assessing assessment quality: Criteria for quality assurance in design of (peer) assessment for learning - A review of research studies’, Studies in Educational Evaluation, vol. 37, pp. 25-34. van Vonderen, A., Didden, R. and Beeking, F. (2012). ‘Effectiveness of instruction and video feedback on staff’s trainer behavior during one-to-one training with children with severe intellectual disability’, Research in Developmental Disabilities, vol. 33, pp. 283-290. Author Biographies Howard Lee is trained in plant ecology (MSc, PhD), and experienced in agroecology (senior lectureship at Wye College, University of London 1990–2002); he was Director of Sustainable Agriculture & Rural Development at Forum for the Future 2002–3. He is currently lecturer and sustainability champion at Hadlow College and in addition the Regional Sustainability Adviser (south east) for the Learning & Skills Improvement Service. Howard is on the Editorial Board and also the Reviews Editor for the International Journal of Agricultural Sustainability. He has published several book chapters in recent years. Louise Carr is the advanced practitioner for HE and an ESOL and study skills specialist at Hadlow College in Kent, an affiliate college to the University of Greenwich. She works closely with staff and students to develop opportunities in the application of e-learning and technology with a particular interest in collaborative learning. 43 http://www.surveymonkey.com/mp/lp/ Compass: The Journal of Learning and Teaching at the University of Greenwich, Issue 6, 2012 98 Case studies Louise Carr and Howard Lee Methods Results Discussion References Author Biographies