Case Studies Compass: Journal of Learning and Teaching, Vol 11, No 1, 2018 Lights, camera, action research! Engaging filmmaking students in feedback David Thompson Manchester Metropolitan University Abstract This paper is a response to repeated National Student Survey evidence of dissatisfaction with ‘assessment and feedback’ in undergraduate film production courses, as well as an expression of genuine interest in how to address the characteristic indifference and laissez- faire attitude of students engaged in filmmaking. It explores how filmmaking students may be effectively engaged with feedback by undertaking it themselves before I issue it formally as their tutor. As opposed to a formative exercise, this activity was undertaken just minutes before their ‘live’ summative feedback. The intention was to create autonomous, independent and proactive learners from the start of their degrees. The research forming the basis of this case study, which I undertook in a previous role at a UK higher education institution, identifies that critiquing without knowing what action to take is a barrier to students’ engaging with their feedback and that being able to maintain a distance from and perspective of their work enables students to take responsibility for themselves. It finds that in order to move on from mistakes, feedback needs to be straight to the point and clear, in order to create an action plan to improve. “Houston, we have a problem” (Apollo 13, 1995) The study was prompted by my observation of a change in engagement with feedback given to students from tutors. There was a less positive engagement by students with my feedback; other lecturers in my department voiced similar concerns that students were making complaints about feedback. As a consequence, I determined to undertake research into my own practice, as simply and effectively as possible, so that the method could be applied by others in my department. Instead of running a survey or asking the members of a staff/student committee for their opinions, I wanted to understand what lay at the root of the problem. “Toto, I’ve a feeling we’re not in Kansas anymore” (Wizard of Oz, 1939) Literature review Bloom’s ‘cognitive’ and ‘affective’ domain taxonomy volumes (1956 and 1964) detail clearly how to assist the student in progressing through higher levels of abstraction. The ‘affective’ domain includes as its final three stages: valuing, organisation and characterisation by value set. Pratt’s ‘Good Teaching: One Size Fits All?’ (2002) also lays out a clear, sequential teaching theory which fits practically alongside Biggs and Tang’s ‘Teaching According to How Students Learn’ (2002), with its focus on theories of teaching and the ‘Cognitive Level of Learning Activities’ chart, again a practical guide alongside Bloom’s ‘Cognitive Taxonomy’. Whilst these theories focused on the actual process of learning, they did not Case Studies Compass: Journal of Learning and Teaching, Vol 11, No 1, 2018 include the ‘affective’ domain details of organisation and application – I felt that both of these aligned well with making the study part of the class; I wanted to ensure I was using, in Pete Boyd’s words, a “transformative approach to thinking about the purposes of assessment” (2007, p. 7). Meer and Chapman (2013) argue that students’ understanding of marking criteria make them active participants in the process, encouraging a community of practice. This is supported by Rust et al. (2005), who suggest that tacit knowledge is gained by participation rather than instruction, and Gordon (2010), who states that students are far more attentive when assessing their peers. Walser (2009) concurs, as well as Hattie and Timperley, who also state categorically: “Feedback is one of the most powerful influences on learning and achievement” (2007). Sadler (1989) even goes as far as to say students “must already possess some of the same evaluative skills as their teacher”. He argues that they are already generating their own feedback. In outlining the seven principles of good feedback, Nicol and Macfarlane-Dick (2006) are also positive, describing feedback as inspiring independent students and therefore self- regulated learners concurring with Bloom. They also allude to its deeper influence which “shows that feedback both regulates and is regulated by motivational beliefs” (op.cit., p.201). Hattie and Timperley (op.cit.) agree, proposing that increased effort and the taking on of more challenging tasks is inspired by correct use of feedback. If students also believe that they can achieve when the intended goal is clear, they are also more likely to increase their effort (Kluger and DeNisi, 1996). Hattie et al. also say that effective instruction is key to enhanced learning so that “teachers can create a learning environment in which students develop self-regulation and error detection skills” (Hattie, Biggs and Purdie, 1996). Hattie (2008) puts feedback at the top of his education table. This sense of role reversal is thought of positively by Campbell (2015), who argues that tutors should not be scared of giving up their own role, inspiring greater student independence, engagement and interaction. Although my research is stand-alone, it is part of a scaffold process of the type recommended by McNiff (2005). “Now which way do we go?” (Wizard of Oz, 1939) Methodology Using a summative script assignment submitted three weeks earlier, the students were given specific instructions to write their own feedback online, by re-reading both the work they had submitted and the original assessment brief for the module. They then had a personal tutorial with me, to receive their actual summative feedback. The process was then completed by a digitally-recorded focus group. Participants I used a group of five foundation-year students, studying BA Film and Television Production, before they began their first year at undergraduate level. This age group was chosen (as opposed to level 6, for example) because, as Meer and Chapman (2014) note, students “would benefit more if this engagement with the marking criteria could happen earlier in their student journey”. Case Studies Compass: Journal of Learning and Teaching, Vol 11, No 1, 2018 Setting The intervention and data collection took place in a versatile computer room, a location familiar to the students and often used for teaching and learning. Change Process My intervention involved students’ engaging in tutor activities by marking and giving their own work feedback just before they received their official feedback from me. The assignment they had completed was a five-page script, with appropriate industry formatting. Instead of getting their grade and feedback straight away, as they usually did, I wanted them to engage in the process actively, to encourage deep learning and engage with the higher cognitive levels. I issued a number of specific, logical and progressive steps to go through for simple completion of the process: 1. Read your scripts out loud in pairs. 2. Read the brief on Blackboard that you were originally set. 3. Write a short summary of the aspects of the brief you think you have hit. 4. Write a list of positive and negative bullet points. 5. Write a summary paragraph, focusing on the positives of your work and what you think you need to work on, moving forward. Data collection The students submitted their feedback reports and then participated in a digitally-recorded focus group to discuss their thoughts about the process. Before the focus-group session began, they were asked to answer two written questions: • What worked about the process? • What didn’t work about the process? The focus-group discussion was then left deliberately open, to allow them to explore their own experience of, and reflections on, receiving feedback throughout the foundation year and to encourage whatever they wanted to speak about to come up. Using thematic analysis, looking at the strengths and limitations of the study design, I then examined the data. The discussion is illustrated by a selection of quotations from anonymised student feedback. “You talking to me?” (Raging Bull, 1980) Writing their own feedback Though, at first, the change process seemed boring to them and there was a lack of enthusiasm, the structure worked once they started to engage themselves practically and Case Studies Compass: Journal of Learning and Teaching, Vol 11, No 1, 2018 write. My having laid out how I composed my feedback to them gave them a framework, the blanks of which they could fill in to build their own critique for themselves. A yet greater enthusiasm was apparent as they got to stand on their own two feet and take this in to the focus group: they all had something to bring through their experience. What also worked was the similarity between their feedback to my own. However, in many circumstances, they were far more critical than I was about their own work. This worked both ways, because then when it came to listing what did work (I had instructed them to do that at the end of their feedback), any self-praise derived from reasoned evidence, not from ego. They were able to analyse and delineate, not merely to praise and be positive for the sake of it. This produced some exciting action plans for moving forward, as illustrated by this comment from one participant: “On the positive side the storyline was strong and interesting and had potential to be good. However to make it more exciting the dialogue and descriptions could be more emotionally descriptive to make it more exciting and keep the audience interested.” (Student B). This was above and beyond what I had expected and showed so many attributes for a level 3 student! First, she had stepped back and analysed; second, she had been accurate and articulate in her critique; third, she was able to get beyond simple praise and inspire and encourage herself to move forward from a place of confidence, certainty and deep learning. Instead of directing, I had therefore become more a facilitator, a guide alongside her much more self-determined way ahead. Focus Group student themes Independence The students liked being able to discover their own strengths and weaknesses for themselves. There was a sense of empowerment in their answers. None of them complained of feeling restrained, only of wanting more detail. They were able to begin to express what they did not want as well as what they did; clear and straight to the point was a common theme: “give me something I can work on” (Student B). In contrast, there was a severe dislike of “critique for critique's sake”, without knowing the ‘how’ of how to change it (Student C). Though Student G, confused, expressed a minority view – “What can I do about it now?” – his comment did reflect a more prevalent attitude: if I can’t do anything about it now, then what use is it? “Well, nobody’s perfect” (Some Like it Hot, 1959) Student G felt that his feedback was neither detailed nor harsh enough; he then described how another tutor’s feedback was too harsh and unjustified. When I pointed out that these two views were contradictory, he fell silent and others began to talk. After some quiet reflection, however, he did seem to have an epiphany, saying that, from the other tutor, there was nothing he could work with, only criticism, with no plan of how to do better; he had wanted detailed feedback, but detailed feedback that was actionable and not simply negatively critical. He was able to analyse and articulate to me the practicalities of his emotions, instead of just getting angry – an ‘affective’ domain characteristic. He was Case Studies Compass: Journal of Learning and Teaching, Vol 11, No 1, 2018 therefore able to communicate exactly what his tutor couldn’t. This was exciting! The process he went through in order to express what he felt was exactly what he needed his tutor to do about his work – not just to “slag it off”, as he put it, but to tell him practically what he needed to do to move forward. The message coming back from the students was therefore clear: feedback was only useful if there was something they could do about it. Consequently, I found myself determining how best to incorporate constructive feedback into the next academic year’s sessions. As Hattie and Timperley (2011, p. 104) conclude, ‘It is the feedback information and interpretations from assessments, not the numbers or grades, that matter... feedback…one of the most powerful influences on learning, too rarely occurs”. Empowerment As a result of the reflection, there was a progressive sense of empowerment, directness and honesty towards me. They began talking about my early lectures, saying that they had been unsure of me at first, before detailing how each session built on those preceding it, so they began to understand and have more confidence in my methods. That I taught them progressively and fairly made sense and worked. Valuing their Learning Overall, there was a sense of enhanced understanding amongst the students: they displayed characteristic ‘affective’ domain qualities; they began valuing their feedback and their role in it. This was displayed in their critiquing another lecturer’s style. Once they had been ‘on the other side’ (writing feedback on their own work) and then been given the space by me, they were able to analyse – a key concept of ‘cognitive’ domain activity. Focus group – tacit observations In the focus group, there was an initial silence and nervousness. As a result, I had to emphasise how important it was to the effectiveness of my research and to my subsequent actions that they be absolutely candid. Once the discussion was not centered on me – and clearly that was an issue from their body language and tentative answers – they were able to relax and analyse properly. Even those who did not contribute were visibly nodding and taking part emotionally, if not vocally. The consensus was that the change had worked because they could distance themselves from their own work, take a step back and analyse. It was apparent that they felt listened to and knew more about themselves and what they needed, analysing and effectively engaging in a community of research amongst themselves. They were able to distinguish what feedback they needed and – crucially in student G’s case – the ‘why’. Through the process, they developed experientially the knowledge and confidence to practice what it took to use feedback and move on. What I learnt was that separating them from their own work, and encouraging them to step back and be objective, defused negative criticism. One of the barriers to feedback in my experience can be students’ disagreement with or sensitivity to negative comments about their own work, especially if they are not expecting it and have spent a long time doing it. This change process enabled them to see for Case Studies Compass: Journal of Learning and Teaching, Vol 11, No 1, 2018 themselves, before they simply ‘got their mark’. The tacit implication is that students with this degree of self-knowledge will be willing to engage in enquiry and analysis as a result. The research showed a positive change because, despite initial resistance from the students, they did grasp that it was a good idea; furthermore, they learned, experientially, from the inside, not simply by being told: learning from the inside out, instead of outside in; showing, not telling. Another observation that arose from this research exercise was the considerable negative commentary and lack of positive commentary, about other lecturers who were not present. This is clearly a factor with significant implications for the information gathered: whereas the feedback about me individually was balanced and limited, the feedback about a lecturer who had taught them the previous semester was much more extensive and trenchant. I am led to conclude that analysis of my practice might have been more informative had the group discussion been handled by another tutor in my absence, thus opening up a freer space for them to scrutinize my methods. “Now, where was I?” (Memento, 2000) A key conclusion from this research is that being able to maintain a distance from and a more impartial perspective upon their work, enables students to take responsibility for themselves. Students emphasised strongly that in order for them to move on from mistakes, the feedback needs to be clear, frank, pertinent and constructively evidenced so that they can plan and carry out strategies for improvement. This research shows that simply critiquing, without knowing what action to take, is a barrier to students’ engaging with feedback. Being able to maintain a distance gives them perspective on the work they have done. It also shows that students engage in feedback and take ownership of the academic process for themselves when they get access to stepping into the shoes of the tutor and are thus empowered. Ownership and independence can be created by issuing a student responsible tutor orientated task, stimulating enquiry and analysis as a result. As these are aspects of Bloom’s ‘cognitive’ domain, such a task is immensely useful in progressing students’ learning, by engaging them with feedback. They can take a step back, in order take an effective leap forward. "Where we go from there is a choice I leave to you." (The Matrix, 1999) Reference list Anderson, LW., Krathwohl, D.R. and Bloom, B.S. (2001) ‘A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives.’ Theory Into Practice, 41 (4). Apollo 13. (1995) Directed by Ron Howard. Los Angeles, California: Universal Pictures. Case Studies Compass: Journal of Learning and Teaching, Vol 11, No 1, 2018 Biggs. J. and Tang, C. (2007) Teaching for quality learning at university (3rd edition). Maidenhead: Oxford University Press. Bloom, B.S. (1956) Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain. New York: David McKay Bloom, B.S. (1964) Taxonomy of educational objectives: The classification of educational goals. Handbook II: Affective domain. New York: David McKay Boyd, P. (2007) Developing Effective Assessment in Higher Education: A Practical Guide. London: McGraw-Hill Education. Campbell, E. (2015) ‘Students as facilitators: an evaluation of student-led group work.’ Practitioner Research in Higher Education, 9(1), 52-58. Chapman, A., Parmar, D. and Trotter, E. (2007) ‘An evaluation of the first year experience from the mature students' perspective: a multi-institutional comparison.’ Practitioner Research in Higher Education, 1(1), 15-19. Hattie, J. A. and Timperley, H. (2007) ‘The Power of Feedback.’ Review of Educational Research, 77(1), 81-112. Hattie, J. A., Biggs, J. and Purdie, N. (1996) ‘Effects of Learning Skills Interventions on Student Learning: A Meta-Analysis.’ Review of Educational Research, 66(2), 99-136. Kluger, A. N. and DeNisi, A. (1996) ‘The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory.’ Psychological Bulletin, 119(2), 254-284. The Matrix (1999) Directed by Lana Wachowski and Lilly Wachowski. Los Angeles, California: Warner Bros. Meer N. and Chapman, A. (2014) ‘Assessment for Confidence: Exploring the impact that low-stakes assessment design has on student retention.’ The International Journal of Management Education, 12(2), 186-192. Meer N. and Chapman, A. (2013) ‘Can we do it like this?: students as partners in the assessment process.’ Higher Education Academy conference, Nottingham Trent University, 22nd–24th April. Memento (2000) Directed by Christopher Nolan. Los Angeles, California: Newmarket. McNiff, J. (2005) Action research: Principles and practices (2nd edition). London: Routledge. Nicol, D.J. and Macfarlane-Dick, D. (2006) ‘Formative assessment and self-regulated learning: a model and seven principles of good feedback practice.’ Studies in Higher Education, 31(2), 199-218. Pratt, D. (2002) 'Good teaching: One size fits all?' New Directions for Adult and Continuing education, 93, 5-11. Case Studies Compass: Journal of Learning and Teaching, Vol 11, No 1, 2018 Raging Bull (1980) Directed by Martin Scorsese. Los Angeles, California: United Artists. Rust, (2005) ‘A Social Constructivist Assessment Process Model: How the research literature shows us this could be best practice.’ Assessment and Evaluation in Higher Education, 30(3), 231-240. Sadler, D.R. (1989) ‘Formative assessment and the design of instructional systems.’ Instructional Science, 18, 119-144. Some Like it Hot (1959) Directed by Billy Wilder. Los Angeles, California: Metro-Goldwyn- Mayer. Wakeford, R. (1999) ‘Principles of Assessment.’ In: Fry, H., Ketteridge, S. and Marshall, S. (eds.) A Handbook for Teaching and Learning in Higher Education. Glasgow: Bell and Bain, 58-69. Walser, T. (2009) ‘An action research study of student self-assessment in higher education.’ Innovations in Higher Education 34, 299-306. The Wizard of Oz (1939) Directed by Victor Fleming. Los Angeles, California: Metro- Goldwyn-Mayer. Yorke, M. (1999) Leaving early: Undergraduate non-completion in higher education. London: Palmer Press.