Literature Review Journal of Learning Development in Higher Education ISSN: 1759-667X Special Edition: Digital Technologies, November 2014 The use of IPAD technology to give multi-modal feedback to British Sign Language/English Interpreting undergraduate students Thaïsa Whistance University of Wolverhampton, UK Abstract Students undertaking the BA Honours British Sign Language/English Interpreting course at the University of Wolverhampton regularly submit video clips to the VLE to be commented on by their tutor and/or their peers. These clips are either of their practise interpretations or presentations delivered in British Sign Language (BSL). This kind of collaborative feedback is extremely valuable in helping students to develop their language skills. The feedback has, historically, been provided in written English. However, this presents a challenge because BSL is a visual language which does not have a standard written format. This case study focuses on the way digital technology has enabled the development of a method of feedback which uses IPAD technology to allow tutors to use audio narration, video modelling and annotation tools to comment upon a student’s video clip. This annotated video is then returned to students for viewing. Feedback was sought from students during the early stages of development of this method and this information has been used to make improvements to the feedback format. Keywords: British Sign Language; Interpreting; feedback; digital technology; IPad; multi modal feedback. Introduction This case study focuses on the use of tablet technology to develop a multi-modal feedback mechanism in response to video clips uploaded by British Sign Language (BSL)/English Interpreting undergraduate students which show them either presenting a piece in BSL or interpreting between English and BSL. For the purposes of this case study, ‘multi-modal’is Whistance The use of IPAD technology to give multi-modal feedback Journal of Learning Development in Higher Education, Special Edition: November 2014 2 used to describe the combination of the visual and auditory modes of English with the visual mode of sign language and pictorial/ graphic representations of sign language spatial features. Students have traditionally received feedback on their video uploads to the VLE (WOLF- Wolverhampton Online Learning Framework) in written English format (i.e. their first language – L1). There are, potentially, some issues with this written method of feedback though, as BSL does not have a standard written format and it can be challenging to describe sign language on paper. There have been attempts to establish a formal method of recording sign language in a written format, but at present there is no commonly used standard method. The complexity of writing BSL is demonstrated in the image below (Figure 1) which shows a dictionary entry with both a photographic image of a sign and a written and coded explanation of the Hand shape, Location, Orientation and Movement of the sign, as well as the Non-Manual Features that must accompany it. Figure 1. British Deaf Association (Brian, 1992, p.294). Figure 2. Hoffman-Dilloway (2011, p.348). Figure 2 shows an example of two different sign writing systems, Stokoe Notation (written by Joe Martin) appears on the left, while Sutton SignWriting (written by Valerie Sutton) appears on the right (mmc1.mpg). Hoffman-Dilloway (2011, p.348) Whistance The use of IPAD technology to give multi-modal feedback Journal of Learning Development in Higher Education, Special Edition: November 2014 3 The issues with attempting to write down sign language have been discussed by Mann et al. (2014, p.4) who allude to the fact that the: …bodily movements (that is, enactment by the signer designed to represent actions of characters, for example, as part of a story) and affective information (for example, the emotions of characters) are meaningful aspects of signed languages that are common in (signed) discourse, yet there are no well-known and systematic ways of capturing them. The complexity of trying to describe sign language via written feedback in a clear and effective manner is what created the impetus to find an alternative way of delivering corrections, praise and suggestions for improvements. Video feedback in physical skills-based disciplines Video annotation software, such as Elan, is in use in interpreter training programmes for student self-reflection purposes and for detailed linguistic analysis (Goswell, 2012), but it would be unrealistic to expect tutors to have the time available to analyse footage at such a detailed level for every student. It is therefore necessary to look at how current and developing technologies can aid tutors in delivering effective feedback in the most efficient manner. Mann et al. (2014) discuss the use of information and communication technologies in the teaching of signed languages and suggest that ‘video based feedback and its impact upon instruction is an interesting area for further research’ (2014, p.4). They briefly look at some of the computer software in use in Germany, Finland, and Switzerland to enhance Sign Language Interpreter training. Other disciplines which require the development of physical skills, such as sports coaching (Boyer et al., 2009; Barter, 2012) are already using multi-modal feedback to enhance and develop skills. There are several mobile apps on the market which provide a mechanism to analyse video footage of, for example, a golf swing where ‘video analysis makes it possible to detect weaknesses in respect of the golf swing, which may lead to future injury; such injury can be corrected or prevented at an early stage’ (Roos and Suruijal, 2014, Whistance The use of IPAD technology to give multi-modal feedback Journal of Learning Development in Higher Education, Special Edition: November 2014 4 p.400). It is applications like Coach’s Eye (TechSmith Corporation, 2014) and Ubersense Coach (Ubersense Inc.), along with those providing interactive whiteboard facilities such as Explain Everything (MorrisCooke), Doceri Interactive Whiteboard (SP Controls Inc., 2014) and Educreations Interactive Whiteboard (Educreations Inc., 2014) which can be utilised in the training of sign language interpreters to correct errors and make suggestions for future skill development, which Price et al. (2010) suggest are discrete potential outcomes of feedback. Initial research An initial research project (Whistance, 2013) explored the potential benefits of providing students with visual (signed) feedback rather than the traditional written feedback. To gather data, two separate tasks were set which involved students signing a piece to camera on a specific topic. This video clip was then uploaded to the VLE in order for the tutor to view it and provide feedback to the students. Whilst this feedback was given in written English for the first task, it was given in BSL for the second. At this point the tutor feedback was simply a video of the tutor using British Sign Language to give feedback. There was no audio or annotation provided and students could not view their own clip concurrently with the tutor feedback. A questionnaire was sent out to the students that had engaged in the two tasks to ascertain their views about the respective feedback methods. The participants all agreed to ethical release of the data collected. The questionnaire responses were anonymous, whilst the video clips uploaded by the students inevitably showed their identity. All of the participants were happy to proceed on this basis. There were 8 (57%) responses to the online questionnaire which was sent out following the task and two students were interviewed (14%). Initial results Whilst the questionnaire was delivered to a limited number of students (one class of 14 final year students 2012/13), the results from the online questionnaire showed a marked preference (87.5%) for feedback to be provided in the visual modality (signed). Several of the respondents made reference to similar themes. Firstly, the benefit of seeing the language modelled by the lecturer: Whistance The use of IPAD technology to give multi-modal feedback Journal of Learning Development in Higher Education, Special Edition: November 2014 5 I did prefer BSL feedback because I found it very useful to see a good clear example of how I can improve, e.g. ‘You signed this…however, signing it this way would be much clearer’. (Respondent 3) Secondly, some respondents recognised the fact that delivering the feedback in L2 (BSL) offered an opportunity for students to develop their receptive skills. Thirdly, the anticipated issue with describing BSL actions in written English was highlighted: …when the feedback was in English it was sometimes difficult to understand what was being referred to and how exactly to improve it. (Respondent 5) Finally, it was noted that tutor signed feedback leads to extra language learning opportunities, as described by Interviewee 1 who had ‘learnt vocab off you that I hadn’t seen before’. Extension of methodology At the interview stage, the point was raised that one of the advantages of written feedback is that students can have it as a constant reference point while they watch their own video clip. The possibility of providing spoken feedback as an alternative method was also brought up during this interview, with a view to ensuring accessibility to those students who may find written feedback more difficult to access, but still prefer to have their feedback delivered in their first language (L1). Therefore it seemed that a method of feedback that was blended and involved spoken, written and signed modalities would be a useful format. This led to further research and the early stages of developing such a mechanism (see Figure 3). Whistance The use of IPAD technology to give multi-modal feedback Journal of Learning Development in Higher Education, Special Edition: November 2014 6 Figure 3. The multi modal feedback mechanism shown in Figure 3 is done using IPAD technology and the interactive whiteboard application ‘Explain Everything’ (MorrisCooke, 2014) which allows the tutor to import a student video from either the IPad photo store or from various cloud storage apps (e.g. iTunes; Dropbox, 2014; Google Drive, 2014). Then, whilst playing this video inside the application, the tutor can provide audio narration, pause and annotate the video and also provide video modelling of any corrections required. This feedback is recorded and captured as a video file which can then be exported back to the cloud storage facility or to YouTube or saved back to the IPad photo store. The student is clearly able to see their own performance and the tutor can point out any errors; such as with hand-shape formation, visual spatial inconsistencies or incorrect use of non-manual features. Areas where the student has performed well can also be highlighted in this way. Students from the 2013/14 final year cohort who received this type of video file responded very favourably to this method of delivering feedback: If the feedback is not what is expected, I feel that watching it on a screen rather than face-to-face is more comforting, as you can watch it where you feel comfortable without having to justify or respond. I felt at ease and genuinely felt like I gained more confidence because the lecturer was able to understand what I was signing on most occasions by giving an auditory summary of what she had understood from my video clip. (Respondent 2) This picture is taken from a video feedback clip (Whistance, 2013) and is done using the application ‘Explain Everything’. (MorrisCooke, 2014) Whistance The use of IPAD technology to give multi-modal feedback Journal of Learning Development in Higher Education, Special Edition: November 2014 7 You are also able to see how something should have been done which is difficult to explain through spoken and written English. You are also able to; literally, highlight some idiosyncrasies in our BSL production and comment on how that affects the meaning of the message. It was always nerve-wracking getting video feedback but no more so than doing it in person! (Respondent 3) …It is done in real time so negative comments and positive comments occur throughout the length of the clip, depending on the quality of the performance that is! I think video feedback is ace because it makes us put the effort into our clips because you are putting the effort into our feedback. (Respondent 6) This seems to indicate that students had really thought about how the feedback had affected them, which may be because, as Wilkinson (2013, p.3) describes, ‘…having a recording of the lecturer’s thoughts and ideas as they are marking the assignment adds a personal element, which may increase the level of engagement the student has with the feedback…’. Further work Whilst it is clear from the responses of the students that this method of providing feedback is popular and is perceived as useful, it is important to quantify the benefits of providing it, as well as examine the impact providing it has on the time tutors need to spend delivering it. Therefore further research is now in progress to attempt to evaluate the efficacy of this method. This method of providing feedback will be used with a wider group of participants and also at varying stages in the programme (it has been used exclusively with students in the final year thus far) in order to better understand its impact. Other applications for this type of technology are also being explored, including the use of slow motion video to pinpoint linguistic features of BSL and also the use of sports coaching apps to allow side by side video analysis of a student interpretation versus an interpretation rendered by a professional interpreter. This side by side analysis may also be useful in looking at the development of particular skill sets and allowing students to compare past and present performance. These methods of analysing an interpretation Whistance The use of IPAD technology to give multi-modal feedback Journal of Learning Development in Higher Education, Special Edition: November 2014 8 offer possibilities to give two distinct types of feedback to the student as Voerman et al. (2014, p.96) describe here: On the one hand, there is progress feedback which compares the actual level of performance with the initial level highlighting the improvement, and on the other there is discrepancy feedback which compares the actual level of performance with the desired level of performance pointing out what is missing or what still has to be done. Conclusion The development of this multimodal feedback mechanism has been possible because of the advances in tablet technology which have made using video analysis and annotation a much easier and user/tutor friendly task. For programmes like BSL/English Interpreting, having technologies available that enable students to view their own performance alongside a modelled performance, or to receive a visual correction, is of great benefit, in the same way as has been shown in sports coaching. There are potentially many other subject areas that could make use of this type of feedback. It may, for example, be useful for those teaching on healthcare related courses or music and performing arts programmes when feeding back on student performance of practical techniques. Another potential application is to use video analysis as a tool for examining student participation in role-plays on courses such as Social Work. Many students engage with mobile technology on a daily basis and it seems likely that this engagement will continue to grow and therefore become even more important in the future of feedback in higher education. Finding effective and efficient ways of providing feedback using the technology available is, therefore, vital. Orsmond et al. (2011, p.249) discuss the ‘consistently low assessment/feedback scores within the NSS’ as evidence that there needs to be a focus on feedback as a dialogic activity. They highlight the difference between what the tutor perceives as effective feedback, and the student perception of that same feedback. Therefore, alongside using digital technology to deliver feedback, it is also important for tutor and student to consider the stage at which the feedback is delivered, as well as what specific type of feedback is most effective for, in this case, language learners, i.e. recasts, elicitations, explanation of grammar points (Han and Hyun Kim, 2008; Kaivanpanah et al., 2012). Whistance The use of IPAD technology to give multi-modal feedback Journal of Learning Development in Higher Education, Special Edition: November 2014 9 References Barter, P. (2012) ‘Using iMmediate video feedback’, Middlesex Journal of Educational Technology, 2(1), pp. 14-20 [Online]. Available at: http://elearningresearch.middlesex.wikispaces.net/Volume+2%2C+Number+1+Dec ember+2012 (Accessed: 11 August 2014). Boyer, E., Miltenberger, R.G., Batsche, C. and Fogel, V. (2009) ‘Video modelling by experts with video feedback to enhance gymnastics skills’, Journal of Applied Behaviour Analysis, 42(4), pp. 855-860 [Online]. Available at: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2790943/ (Accessed: 27 July 2014). Brian, D. (ed.) (1992) Dictionary of British Sign Language/English. London: Faber and Faber. Dropbox (2014) Dropbox (Version 3.5.1) [Mobile application software]. Available at: https://itunes.apple.com/gb/app/dropbox/id327630330?mt=8 (Accessed: 4 November 2014). Educreations Inc. (2014) Educreations Interactive Whiteboard (Version 2.0.3) [Mobile application software]. Available at: https://itunes.apple.com/gb/app/educreations- interactive-whiteboard/id478617061?mt=8 (Accessed: 4 November 2014). Google, Inc. (2014) Google Drive (Version 3.2.2) [Mobile application software]. Available at: https://itunes.apple.com/gb/app/google-drive-free-online- storage/id507874739?mt=8 (Accessed: 4 November 2014). Goswell, D. (2012) ‘Do you see what I see? Using ELAN for self-analysis and reflection’, International Journal of Interpreter Education, 4(1), pp. 72-81 [Online]. Available at: http://www.cit-asl.org/new/ijie/volume-4-1/ (Accessed: 10 September 2014). Han, Z. and Hyun Kim, J. (2008) ‘Corrective recasts: what teachers might want to know’, The Language Learning Journal, 3(1), pp. 35-44. http://elearningresearch.middlesex.wikispaces.net/Volume+2%2C+Number+1+December+2012 http://elearningresearch.middlesex.wikispaces.net/Volume+2%2C+Number+1+December+2012 http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2790943/ https://itunes.apple.com/gb/app/dropbox/id327630330?mt=8 https://itunes.apple.com/gb/app/educreations-interactive-whiteboard/id478617061?mt=8 https://itunes.apple.com/gb/app/educreations-interactive-whiteboard/id478617061?mt=8 https://itunes.apple.com/gb/app/google-drive-free-online-storage/id507874739?mt=8 https://itunes.apple.com/gb/app/google-drive-free-online-storage/id507874739?mt=8 http://www.cit-asl.org/new/ijie/volume-4-1/ Whistance The use of IPAD technology to give multi-modal feedback Journal of Learning Development in Higher Education, Special Edition: November 2014 10 Hoffmann-Dilloway, E. (2011) ‘Writing the smile: language ideologies in, and through, sign language scripts’, Language & Communication, 31(4), pp. 345-355 [Online]. Available at: http://www.sciencedirect.com/science/article/pii/S0271530911000474 (Accessed: 12 September 2014). Kaivanpanah, S., Alavi, S.M. and Sepehrinia, S. (2012) ‘Preferences for interactional feedback: differences between learners and teachers’, The Language Learning Journal, iFirst article, pp. 1-20 [Online]. Available at: http://dx.doi.org/10.1080/09571736.2012.705571 (Accessed: 11 June 2013). Mann, M., Haug, T., Kollien, S. and Quinto-Pozos, D. (2014) ‘Teaching signed languages’, in Chappelle, C.A. (ed.) The encyclopedia of applied linguistics. Blackwell Publishing Ltd., pp. 1-6 [Online]. Available at: http://onlinelibrary.wiley.com/doi/10.1002/9781405198431.wbeal1436/pdf (Accessed: 10 September 2014). MorrisCooke (2014) Explain Everything (Version 2.51) [Mobile application software]. Available at: https://itunes.apple.com/gb/app/explain-everything/id431493086?mt=8 (Accessed: 12 September 2014). Orsmond, P., Maw, S.J., Park, J.R, Gomez, S. and Crook, A.C. (2011) ‘Moving feedback forward: theory to practice’, Assessment & Evaluation in Higher Education, 38(2), pp. 240-252 [Online]. Available at: http://www.tandfonline.com/toc/caeh20/38/2#.VBAiQmOgunI (Accessed: 10 September 2014) Price, M., Handley, K., Millar, J. and O'Donovan, B. (2010) ‘Feedback: all that effort, but what is the effect?’, Assessment & Evaluation in Higher Education, 35(3), pp. 277- 289 [Online]. Available at: http://www.tandfonline.com/toc/caeh20/35/3#.VBAijmOgunI (Accessed: 10 September 2014). http://www.sciencedirect.com/science/article/pii/S0271530911000474 http://dx.doi.org/10.1080/09571736.2012.705571 http://onlinelibrary.wiley.com/doi/10.1002/9781405198431.wbeal1436/pdf https://itunes.apple.com/gb/app/explain-everything/id431493086?mt=8%20 http://www.tandfonline.com/toc/caeh20/38/2%23.VBAiQmOgunI http://www.tandfonline.com/toc/caeh20/35/3#.VBAijmOgunI Whistance The use of IPAD technology to give multi-modal feedback Journal of Learning Development in Higher Education, Special Edition: November 2014 11 Roos, S. and Suruijal, J. (2014) ‘Video analysis in golf coaching: an insider perspective’, Mediterranean Journal of Social Sciences, 5(21), pp. 399-404 [Online]. Available at: http://www.mcser.org/journal/index.php/mjss/article/view/4214 (Accessed: 4 November 2014). SP Controls Inc. (2014) Doceri Interactive Whiteboard (version 2.1.4) [Mobile application software]. Available at: https://itunes.apple.com/gb/app/doceri-interactive- whiteboard/id412443803?mt=8 (Accessed: 4 November 2014). TechSmith Corporation (2014) Coach’s Eye – Instant Replay Video Analysis (Version 4.5.2) [Mobile application software]. Available at: https://itunes.apple.com/gb/app/coachs-eye-instant-replay/id472006138?mt=8 (Accessed: 4 November 2014). Ubersense Inc. (2014) Ubersense Coach: Slow Motion Video Analysis (version 4.1.0) [Mobile application software]. Available at: https://itunes.apple.com/gb/app/ubersense-coach-slow-motion/id470428362?mt=8 (Accessed: 4 November 2014). Voerman, L., Korthagen, F.A.J., Meijer, P.C. and Simons, R.J. (2014) ‘Feedback revisited: adding perspectives based on positive psychology. Implications for theory and classroom practice’, Teaching and Teacher Education, 43(October 2014), pp. 91-98 [Online]. Available at: http://www.sciencedirect.com/science/article/pii/S0742051X14000766 (Accessed: 10 September 2014). Whistance, T. (2013) Which method of feedback (written English or videoed signed feedback) is the most accessible and easy to apply in response to students who have submitted a British Sign Language video clip via the VLE? Unpublished Post Graduate Certificate in Academic Practice in Higher Education research project. University of Wolverhampton. http://www.mcser.org/journal/index.php/mjss/article/view/4214 https://itunes.apple.com/gb/app/doceri-interactive-whiteboard/id412443803?mt=8 https://itunes.apple.com/gb/app/doceri-interactive-whiteboard/id412443803?mt=8 https://itunes.apple.com/gb/app/coachs-eye-instant-replay/id472006138?mt=8 https://itunes.apple.com/gb/app/ubersense-coach-slow-motion/id470428362?mt=8%20 http://www.sciencedirect.com/science/article/pii/S0742051X14000766 Whistance The use of IPAD technology to give multi-modal feedback Journal of Learning Development in Higher Education, Special Edition: November 2014 12 Wilkinson, S. (2013) ‘Incorporating audio feedback to enhance inclusivity of courses’, Journal of Learning Development in Higher Education, Issue 6, November, pp. 1-4 [Online]. Available at: http://www.aldinhe.ac.uk/ojs/index.php?journal=jldhe&page=article&op=view&path[] =218&path[]=143 (Accessed: 10 September 2014). Author details Thaïsa Whistance is a practising Registered Sign Language Interpreter (NRCPD) and she is a Lecturer in BSL/English Interpreting at the University of Wolverhampton where she teaches Consecutive and Simultaneous interpreting. She is also a Fellow of the Higher Education Academy. http://www.aldinhe.ac.uk/ojs/index.php?journal=jldhe&page=article&op=view&path%5b%5d=218&path%5b%5d=143 http://www.aldinhe.ac.uk/ojs/index.php?journal=jldhe&page=article&op=view&path%5b%5d=218&path%5b%5d=143 The use of IPAD technology to give multi-modal feedback to British Sign Language/English Interpreting undergraduate students Abstract Keywords: British Sign Language; Interpreting; feedback; digital technology; IPad; multi modal feedback. Introduction Video feedback in physical skills-based disciplines Initial research Initial results Extension of methodology Further work Conclusion References Author details