Case Studies Compass: Journal of Learning and Teaching, Vol 5, No 9, 2014 Learning videos - do they work for you? Gabriela Daniels University of the Arts, London An exploration of the value of integrating on-line instructional videos into the students’ learning experience in practical workshops Over the last two decades, learning technologies (accessed via personal computers and, more recently, mobile electronic devices such as tablet computers and smart phones) have pervaded all stages of formal education. Their increased use has been furthered by widespread enhanced connectivity and the development of systematic approaches for integrating digital technology into pedagogic practices. One common term applied to the resources created by digital technologies is Learning Objects (LOs). Wiley (2000) offers a definition of LOs: “…any digital resource that can be reused to support learning. This definition includes everything that can be delivered across the network on demand, be it large or small.” Wiley’s definition encompasses a very broad range of digital resources, ranging from images, pre-recorded videos and animations to web pages, which aim to provide complete experiences via a combination of applications. This variety offers a range of opportunities for educators to develop and integrate LOs into the curricula in a phased and practical way, reflecting subject-specific needs and organisational technical capacity. This study evaluates the effectiveness of one type of LO - specialist instructional videos - produced with the intention of offering on-demand instruction and enhanced visual presentation of complex processes in an art and design teaching context. The study specifically focuses on the reusability potential of the videos and how instructional design is likely to affect it. Two important organisational objectives drove the study: to develop sustainable and flexible pedagogic practices and to improve the internal organisational effectiveness of the process of video creation. Both staff and students were involved in the study. The constructivist approach to learning puts the emphasis on supporting learners in their construction of knowledge (Duffy and Cunningham, 1996). Consequently, the learners’ own context and engagement with the presented learning processes and aids are at the heart of current pedagogic practices. The proliferation of digital Learning Objects over the last fifteen years has prompted a wave of pedagogic research, seeking to review them in the light of constructivist learning. Some review articles, voicing critical views of LOs, state that they “…do not cause learning, but provide availability”, owing to the lack of sound pedagogic approaches in their design and to their poor searching capacity (Yahya and Yusoff, 2008). Other critics of the LOs suggest that their structures offer rather prescribed pathways within the learning processes, limiting learners’ active participation (Bannan-Ritland et al, 2000). These observations are consistent with the way Wiley, within his broader LOs classification, describes videos as “combined closed systems” which are single-purpose, providing Case Studies Compass: Journal of Learning and Teaching, Vol 5, No 9, 2014 instruction and practice and with constituents that cannot be used separately - audio and visual information (Wiley, 2000). Despite the acknowledged limited reusability of learning videos, their appeal to many teachers and learners is undisputed. Videos are broadly described as a multi-media learning resource (audio and visual) and their proliferation (YouTube, Vimeo and password- protected platforms) provides shared experiences to groups of learners. Their use for instruction and teaching has been supported by cognitive theoretical frameworks such as The Dual Coding Theory, further developed into a Multimedia Learning Theory (Mayer and Moreno, 2007). Based on empirical studies, these theories suggest that learners apply two types of information process systems: verbal (text, auditory narration) and visual (images/actions). In the initial stages of the cognitive process, each type of information is processed separately by the brain. At the final stage, the learner integrates and builds his/her own knowledge structure. On that basis, Mayer and Moreno (2007) suggest that the design of multimedia resources complies with the following fundamental principles: 1. The use of multimedia for instruction is likely to be more effective than text only. 2. Text-based information in a multimedia resource should be presented as an audio narration, not as an on-screen text. 3. The above principles are more important to learners with low prior knowledge of the subject. 4. Short audio highlights are more effective than detailed explanations. Despite the documented validity of the multimedia cognitive theory, studies into the impact of videos on attainment have so far offered inconclusive evidence of their benefits in comparison with traditional instructional methods, such as face-to-face and text-based instruction. However, several studies report improved student motivation and increased enjoyment in the subject (Whatley and Ahmad, 2007; Teng et al, 2009; Cherrett et al, 2009), which might have a more pronounced long-term and positive effect on learning than a short- term study can evidence. It is also important to note that most evaluation studies have focused on the students’ interaction with videos in a formal teaching context, as it provides the opportunity for quantitative comparisons of student outcomes. Whilst this approach still remains a valid evaluation methodology, the current prevalent use of smart phones by learners and teachers is starting to affect teaching and learning practices in a way that is harder to measure and evaluate. Mobile learning (mLearning) is a new form of e-learning that poses fresh challenges and opportunities. Wang and Cheng (2012) discuss the potential of mLearning in aiding the development of skills and knowledge in social and informal learning environments: “Informal learning emphasises the autonomy of deepening awareness, asking learners to combine study with some research and exploration…” Informal learning has always been a part of constructivist learning and mobile technology has expanded its potential exponentially owing to improved access to the available learning materials (LOs specifically). Thus, whilst the use of video recordings in formal learning Case Studies Compass: Journal of Learning and Teaching, Vol 5, No 9, 2014 environments might still compare unfavourably with face-to-face teaching and instructions, mobile technology provides opportunities for integrating on-line videos into an emerging new learning paradigm that mixes formal with informal learning. Finally, the expectation of reusability suggests that learners should be able to adapt LOs to their own context. This requires the consideration of two critical types of content: the LO- specific content and LO metadata. Metadata is defined as “descriptive information about the resource” (Wiley, 2000) and enables the searching and cataloguing of LOs by learners for their private needs. In addition, sequence and granularity - how big the LOs can be - are key elements of design, specifically in the case of more complex LOs. Those design principles also apply to the creation of on-line videos, as a stand-alone resource or as an element of a more complex LO. Ritzhaupt (2010) defines several groups of LOs stakeholders, representing a new economy of educational content development: the learners, the authors and instructional designers and the LO developers. Furthermore, Yahya and Yusoff (2008) also discuss the need to use experts’ feedback on the LOs’ past usage when planning new LOs with high potential reusability. This study aimed to collect information from representatives of all stakeholders’ groups, thus hoping to improve the educational and internal organisational effectiveness of the process of video creation and integration in the learning experience. Study methodology Qualitative data was collected from the following groups: academic and technical staff who had created the selected instructional videos; professional staff who were involved in the planning, filming and editing; academic and technical staff, who were not creators, but could be using the videos. The former two groups were interviewed individually (four interviews in total), whilst the latter group was interviewed via two focus groups, each session beginning with the viewing of a selection of three instructional videos. Data was collected, reviewed and compared against the following three criteria: alignment with multimedia cognition theory, potential for constructivist learning and reusability of resources. Quantitative data was collected from students (n=44) and was combined with some qualitative statements. Student groups were shown three full-length videos and asked to complete questionnaires, which required rating the videos against the criteria, stating whether or not they would view them in the future and providing information about the locality and type of mobile devices that they were most likely to use. The data was analysed using t-Test: Paired Two Sample for Means. Qualitative evaluation Interviews with the video creators revealed the following: Alignment with the multimedia cognition theory There was an agreement that the videos demonstrated techniques and processes which benefited from visual presentation: “…because it is a tacit skill…so for me it was really Case Studies Compass: Journal of Learning and Teaching, Vol 5, No 9, 2014 important it was close up, clear and from different angles…” Where more time and consideration were given to planning the video, as a part of funded projects, the use of a script was seen by the creator as key to providing clear and concise instructions. Where staff had less time to plan their videos, they could point to a range of possible improvements which could have enhanced the videos in line with the principles of multimedia cognition theory: present close-up views of area and movement; provide concise talk. Potential for constructivist learning The intention of the creators was that the videos could be used as an in-class resource and/or as an on-demand resource, available via the college VLE: “…being able to show students at close range, when you’re demonstrating how to sew a seam; demonstrating this to twenty students sitting around a sewing machine is not practical.” “They can dip in and out; they can look at one thing, fully understand that, then they can go back. We try to divide it into chunks, so they can actually understand one chunk, before they move on to the next.” “Demonstrate something different that can inspire.” The universal aim of the video creators was to enable the transition of a formal learning opportunity into an informal one to fit the student’s personal learning needs and/or to enhance a contextual knowledge creation. This approach to using on-line videos appeared consistent with the literature, suggesting that videos are context-dependent and represent informal learning opportunities. Another set of data was gathered by an interview with a video content developer. The importance of key messages and metadata, as well as concise presentations was highlighted. The use of a script in preparing for the video filming and/or voice over was seen as key to managing the duration and the content of the video: “…the video needs to be simple, short and directly to the point, and very clear.” These criteria are consistent with general good practice in preparing other visual aids (such as PowerPoint or similar presentations), but were seen as even more important in the context of the significant investment of time needed for video production (filming and - even more so - editing). Reusability To examine the reusability value, two focus groups were set up and asked to view and discuss selected internally-produced instructional videos, as well as the use of videos in classes and in workshops in general. The technical focus group thought the videos were helpful in a variety of situations, such as: to enable students to remind themselves of the process at a time when they needed to revisit it; to benefit students who are not native English speakers and who might not have fully understood the workshop instructions, since following visual presentation is easier than following a set of written instructions. Case Studies Compass: Journal of Learning and Teaching, Vol 5, No 9, 2014 The overwhelming view was that students could be referred to the videos as follow-up resources after a face-to-face instruction session. The student location at the time of using the videos was discussed too, as a traditional ban on the use of mobile phones in workshops could be preventing students from using those to stream videos whilst working in a workshop. However, this regulation is somewhat inconsistent with the widely-practised informal recordings of classes and demonstrations undertaken by students with lecturers’ permission. The need for legitimatising these new learning practices became clearer throughout the discussion. The academic focus group identified similar reasons to reuse the videos: to point students to videos so that they can catch up with details that might have been missed in class; to help international students; to deploy them as a motivational tool. An additional benefit of using instructional videos of processes rather than book illustrations was identified as the potential of the videos to show more clearly the development of a product from 2D to 3D. Academic staff also expressed the need to produce additional resources linked to videos to enhance constructivist learning, such as machine-threading diagrams and a glossary of terms (particularly helpful to international students). In summary, the reusability value of instructional videos was confirmed amongst different staff groups, referring mainly to informal, learner-led situations. Student data analysis Forty-four students following a range of garment design and realisation courses in the School of Fashion Design and Technology at the London College of Fashion were involved in the study. The most common age group was 21-25 and most students were female (89%). Most students (89%) had not seen the videos before, although they were available via the college VLE. The device most often used for viewing the videos, as well as for accessing all types of electronic resources, was the laptop (Fig.1). This information was further confirmed by the qualitative data collected in the short focus group discussions after the formal video viewing. Fig.1. Student usage of electronic devices 0 10 20 30 40 50 60 70 80 Electronic resources - any Videos % of interviewed students Case Studies Compass: Journal of Learning and Teaching, Vol 5, No 9, 2014 Constructivism The students valued the videos as a use-at-your-own-time resource, suitable either for reviewing a process that might not have been fully understood when demonstrated at college, or when something had been forgotten. In that context, the following features of the videos were identified as useful: - repeatability (video can be watched at any time); - viewing can be paused and resumed at student’s own pace; - provides an easier-to-comprehend visual narrative, where written instruction is less helpful. Multimedia design Three full-length videos were shown to the students: French Seam (FS), demonstrating a sewing technique (camera on the work area, voice over, two minutes); Over Locker (OL), providing an introduction to a specialist machine (talking head, zoom in and out on machine areas and the presenter, three minutes); Bespoke Tailoring-Jacket (BTJ), offering an overview of measuring for bespoke jacket creation (expert presenter and a model, approximately ten minutes). All videos had been produced internally, with the anticipated audience being the student group from within which the survey groups were formed. The videos were rated against three production-quality aspects - duration, pace and AV quality (sound and visual) using the Likert scale of 1 to 5, where 1 was the poorest score and 5 the most favourable. The students were also asked to score their intention to view the videos again, using the Likert scale in a similar manner. For the purposes of this study, ratings of 4 and 5 were grouped in a “positive” group of responses, whilst ratings 1, 2 and 3 were grouped in a “less favourable” group. 60 -70% of the students gave positive ratings to the FS video in all three criteria; the BTJ video received 60-70% of positive scores in the Overall Quality and Pace criteria, but 50% of positive scores for Duration. The OL videos received between 40% and 50% positive responses (Fig. 2). Fig.2. Percentage of students giving each video “positive rating” against the three production-quality criteria. 0 20 40 60 80 Duration Pace Overall Quality FS OL BTJ % interviewed students Case Studies Compass: Journal of Learning and Teaching, Vol 5, No 9, 2014 The two-sample t-paired test was applied on each quality criterion (Duration, Pace and Overall Quality) rating set for the videos and the respective rating for future viewings (Table 1). The test checks whether the mean value for each quality rating equals that of the intended viewings. For most data sets, the production-quality rating means were statistically higher than the respective intended viewings. This difference can be a result of the interviewees’ different rating approaches when responding either to a question regarding an object or to one regarding intentions. It is also likely that showing instructional videos in an out-of-studio context might have influenced the participants’ perceived need to see the processes again. Therefore a correlation coefficient for each set of data was also calculated, aiming to identify whether an increase in production quality ratings was matched by an increased intention to view again (Table 1). For BTJ and FS videos, the Duration and Pace ratings were correlated with the likeliness to view again, whilst the AV quality of the video was not. This might signify that, where the instructional content is highly relevant, students are more likely to view again short and concise videos, whilst being less concerned with additional production values. There were no correlations of values for the OL video, suggesting that the students were unsure about the overall effectiveness of the video. After each viewing, the student groups were also asked to share views in a semi-structured focus group discussion. The multimedia style of the instructional videos seemed pertinent to demonstrations of processes: “…it's easier to obviously see how it's been made as opposed to if it's just been written down.” It is possible that the absence of interactive elements increased the importance of video length and pace for repeat viewings. In that context, the qualitative data appears to support the findings of the quantitative data: “As long as they are short and sweet kind of thing, and they drive on, then I will watch them.” “… it's hard to say because in some (videos) like the jacket's there is lot to do, so you can only shorten it to an extent, but I would say that the French Seam was really good it was just like a few minutes.” Table 1. Statistical analysis of the data Video: OL Duration View Again Pace View again Quality AV View Again Mean score 3.49 2.97 3.51 2.97 3.56 2.97 Variance 0.89 1.66 0.84 1.66 0.78 1.66 Observations 39.00 39.00 39.00 39.00 39.00 39.00 Correlation co-efficient 0.29 0.02 0.39 No correlation No correlation Positive correlation Video: BTJ Duration View Again Pace View Again Quality AV View Again Mean 3.57 3.62 3.62 3.62 4.05 3.62 Variance 1.70 0.80 1.35 0.80 0.72 0.80 Observations 37.00 37.00 37.00 37.00 37.00 37.00 Correlation co-efficient 0.43 0.42 0.25 Positive correlation Positive correlation No correlation Video: FS Duration View again Pace View again Quality AV View again Mean 4.34 3.89 4.09 3.89 4.23 3.89 Variance 1.02 1.41 0.92 1.41 0.88 1.41 Observations 44.00 44.00 44.00 44.00 44.00 44.00 Correlation co-efficient 0.40 0.34 0.15 Positive correlation Positive correlation No correlation Case Studies Compass: Journal of Learning and Teaching, Vol 5, No 9, 2014 The students also valued the videos as providing the opportunity to see an instruction again and to refresh their memory, perhaps after class, or to fill in gaps in knowledge after a group demo. They had viewed YouTube videos of related processes, but found the ones shown to them in this study more informative. One student proposed that the videos be used for constructing the technical files, which each student is required to do as a part of the realisation units. Good Instructional Video Features A summary of the characteristics of a “good instructional video” that emerged from this study is outlined below. It is suggested that, if videos broadly conform to these characteristics, their reusability value is likely to increase:  key contextual information should be included in all instructional videos, even those that constitute a long chaptered process;  a glossary of terms should be provided;  voice over with fluent audio presentations should be aimed for;  audio information should be related to what is on the screen, rather than general information;  the videos should be short and clear;  editing cuts and switching between different filming angles should be minimised. Reusability Overall, the main reason expressed by the students for repeat viewings of the videos matched the creators’ intentions: to enable the construction of knowledge via practice at a time selected by the students. These findings resonate with the theoretical discussions about videos providing knowledge or instruction on demand. “…we photograph each stuff… Instead of that you can go home, watch this and make your technical file notes while watching this…” “…you are going too fast, and you have so much to do. It's easier to forget, so if you go home and if I could just go, oh, how did they do French Seam- I can just go in there so that's quite good…” “….you can stop that as many times as you want and go back to it. In a class there are always lots of people; you can't always ask as many questions as you want.” As the videos were shown to students in an out-of-studio context, they noticed the helpfulness of some concise contextual information, e.g. choice of fabrics and techniques. This relates to the suggestions for including glossaries, which, in turn, relates to the need to consider the video metadata at the time of planning the video. Conclusion The development of instructional videos should be guided by the understanding of the user’s context and the anticipated benefits, which in turn should influence the design aspects of the video. This study aimed to evaluate instructional videos by gathering information from a Case Studies Compass: Journal of Learning and Teaching, Vol 5, No 9, 2014 range of stakeholders, an approach broadly in line with the structure for evaluating LOs suggested by Williams (2000), i.e. clarify who wants to use them, identify what users expect from LOs and compare the gathered data about how the objects measure up against the criteria. The staff (creators and users) expressed broadly similar intentions for the use of the instructional videos, primarily to enhance informal learning. Hence, the integration of videos in the personal construct of knowledge was left to the students’ individual initiative and was expected to take place mainly outside the workshop environment. The creators and users specifically valued the capacity of instructional videos to provide visualisation of 3D processes, to support dyslexic students and/or international students by reducing the dependency on text-based instruction. These values, combined with the “on-demand” and mobile access afforded by modern technology, and their contextualisation potential suggest that instructional videos can become a more prominent component of student learning. Improved pedagogy and better understanding of the principles of multimedia cognitive theory can aid their integration in more formal learning scenarios, too. The students identified several contexts within which they would view the videos (for example, at home, to prepare a technical file; to review a partly-forgotten technique) and these matched the scenarios anticipated by the creators and users. However, it is not unreasonable to assume that an initial introduction within a formal learning setting will enhance the potential benefits. Berk (2009) suggests a range of techniques and approaches to multimedia teaching which would apply to most teaching environments and styles of videos, such as set discussion points or activities that enable the viewers to focus on an aspect of the video, or asking students to view and collaborate on a task. Although the majority of students in this study were more likely to view videos on laptops at home, it was notable that they still preferred very short videos. This suggests that the reusability of an instructional video is likely to be assessed by the students in a more general practical context and not on the merit of its content alone. The likely increase of informal mobile learning in the future will strengthen the importance of length/duration considerations, as well as the visual and auditory qualities (Shen and Wang, 2012). The results of the student survey also suggest that a preliminary evaluation of instructional videos as a part of their development process can be helpful, as it highlights clearly those design features that are most important to the subject content and to the context of use. In summary, videos with their multimedia approach to instruction will continue to be used in art and design education because of their visualisation capacity; however, as with any other learning material or learning object, they should be actively promoted and used by creators and users in a variety of formal and informal learning situations. A consistent approach to the introduction and evaluation of all types of learning objects, specifically in their early adoption stages, is recommended. Case Studies Compass: Journal of Learning and Teaching, Vol 5, No 9, 2014 References Bannan-Ritland, B., Dabbagh, N. & Murphy, K. (2000) ‘Learning object systems as constructivist learning environments: Related assumptions, theories, and applications.’ In D. A. Wiley (Ed.), The Instructional Use of Learning Objects: Online Version. Available at: http://reusability.org/read/chapters/bannan-ritland.doc (Accessed on: 13 August 2012). Beck, R. (2009) ‘Multimedia teaching with video clips: TV, movies, YouTube, and mtvU in the college classroom.’ International Journal of Technology in Teaching and Learning, 5 (1), 1-21. Cherrett, T. et al (2009) ‘Making training more cognitively effective: Making videos interactive.’ British Journal of Educational Technology, 40 (6). Choi, J. and Johnson S. (2005) ‘The effect of context-based video instruction on learning and motivation in on-line courses.’ The American Journal of Distance Education, 19 (4) 215-227. Duffy, T. and Cunnigham, D (1996) ‘Constructivism: Implications for the design and delivery of instruction’, as cited by Ritzhaupt, A. (2010) ‘Learning objects systems and strategy: a description and discussion.’ Interdisciplinary Journal of E-Learning and Learning Objects. 6. Available at: http://www.ijello.org/Volume6/IJELLOv6p217-238Ritzhaupt701.pdf (Accessed on: 25 March 2013). Ellis, R. and Child, M. (1999) ‘The effectiveness of video as a learning tool in on-line multimedia modules.’ Journal of Educational Media, 24 (3). Mayer, R.E. and Moreno, R. (2005) A cognitive theory of multimedia learning: Implications for design principles. Available at: http://www.unm.edu/~moreno/PDFS/chi.pdf (Accessed on: 18 July 2012). Nikoi, S. (2011) ‘CORRE: a framework for evaluating and transforming teaching materials into open educational resources.’ Open Learning, 26 (3), 191-207. Richardson, J. (2012) ‘Face-to-face versus on-line tuition: Preference, performance and pass rates in white and ethnic minority students.’ British Journal of Educational Technology, 43 (1), 17-27. Ritzhaupt, A. (2010) ‘Learning objects systems and strategy: a description and discussion.’ Interdisciplinary Journal of E-Learning and Learning Objects, 6. Available at: http://www.ijello.org/Volume6/IJELLOv6p217-238Ritzhaupt701.pdf (Accessed on: 25 March 2013). Teng, Y. et al (2009) Create motivating YouTube videos: Using dual coding theory and multimedia learning theory to investigate viewer perceptions. Available at: http://www.publicationshare.com/AERA _09_Youtube_0410_2009.htm (Accessed on: 13 August 2012). http://reusability.org/read/chapters/bannan-ritland.doc http://www.unm.edu/~moreno/PDFS/chi.pdf http://www.publicationshare.com/AERA%20_09_Youtube_0410_2009.htm Case Studies Compass: Journal of Learning and Teaching, Vol 5, No 9, 2014 Verleur, R., Heuvelman, A. and Verhagen, P. (2011) ‘Trigger Videos on the Web: Impact of audiovisual design’, British Journal of Educational Technology. 42 (4), 573-582. Yahya, Y. and Yusoff, M. (2008). ‘Towards a comprehensive learning object metadata: Incorporation of context to stipulate meaningful learning and enhance learning object reusability.’ Interdisciplinary Journal of E-Learning and Learning Objects. Available at: http://www.ijello.org/Volume4/IJELLOv4p013-048Yahya185.pdf (Accessed on: 10 August 2012). Wang, M. and Shen, R. (2012) ‘Message design for mobile learning: Learning theories, human cognition and design principles.’ British Journal of Educational Technology, July 43, PT.4, 561-575. Whatley, J. and Ahmad, A. (2007) ‘Using video to record summary lectures and to aid students’ revision.’ Interdisciplinary Journal of Knowledge and Learning Object, 3. Available at: http://www.ijello.org/Volume3/IJKLOv3p185-196Whatley367.pdf (Accessed on: 15 August 2013). Wiley, D. A. (2000) ‘Connecting learning objects to instructional design theory: A definition, a metaphor, and a taxonomy.’ In D. A. Wiley (Ed.), The Instructional Use of Learning Objects: Online Version. Available at: http://reusability.org/read/chapters/wiley.doc (Accessed on: 13 August 2012). Williams,D. (2000) Evaluation of Learning Objects and instruction using learning objects. Available at: http://www.reusability.org/read/ (Accessed on: 20 April 2013). http://www.ijello.org/Volume4/IJELLOv4p013-048Yahya185.pdf http://www.ijello.org/Volume3/IJKLOv3p185-196Whatley367.pdf http://reusability.org/read/chapters/wiley.doc http://www.reusability.org/read/