Microsoft Word - vonkonsky.doc Australasian Journal of Educational Technology 2009, 25(4), 581-595 Lecture attendance and web based lecture technologies: A comparison of student perceptions and usage patterns Brian R. von Konsky, Jim Ivins and Susan J. Gribble Curtin University of Technology This paper investigates the impact of web based lecture recordings on learning and attendance at lectures. Student opinions regarding the perceived value of the recordings were evaluated in the context of usage patterns and final marks, and compared with attendance data and student perceptions regarding the usefulness of lectures. The availability of recordings was not seen to impact lecture attendance, although students showed some tendency to listen to the recording for a missed lecture. Students who achieved a high mark tended to supplement lecture attendance with recording usage more than students who achieved a low mark, but they did so with greater variation. If students perceived that a learning experience was of value to their learning, they were more likely to use it. Individual case studies describing perceptions, usage patterns, and attendance records of selected students highlight the fact that there is great variation in successful learning patterns, and suggest that engagement is an important factor impacting learning. Although the use of recordings to supplement lectures was seen to enhance the learning of some students, its uptake and effectiveness was not uniform across the cohort. This observation highlights the need for a range of learning modes in engineering education, appealing to a diverse set of individual learning styles. Future work is described in the context of these findings. Introduction There are many pedagogical and logistical reasons to utilise web based lecture technologies (WBLT) to make lecture recordings available to students via the Internet. These include (Gosper, Green, McNeil, Phillips, Preston & Woo, 2008) • supporting students who are unable to attend class; • providing a study tool for review and revision; • catering for individual learning strategies and styles; • supplementing face to face lectures, but at a time and place of the student’s choosing; • accommodating student expectations regarding the digital delivery of course material; and • facilitating distance education as an alternate delivery mode. There is strong evidence that students place significant value on lecture recordings deployed via the Internet to personal computers and MP3 players. Research funded by the Australian Learning and Teaching Council (ALTC) conducted by Gosper et al (2008) showed that 66.8% of students surveyed believed that WBLT helped them to achieve better results, and 79.9% of students believed that reviewing lecture recordings made it easier for them to learn. 582 Australasian Journal of Educational Technology, 2009, 25(4) Academic staff who took part in the survey concurred to a lesser extent: 30.2% of academic staff believed that WBLT helped students to achieve better results, and 48.9% believed that the technology made it easier for students to learn. Many academic staff acknowledged that WBLT can enhance some aspects of traditional lectures, but there was concern about the extent to which the technology impacted lecture attendance. In particular, 55.1% of academic staff reported that lecture attendance had decreased as a result of WBLT, a concern that has been shown to be a barrier to WBLT adoption (Chang, 2007). Moreover, when asked for reasons for non-attendance at lectures, 68.3% of students surveyed reported that they could learn as well from WBLT as they could from attending the corresponding lecture in person. The ALTC study by Gosper, Green et al. (2008) surveyed perceptions of the impact of WBLT usage on learning and reasons for non-attendance at lectures. Both attendance patterns and recording usage were self reported on a five-point Likert scale. However, measurements of lecture attendance and WBLT usage statistics for individual students were not considered in the context of final marks or other independent measures of student learning. The principal contributions of the study reported here are to: • measure the impact of WBLT usage on lecture attendance; and • determine the extent to which supplementing lecture attendance with WBLT deployed lecture recordings leads to the achievement of expected learning outcomes. The approach was to analyse lecture attendance records and the extent to which students accessed streamed lecture recordings, and to correlate this information with final marks for individual students. A secondary purpose was to compare surveyed perceptions regarding the contribution of lecture attendance and lecture recordings to learning, and how these perceptions impacted attendance and recording use. Background It is widely argued that students respond best to integrated learning experiences that support individual learning strategies and styles (Felder, 1996; Felder & Brent, 2005). It has been argued that differences in learning styles should be taken into consideration when academic staff are planning lectures, particularly for lectures that are to be recorded (Fardon, 2003). Considering the value that different types of students place on available course material and opportunities for learning, Allert (2004) correlated preferred learning experiences with final results for introductory computer programming students. He found that students with higher final marks generally expressed a preference for traditional lectures over other learning experiences. In contrast, weaker students generally expressed a preference for educational software designed to assist them in visualising complex concepts. Allert’s methodology was to correlate student opinions of preferred learning experiences with final marks. However, the extent to which participation in any one learning experience influenced student perceptions or contributed to learning was not directly measured. Additionally, preferences regarding WBLT were not considered in Allert’s study. Online course material usage patterns, including audio recordings and video clip downloads, and the relationship between these patterns and final marks were studied at Pennsylvania State University for a subject delivered entirely online (Mathews, von Konsky, Ivins and Gribble 583 Haughton, Pisupati, Scaroni & DiBiase, 2004). A goal for that study was to assess bandwidth requirements, trends in usage patterns, and the extent to which goals for flexible delivery of course content had been met. For this purpose, server log files were analysed for aggregate trends. The dates on which students electronically submitted assignments or took online quizzes were also analysed. Initially, the Pennsylvania State investigators observed that many students began work on the due date. As a consequence, weekly milestones were instituted to motivate students to remain current. Students were expected to submit a short reflective essay each week to prepare them for work that was to follow later in the week. Results generally showed that students were less likely to get a good overall mark if this “wake up the brain” essay was submitted closer to the end of the week. However, some students did not fit this profile and were still able to commence work on the due date and do well. Brotherton and Abowd (2004) analysed server logs characterising usage of their eClass system. The purpose of eClass was to allow students to pay closer attention during class by providing a mechanism to capture notes written by the lecturer at the white board, alleviating the need for taking duplicate notes. These notes were automatically integrated with a video recording of the lecture and other lecture artefacts for later use by students. Brotherton and Abowd found that students tended to use the system in a sequence of study sessions lasting an average of only 4 minutes and 30 seconds, and that system usage peaked just before tests and exams. Using a control group that did not have access to eClass, no correlation between overall system usage and aggregate results was seen. Students reported that the system did not encourage them to skip class, although a statistical analysis of attendance data to corroborate this was generally inconclusive. It was not possible to correlate usage data with individual results or to examine case studies documenting unique learning strategies, because server logs could not identify individual students, survey questionnaires were anonymous, and attendance records were based on headcounts. Similarly, in an analysis of podcast download statistics conducted at Queensland University of Technology, Moss (2007) found that downloads peaked before major assessment milestones. However, after download, it was not possible to directly measure how students utilised podcasts, whether they listened to entire lectures, or the number of times that podcasts were played by students. While there was evidence that podcasts had an impact on lecture attendance, Moss did not take formal measures of attendance or correlate attendance or WBLT usage patterns with final results for individual students. McGarr (2009) argues that the extent to which podcasts impact learning and the traditional lecture may be directly related to how lecturers intend the podcasts to be used by students. He lists common uses of podcasts, as reported in the educational literature (McGarr, 2009): • Substitutional use: Podcasts capture traditional lectures for later playback by students, which may be used as a substitute for lecture attendance, for later revision during focused study, or when multi-tasking while undertaking activities such as commuting or physical exercise. • Supplementary use: Podcasts are produced by lecturers to supplement, enhance, or summarise material presented during class or in outside reading. • Creative use: Podcasts are produced by the learners as part of a formal learning exercise. 584 Australasian Journal of Educational Technology, 2009, 25(4) Methodology In the study reported in this paper, attendance data and WBLT usage patterns were studied in an undergraduate software engineering subject taught at Curtin University of Technology in the third year of accredited degree programs in computer science, information technology and software engineering. The subject covers advanced topics in software engineering, including processes for quality management, and verification and validation. Requirements analysis using formal methods and graphical modelling languages is also presented in the context of managing project quality and safety (von Konsky, Robey & Nair, 2004). The tuition pattern consisted of one 2-hour lecture and one 1-hour tutorial each week, with tutorial sessions commencing in the second week of instruction. Classes were conducted during 12 teaching weeks in a 15-week semester, followed by a 2-week final examination period. Subject material was delivered by a single lecturer, so student perceptions, attendance at lectures, and recording usage patterns were not influenced by factors related to team teaching. The subject was taught in a traditional large lecture format. Final marks were computed based on results from a mid-semester test, a group assignment, and a final examination constituting 20%, 20%, and 60% of the overall assessment, respectively. To pass the subject, students were required to achieve 50% of the overall marks, and 50% of the marks for the final examination. Final results at the authors’ institution are reported using a Grade-Mark combination, where the Grade is F for failing students or a number between 5 and 10 for passing students, and the Mark is a number between 0 and 100. Grade-Mark combinations range between F-0 and 10-100. Audio recordings were made of each lecture and deployed using the Lectopia digital streaming system (Curtin University of Technology, 2008; Fardon & Ludewig, 2000). Recordings were digitised and converted to a variety of popular formats at various bandwidths to accommodate streaming across dialup and broadband connections. Once digitised, the lecture recordings were made available on the Internet. Students could access the recordings only after verifying their identity using a unique username and password. Accessing the link to a lecture recording automatically caused the authenticated username, the Uniform Resource Locator (URL) for the given recording, and the date and time of access to be stored in a mySQL database. Web browsers were then automatically directed to the Lectopia server, which contained a brief synopsis of the lecture, the slides that were used in the live presentation, and links to audio streams in various formats and bandwidths. Using information in the database, it was possible to count the total number of times that each student initiated a particular Lectopia recording stream. It was also possible to determine the number of distinct occasions on which the stream was initiated. The distinct stream initiation metric is defined as the number of initiations for a particular recording that occurred on different days. The purpose of using the distinct stream initiation metric was to eliminate noise introduced by mouse button stutters and surfing not leading to significant activity. In this paper, all references to stream initiations are distinct initiations unless otherwise noted. Although the Lectopia system supports downloading recordings in addition to streaming them, this feature was disabled. This was necessary to log the dates and times when a student listened to all or part of each lecture recording. This would not have been possible if students were allowed to download lectures, aside from logging von Konsky, Ivins and Gribble 585 the date and time of the initial download. Some students expressed dismay at this aspect of the methodology because it prevented them from listening to lectures in unconventional places, such as on a personal MP3 player while riding the bus to and from the University. As is often the case in engineering education, lectures for this subject used slides containing textual, mathematical, and graphical content. Consequently, presentation slides were made available on the Internet in conjunction with the audio recordings. Students also possessed printed copies of this material that could be used during lectures and while listening to recordings. Students were expected to access the slides manually in either electronic or printed form while listening to streamed lecture recordings. However, the extent to which students accessed slides while listening to recordings was not measured in this study because the use of printed slides could not be measured. Lecture attendance was taken using paper slips that were distributed to students during a five-minute break in the middle of each lecture. These attendance slips required students to provide identifying information to indicate their presence in the lecture. Care was taken to ensure that each student was given only one attendance slip to prevent students from inaccurately recording the attendance of friends who were not actually present. During the final lecture, students were surveyed to capture opinions regarding the value of lecture attendance and recordings. The survey also captured self assessed participation levels and estimates of total effort. The purpose was to determine whether student opinions regarding the usefulness of any one medium impacted their use of that medium, or impacted their overall mark. Survey responses, attendance data, and Lectopia streaming records were analysed against final marks using descriptive statistics for students who provided informed consent. This study was approved in accordance with the Human Research Ethics process prescribed by Curtin University of Technology. Results Of the 148 enrolled students, 108 gave informed consent to participate in the study and completed the end of semester survey. The consenting sub-population constituted about 73% of the entire class. A Chi-squared test of association showed that the consenting and non-consenting population was statistically similar with respect to degree program (Chi-squared = 3.32, d.f. = 3, p > 0.05) and gender mix (Chi-squared = 0.67, d.f. = 1, p > 0.05). However, failing students were somewhat less likely to provide consent (Chi-squared = 10.18, d.f. = 1, p < 0.05), as were international students (Chi- squared = 4.54, d.f. = 1, p < 0.05). While failing and international students may be somewhat under-represented, the high response rate and the statistical similarity between consenting and non-consenting students in other demographic categories suggest that the sample is generally representative of the cohort as a whole. All further data presented in this paper were obtained only from students giving their informed consent to participate. Lecture attendance is shown in Figure 1. Some weeks had no lecture due to a public holiday (week 4), a non-contact study period (week 5), and a mid-semester test (week 9). Therefore, a total of 10 lectures were delivered in the 10 remaining teaching weeks of the semester. Attendance was initially high, but tapered off later in the semester 586 Australasian Journal of Educational Technology, 2009, 25(4) during the period when assignments usually become due and many students feel under pressure to meet deadlines. Anecdotally, this attendance pattern is consistent with that experienced in previous semesters, and with that reported by Brotherton and Abowd (2004). 0 10 20 30 40 50 60 70 80 90 100 W1 W2 W3 W4 W5 W6 W7 W8 W9 W10 W11 W12 W13 W14 Week Lecture attendance % Figure 1: The percentage of students attending lecture by week. 0 20 40 60 80 100 120 140 160 W1 W2 W3 W4 W5 W6 W7 W8 W9 W10 W11 W12 W13 W14 W15 W16 Week Number of distinct WBLT streams Figure 2: The number of distinct WBLT audio streams initiated by week. von Konsky, Ivins and Gribble 587 The number of distinct recording streams initiated on a weekly basis is shown in Figure 2. These data suggest recordings were used extensively for review purposes prior to the mid-semester test in week 8 and immediately prior to the examination during weeks 15 and 16. Usage of the Lectopia system to stream recordings was unremarkable in week 4 when there was no lecture due to a public holiday, and during the non-contact study periods in weeks 5 and 9. Students are meant to use the non-contact study weeks for independent study and review of subject material. However, no change in usage patterns was observed during these periods to suggest that students utilised recordings for that purpose. The average mix of attendance and Lectopia usage for corresponding lectures is shown in Figure 3 for students in failing, low pass, and high pass grade ranges. The figure shows the percentage of lectures attended by students in each grade range that were: • supplemented by lecture recordings • attended but not supplemented by accessing the corresponding recording • accessed via the recording only • not attended and for which the corresponding recording was never accessed. 14 12 10 66 55 54 10 9 15 23 27 5 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Fail Pass (<7) Pass (>=7) Grade range Lecture and WBLT mix (%) Both Recording only Lecture only None (n=10) (n=54) (n=43) Figure 3: Average lecture attendance and WBLT mix by grade range. Passing students were more likely to supplement face to face lectures with recordings than were failing students, and they were more likely to do so for lectures presented earlier in the semester. The data also showed that failing students generally did not use the Lectopia system to supplement or replace lectures presented during the latter part of the semester. The exception was the final lecture, which was a review. The purpose of that lecture was to prepare students for the final examination, and it was well attended by all students. 588 Australasian Journal of Educational Technology, 2009, 25(4) Figure 4: Distribution of lecture attendance by grade range Figure 5: Distribution of distinct recording streams by grade range Figure 4 shows the distribution of lecture attendance by grade range. The median number of lectures attended by students who failed the subject was 8 out of 10 lectures. The median number attended by students passing the subject and attaining a grade less than 7 was also 8 out of 10 lectures. The median number of lectures attended by students who achieved a grade of 7 or greater was 9 out of 10 lectures. In other words, half of the top performing students missed only 1 lecture or had perfect attendance. Half of all students in other grade ranges missed only 2 lectures or less. Moreover, a linear regression analysis showed lecture attendance was not a von Konsky, Ivins and Gribble 589 statistically significant predictor of final mark (r2 = 0.029, b = 0.816, n = 108, t = 1.766, p > 0.05). Although the data suggest that students generally attended lectures, when lectures were not attended the corresponding recording was accessed 41.6% of the time. The distribution of distinct recording streams initiated is shown in Figure 5 for failing students, passing students achieving a final grade less than 7, and passing students achieving a grade of 7 or greater. The median number of distinct recording streams initiated was similar for students in each of these three grade ranges. There was greater variation in the extent of recording utilisation by passing students. The number of distinct recording streams initiated was a statistically significant but weak predictor of final mark (r2 = 0.048, b = 0.306, n = 108, t = 2.314, p < 0.05). Not surprisingly, perceptions regarding the value of lecture attendance and WBLT impacted the extent to which students took advantage of those experiences: 45% believed that live lectures made a strong contribution to learning; 42% believed that live lectures somewhat contributed to learning. Students who had a positive perception regarding the usefulness of lectures had better overall attendance. This relationship is shown in Figure 6. Student perceptions regarding the contribution of recordings to their learning revealed that 22% believed that recordings strongly contributed to learning; 31% believed that recordings somewhat contributed to their learning. The extent to which student perceptions influenced recording usage is shown in Figure 7. The greater the perceived value of recordings, the more likely it was that students took advantage of this learning opportunity. However, students who perceived that recordings contributed to their learning utilised the system with greater variation than did those who perceived recordings to be of less value. Table 1 shows effort estimates reported by students on the end of semester survey. Median statistics show that half of all students reported spending 3 hours or more listening to recordings, at least 10 hours engaged in subject reading, 3 hours surfing the web, 10 hours reviewing subject material, 5 hours working on the individual part of the assignment, and 15 hours working on the group component of the assignment. A high standard deviation was associated with most of these estimates. Estimates associated with personal and group effort on the assignment were generally consistent with measured values for these attributes as captured in Personal Software Process logs (Humphrey, 1995, 2000) using the methodology described by von Konsky et al. (2005). Table 1: Student effort estimates Survey question Mean Std dev Median Hours spent listening to recordings 4.7 5.1 3.0 Hours spent reading material for this unit 13.4 12.8 10.0 Hours spent surfing the web for this unit 4.6 6.5 3.0 Hours spent in revision 14.5 12.8 10.0 Hours spent on individual parts of the assignment 5.4 3.0 5.0 Hours spent on group parts of the assignment 16.3 7.5 15.0 590 Australasian Journal of Educational Technology, 2009, 25(4) Figure 6: Lecture attendance by students with similar perceptions 5 – Strongly contributed to learning; 4 – Somewhat contributed to learning; 3 – Neutral contribution to learning; 2 – Little contribution to learning; 1 – No contribution to learning Figure 7: Distribution of recording usage for students with similar perceptions 5 – Strongly contributed to learning; 4 – Somewhat contributed to learning; 3 – Neutral contribution to learning; 2 – Little contribution to learning; 1 – No contribution to learning Discussion While Lectopia usage may have been a useful learning strategy for some students, its use was not required to achieve a successful academic outcome. Similarly, its use did von Konsky, Ivins and Gribble 591 not guarantee that learning would always take place, and could not be used to predict the level of scholastic achievement. The large variation in Lectopia usage by passing students suggests that some took advantage of this learning experience to a greater extent than did others, and with varying degrees of success. Collectively the data suggest that students generally attended lectures and that lecture attendance was not affected by the introduction of the Lectopia system. Additionally, lecture attendance by itself could not be used to predict academic performance and final mark. In fact, a few outliers did quite well with minimal lecture attendance. Similarly, while passing students had a greater tendency to supplement lectures with recordings than did failing students, they did so with great variation. This demonstrates that students have widely differing strategies for successful learning and that no one pattern can be said to lead to consistent outcomes for all individuals. Perhaps more importantly, it suggests that factors other than participation in any one activity were actually influencing learning. Review of individual cases Several interesting cases were selected to highlight the great variation in the learning strategies used by students, and to demonstrate that no particular combination of learning modes for either failing or passing students could be generalised for the cohort as a whole. For the purposes of this discussion, individual cases are referred to as Students A, B, C, and D. These cases are summarised in Table 2. Table 2: Case studies summary Metric Student A Student B Student C Student D Final grade-Mark awarded 7-70 F-46 7-78 8-88 Listen to whole recording? agree neutral strongly agree strongly disagree Number of distinct recording streams 6 8 37 7 Number of lectures streamed 5 4 9 3 Numbers of live lectures attended 2 7 9 10 Number of tutorial sessions attended 6 6 9 11 Hours listening to recordings 8 5 10 1 Hours spent reading for the subject 20 0 6 2 Hours surfing the web for the subject 6 5 1 1 Hours reviewing other course material 10 0 2 10 Student A achieved a final result of 7-70, but attended only the introductory lecture in week 1 and the review lecture in the final teaching week of the semester. Instead of relying on lecture attendance as the primary focus for learning, Student A initiated 6 distinct recording streams for 5 of the 10 lectures. Of these, 4 of the 6 streams were for lectures presented early in the semester. These were on topics that built an understanding of industry standards for quality management or were more theoretical than those presented later in the semester. Student A reported listening to 8 hours of recordings and agreed with the statement “when I listened to recordings, I tended to listen to the entire lecture”. Student A attended 6 of the 11 tutorial sessions. Of these, 4 were for topics arising later in the semester that stressed practical aspects of the subject material and were related to the group assignment. Perhaps more importantly, on the survey completed several weeks prior to the final examination, Student A reported having already spent 20 hours reading for this subject, 6 hours surfing the web, and 10 hours reviewing other course material. The learning strategy employed by Student A 592 Australasian Journal of Educational Technology, 2009, 25(4) appears to have relied heavily on self study utilising reading and the web. This was supplemented with the use of recordings for the earlier, more theoretical aspects of the subject, and with tutorial attendance for the later, more practical aspects. For Student A , this strategy was relatively successful. While Student A did not tend to take advantage of live lectures, the pattern exhibited suggests that the student was actively engaged during tutorial sessions and private study that included reading and the use of some recordings. In comparison, Student B achieved a final result of F-46, attended 7 of the 10 lectures, initiated 8 distinct recording streams for the first 4 lectures only, and attended 6 tutorials scattered throughout the semester. Student B reported spending 5 hours listening to recordings, and was neutral to the survey question about tending to listen to entire recordings. For the first three lectures, Student B supplemented lecture attendance using recordings. Student B did not attend the fourth lecture, but listened to the corresponding recording. A total of 2 lectures were not covered by either physical attendance or recording usage. At the time of the survey several weeks before the final examination, Student B reported having done no reading for this subject and having spent no time reviewing subject material. However, the student reported having spent 5 hours surfing the web for this subject. The failing strategy employed by Student B suggests that little effort was spent on this subject outside of scheduled class time, and brings into question the student’s level of commitment and class participation. Student C achieved a final mark of 7-78, attending 9 of the 10 lectures and 9 of the 11 tutorial sessions. This student initiated 37 distinct recording streams for 9 of the 10 lectures, and strongly agreed with the statement “when I listened to recordings, I tended to listen to the entire lecture”. Student C reported listening to a total of 10 hours of recordings, reading for 6 hours, surfing the web for 1 hour, and reviewing subject material for 2 hours. This successful strategy used a combination of lecture attendance and Lectopia usage, but little additional time outside of scheduled classes for reading or reviewing course material. Student D achieved a final result of 8-88, which was the top mark awarded. Student D attended all 10 lectures and all 11 tutorial sessions. The student initiated 7 recording streams for 3 lectures, two of which were in the second half of the semester. However, Student D strongly disagreed with the statement “when I listened to recordings, I tended to listen to the entire lecture”. In fact, the student reported listening to only 1 hour of recordings. At the time of the survey, Student D reported having spent 2 hours reading for the subject, 1 hour surfing the web, and 10 hours reviewing material. As noted in Table 1, half of all students estimated spending more time reviewing material for this subject than did Student D. On average, the cohort estimated spending significantly more time reading for this subject than did Student D. The successful strategy of Student D placed emphasis on attendance at lectures and tutorial sessions. This was supplemented with the equivalent of around 1 hour per week reviewing material outside of scheduled class time. The successful learning strategy employed by Student D placed little emphasis on recordings, reading, or the web. These results suggest that Student D paid attention and was likely to have been an active participant during lectures and tutorial sessions. Additionally, the average effort expended by Student D on reviewing subject material also suggests the efficient use of private study time in a manner that reinforced class-based learning. von Konsky, Ivins and Gribble 593 The importance of engagement It would be inappropriate to use the case studies from Students A, B, C, and D t o generalise about the educational efficacy of an individual learning strategy for all students. Taking data for the cohort as a whole, lecture attendance, practical attendance, and recording usage were not well correlated with final mark. This is despite the fact that most students felt that lectures and recordings contributed to learning. Final marks were clearly influenced by other factors. Physical presence during a lecture does not mean that a student is paying attention, synthesising new information in the context of prior understanding, or developing insights that will foster learning. Similarly, playing a lecture recording does not necessarily mean that learning will take place. Sitting in a room while a recording is playing, perhaps while simultaneously engaged in other activities, may lead some students to the incorrect view that learning must be taking place. They may feel that listening to complex material multiple times will allow it to “sink in”. For these students, repetition and rote memorisation is likely to be the principal strategy being employed. Ideally, students should be engaged in educational experiences and actively synthesising new information using a personal framework that builds on prior knowledge and scaffolds in preparation for that which is to come. Passive learning, in which knowledge is transferred from its source to the student, is generally less successful. Not only is retention more difficult, but passive learning generally does not develop the skills necessary to promote further inquiry or enable students to apply new knowledge in practice. Future work Measuring the level of student participation in lectures and tutorial sessions, and the utilisation pattern of online recordings and other educational material is an important first step in assessing the impact of these experiences on learning. Importantly, this study has shown that while surveys may summarise opinions regarding the efficacy of learning experiences, these perceptions are not necessarily correlated with actual learning. Future work should continue to correlate participation with actual measurements of outcome attainment, in addition to student opinions regarding the contribution of the various experiences to learning. Future work should also endeavour to measure the extent to which students are engaged during their participation, and the extent to which this impacts learning. Doing so should better enable successful learning strategies to be identified. Ideally, lectures should be interactive, presenting material in a manner that encourages students to work with the learning facilitator, to ask questions and to propose solutions to problems. In future studies, assessing the extent of student engagement during lectures could take several forms. For example, students could be required to submit class notes, written questions, or a brief handwritten synopsis immediately upon conclusion of a given lecture. Similarly, assessing the level of student engagement while using lecture recordings is also feasible. In part, this would entail measuring interaction with related online material while listening to recordings, and build upon the prior work of Brotherton and Abowd (2004). 594 Australasian Journal of Educational Technology, 2009, 25(4) Conclusions Making lecture recordings available online to students shortly after the event did not have a significant impact on lecture attendance. While the act of taking attendance during lectures may have encouraged attendance in some cases, attendance patterns were generally consistent with those of previous semesters. No direct correlation was seen between lecture attendance and final mark for the cohort as a whole. The pattern of lecture attendance was similar for students in all grade ranges, but passing students were more likely to supplement attendance at lectures with the use of recordings. Passing students used recordings with greater variation than did failing students. The extent to which students utilised recordings generally depended on the extent to which recordings were perceived to contribute to learning. No direct correlation between Lectopia usage and final mark was seen, but passing students listened to recordings with greater frequency than did failing students. Across all grade ranges, students tended to use recordings more frequently for lectures presented earlier in the semester, and for reviewing material before the mid-semester test and the final examination. Perhaps most importantly, this study shows that if students perceive that something is of value to their learning, they will tend to use it. This perception and prior history with similar experiences influence individual learning strategies, which were shown to vary widely from student to student. While listening to recordings may have influenced the final mark of some students, this could not be demonstrated conclusively. However, a review of individual case studies suggests that the level of engagement with any one learning experience is an important factor that is likely to impact learning, and represents an opportunity for further research. Acknowledgment The authors wish to thank Martin Hill, Digital Media Specialist, Information Management Services, Curtin University of Technology, for assistance in coordinating and deploying recordings on the Lectopia system. The authors are also grateful for technical support from Callum Fairnie and James Budworth, Technical Officers. References Allert, J. D. (2004). The effectiveness of innovative approaches to CSI: Comparing opinion to outcome. In Proceedings of the 27th Australasian conference on Computer science - Volume 26, pp. 151-157. Brotherton, J. A. & Abowd, G. D. (2004). Lessons learned from eClass: Assessing automated capture and access in the classroom. ACM Transactions on Computer-Human Interaction (TOCHI), 11(2), 121-155. Chang, S. (2007). Academic perceptions of the use of Lectopia: A University of Melbourne example. In ICT: Providing choices for learners and learning. Proceedings ascilite Singapore 2007. http://www.ascilite.org.au/conferences/singapore07/procs/chang.pdf Curtin University of Technology (2008). Curtin iLectures - About the iLecture system. [viewed 15 Oct 2008] http://ilectures.curtin.edu.au/ von Konsky, Ivins and Gribble 595 Fardon, M. (2003). Internet streaming of lectures: A matter of style. In Proceedings Educause in Australasia 2003, Adelaide. [verified 7 Sep 2009] http://www.caudit.edu.au/educause australasia/2003/EDUCAUSE/PDF/AUTHOR/ED031019.PDF Fardon, M. & Ludewig, A. (2000). iLectures: A catalyst for teaching and learning? In Learning to choose, choosing to learn. Proceedings ASCILITE Coffs Harbour 2000. http://www.ascilite.org.au/conferences/coffs00/papers/mike_fardon.pdf Felder, R. M. (1996). Matters of style. ASEE Prism, 6(4), 18-23. [verified 7 Sep 2009] http://www4.ncsu.edu/unity/lockers/users/f/felder/public/Papers/LS-Prism.htm Felder, R. M. & Brent, R. (2005). Understanding student differences. Journal of Engineering Education, 94(1), 57-72. [verified 7 Sep 2009] http://www4.ncsu.edu/unity/lockers/ users/f/felder/public/Papers/Understanding_Differences.pdf Gosper, M., Green, D., McNeil, M., Phillips, R., Preston, G. & Woo, K. (2008). The impact of web- based lecture technologies on current and future practices in learning and teaching. Australian Learning and Teaching Council. http://www.altc.edu.au/resource-impact-webbased- lecture-technologies-macquarie-2008 Humphrey, W. S. (1995). A discipline for software engineering. Sydney: Addison-Wesley. Humphrey, W. S. (2000). Introduction to the personal software process. Sydney: Addison-Wesley. Mathews, J. P., Haughton, N., Pisupati, S., Scaroni, A. W. & DiBiase, D. (2004). For an online course encompassing 'traditional campus students': How, where, and when students work and engage with the course material. Paper presented at Frontiers in Education, 2004. [abstract only] http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=1408567 McGarr, O. (2009). A review of podcasting in higher education: Its influence on the traditional lecture. Australasian Journal of Educational Technology, 25(3), 309-321. http://www.ascilite.org.au/ajet/ajet25/mcgarr.html Moss, N. D. (2007). Podcasting of lectures: Learning from download statistics. Paper presented at the 3rd International Pedagogies and Learning Conference. von Konsky, B. R., Ivins, J. & Robey, M. (2005). Using PSP to evaluate student effort in achieving learning - outcomes in a software engineering assignment. In Seventh Australasian Computing Education Conference (ACE2005), Newcastle, Australia. [verified 8 Sep 2009] http://crpit.com/confpapers/CRPITV42vonKonsky.pdf von Konsky, B. R., Robey, M. & Nair, S. (2004). Integrating design formalisms in software engineering education. In Proceedings of the 17th Conference on Software Engineering Education and Training (CSEET'04) [abstract only] http://doi.ieeecomputersociety.org/10.1109/CSEE.2004.1276514 Contact author: Brian R. von Konsky PhD (Curtin) MACS PCP, Office of Teaching & Learning, and Digital Ecosystems for Business Intelligence Institute, Curtin University of Technology. Email: B.vonKonsky@curtin.edu.au