taplin.pdf Australasian Journal of Educational Technology 2011, 27(2), 175-191 Students’ satisfaction and valuation of web-based lecture recording technologies Ross H. Taplin, Lee Hun Low and Alistair M. Brown Curtin University of Technology This paper explores students’ satisfaction and valuation of web-based lecture recording technologies (WBLT) that enable students to download recordings of lectures they could not attend or wish to review for revision purposes. The study was undertaken among undergraduates and postgraduates in accounting at an Australian university. In addition to traditional Likert scale questions capturing the extent to which students appreciate WBLT, we introduce new valuations such as the price they are willing to pay to access WBLT. The findings revealed that the majority of students rarely used WBLT and placed a very low monetary value on them, placing into question the high percentage ‘positive’ and ‘appreciative’ feedback found in this and earlier studies. Although most students appreciate WBLT when it is perceived to be free, most are unwilling to pay for access, suggesting they are not ‘truly’ valued. Importantly, while most students do not value WBLT, some students value them very highly, raising issues of equity and access not captured by average values across students. Students who regularly attend face to face lectures did not use WBLT as much, and those students who rarely used WBLT were more likely to accept a $50 reduction in fees if WBLT were not made available. It appears university administrations are correctly emphasising that WBLT should not be viewed as a replacement for traditional face to face lectures. Introduction Web-based lecture recording technologies (WBLT) are recording systems which digitally capture face to face lectures for delivery via the web (McNeill, et al., 2007). Atkinson (2009) notes that the term ‘web based lecture recording techonology’ is cumbersome to use and is usually shortened to ‘web based lecture technologies’ or ‘WBLT’ or substituted by brand names such as Lectopia. Students who are unable to attend or who wish to review the lecture can download the recording at a time convenient to them. This flexibility has clear advantages to many students and thus is popular among universities in Australia as a tool for delivering lecture recordings to students in ‘real’ time. It releases the institution from the need to provide lecture theatres capable of seating an entire student cohort (Brabazon, 2002) and caters for a variety of student learning styles (von Konsky, Ivins & Gribble, 2009). At Curtin University, WBLT is implemented with the iLecture system (see Appendix A for a description of iLectures at Curtin University). In Australia, 50% of universities use Lectopia (iLecture) to digitally deliver lecture materials (Lectopia, 2009). Certainly, in terms of iLecture use at Curtin University, the technology is trumpeted in the following terms: 176 Australasian Journal of Educational Technology, 2011, 27(2) At the time of writing over 630 units from Business, Nursing, Engineering, Humanities and other schools, multiple seminars and conferences have used or are currently using the system with almost 14,000 lectures, seminars and conference sessions recorded and have been viewed almost 400,000 times. The iLecture system has seen continued growth in usage by staff and continued dramatic increase in the levels of usage by students. We are now recording up to 200 hours of lectures and seminars each week. The number of lectures recorded in 2007 (3,300) is over 200% that of 2006 (1,500) and the number of lecture recordings viewed by students is up over 330% from 45,000 in 2006 to 150,000 in 2007. Note that these figures represent students selecting a format from a pop-up menu, clicking thru a copyright notice and finally viewing the hour(s) long audio or video stream, so it is a far more reliable metric than web page hits. These figures do not include podcast iLecture formats, so the real total is higher again (Curtin iLectures, 2009). There are, however, a number of concerns associated with the use of this technology (Fardon, 2003). It can result in what McKinlay (2007) calls the ‘vanishing student trick’, leaving lecturers with a depleted audience (Massingham & Herrington, 2006; McNeill, 2007). In a survey of staff at the University of Tasmania it was estimated that the use of WBLT resulted in a drop in attendance of between 10% and 33% (McKinlay, 2007). An associated concern for some academics is the belief that internal students who aim to utilise WBLT will not exhibit the self-discipline required to keep up to date with the recordings and other course work, and will leave it all until just before the exams (Woo, Gosper, McNeill, Preston, Green & Phillips, 2008). The idea that WBLT results in low attendance rates has, however, been called into question by other research including that by Phillips, Gosper, McNeill, Woo, Preston, Green & Preston (2007) and von Konsky et al (2009), who argue that other factors contribute to falling attendance rates. These include the changing lifestyles of student and their changed perceptions of the learning experience provided. In the past decade there has been considerable academic discussion of the notion that distance teaching and face to face pedagogies are converging (Trindade, Carmo & Bidarra, 2000; Rumble, 2001; Woo et al, 2008), with the differences between the two becoming less distinct due to the use of computers and the web in classrooms while, at the same time, many distance education institutions now provide dual mode courses where students must attend a certain portion of the course in face to face mode (Bryant, Kahle & Schafer, 2005; Woo et al, 2008). This links to an important consideration in distance education, the selection and use of an appropriate pedagogy (Naidu, 2007; Sammons, 2007; Henry & Meadows, 2008) and its resultant impact on the quality of such courses (Weisenberg & Stacey, 2005; Larreamendy-Joerns & Leinhardt, 2006; Daniel, Kanwar & Uvalic-Trumbic, 2008). Academic attention has focused on the rush by tertiary institutions to supply courses by distance mode and the ensuing poorly conceived courses being viewed as the “McDonaldization of higher education” (Rumble, 2001: 33); resulting in a “second-rate, impersonal education option” (Baggaley, 2008: 41). WBLT could be seen as contributing to this blurring effect and could be seen to fall within the distance education model. There are however some important differences. WBLT is not designed as a distance education tool, but rather as a support mechanism for students who are unable to attend lectures for a variety of reasons (Williams & Fardon, 2007). Generally, no time or money are invested in designing or modifying lecture materials for use in the non-contiguous learning environment, though research suggests this should be considered (Gosper, McNeil, Woo, Phillips, Preston & Green, 2007). The lecture is typically recorded and made available for students to access at a Taplin, Low and Brown 177 later date (Curtin iLectures, 2009; McKenzie, 2008). This is confirmed by Phillips et al (2007) who conducted a survey of 155 academic teaching staff from across four Australian universities and found 75% made no adaptations to their units to accommodate the use of WBLT. In an associated study, McNeill et al (2007) report that students suggest a number of ways by which the usefulness of recorded lectures could be improved. These include supplementing recordings with visuals and notes, speaking clearly and slowly, and ensuring that lectures begin on time and do not run over time, to ensure that the concluding minutes of lectures are not cut off. In general, research indicates that students learn best in interactive, collaborative educational settings (Craig, Wozniak, Hyde & Burn, 2009; Preston, Phillips, Gosper, McNeill, Woo & Green, 2010). In the distance education, many institutions go to great lengths to ensure that students are provided with opportunities for interactive communication with their teacher and peers, either via communications technologies, or through the provision of some face to face course components in what have been termed ‘blended courses’ (McElroy & Blount, 2006; Tallent-Runnels, Thomas, Lan, Cooper, Ahern, Shaw & Liu, 2006). Students generally participate in weekly tutorials and hence are provided with an opportunity to clarify any questions they may have about the course content and engage in discussions with their lecturer or tutor and peers. Overall, it has been shown that students tend to use the recordings to supplement the face to face lectures rather than to replace them (McElroy & Blount, 2006; McNeill et al, 2007; Williams & Fardon, 2007). Further, research shows that students generally consider that WBLT enhance their learning experience and improve the quality of their education. In a survey of 815 students using WBLT, Gosper, Green, McNeil, Phillips, Preston and Woo (2008) found that 79.9% felt that WBLT facilitated and enhanced their learning. Students particularly value lecture recordings for the opportunity they provide to review lectures and prepare for exams and, hence, they see WBLT as being important in helping them achieve better results (Williams & Fardon, 2007; Gosper et al, 2007; Gosper et al., 2008; McKenzie, 2008). This is confirmed by research conducted at Queensland University of Technology (Moss, 2007) which found that downloads of podcasts tend to peak prior to major assessments. More recent research by von Konsky et al (2009) confirms that students who supplement lecture attendance with recordings tend to be those who are successful in their studies. Similarly, Gosper et al (2007) report 76% of the students they surveyed had a positive experience of WBLT, while 66.7% considered it had helped them achieve improved results. Likewise, Williams & Fardon (2007) conducted a survey of 1,070 students at The University of Western Australia and found that over 99% of the students rated the recordings as either ‘essential’ or ‘very useful’. Further, many of the students who participated in the study believed all lectures should be recorded, with a number of them making a direct link between the fees they pay and the provision of recordings. In a survey of second year accounting students at Macquarie University, McElroy & Blount (2006) also noted that students expect value for money in the services provided by their tertiary institution, including the provision of lecture recordings. Woo et al (2008) found that staff perceived advantages for external students using WBLT, but questioned the extent to which these advantages applied to internal students. Woo et al (2008) also found that students were positive about the benefits of WBLT for their learning. Gosper et al’s (2008) mixed methodological approach to students’ perceptions of WBLT also found positive responses. 178 Australasian Journal of Educational Technology, 2011, 27(2) Regardless of age, gender, enrolment mode or attendance pattern, 76% of students reported positive experiences with WBLT almost always or frequently… 80% of students compared with 49% of staff agreed that WBLT made it easier for students to learn (Gosper et al, 2008; p. viii). Gosper et al’s (2008) results must, of course, be treated with caution. They chose not to survey students who elected not to use WBLT (perhaps, these students were voting with their feet), and it is unclear whether this sample design applied to staff members as well. The results of a series of vignettes and case studies (six in total, some described as ‘developmental’ and others described as ‘investigative’) conducted by Gosper et al (2008; p. 35) on top of their survey suggest that students were ‘appreciative’ of WBLT but again we must be circumspect about the research design of this phase. The effect of WBLT on student attendance at face to face lectures has been considered by Massingham & Herrington (2006). They found that while technology was becoming pervasive in education, its benefits were unclear, and that it was a poor alternative to face to face lectures. Students’ absenteeism, according to Massingham & Herrington (2006), was more likely to result from students’ attitudes to learning and motivation. The purpose of this research is to explore the value students' assign to WBLT. In addition to the traditional measures, such as the extent to which students believe WBLT assists their learning, we estimate the monetary value they assign to WBLT. This enables a comparison of the costs and benefits of WBLT on the same scale and hence allows a formal cost-benefit analysis to be conducted. The provision of iLectures or any other WBLT requires significant funding in terms of initial equipment costs and recurrent running costs. Making the lectures available online costs in the order of A$150 per lecture (Craig, Wozniak, Hyde & Burn, 2009:164). According to Krishnapillai (2010), recurrent costs are US$200 per hour, excluding the costs incurred on items such as utilities, professional fees, student opportunity costs, and monthly network expenses. In recent times, for example, the cost of iLectures at Curtin University includes: “Costs to install an iLecture Automatic Digitiser in an existing venue ranges from about $5,000 up depending on configuration and existing AV equipment in each venue” (Curtin iLecture, 2009). Hardware costs - LCD projectors, technological resource kits - are also formidable (Okada & Kambayashi, 2002: 314). While students may appreciate WBLT, currently they do not directly pay a price for this service, and are unlikely to be aware of the opportunity cost involved. The decision by a university to provide WBLT implies other services that may otherwise have been provided cannot be afforded. For example, providing WBLT may contribute to higher student to staff ratios, or less variety in courses offered, than would be the case if resources were not assigned to WBLT. If universities are to make sound decisions concerning the types of services they provide to students, then a cost-benefit analysis is desirable. As the literature stands, no one has done a study to estimate how much students are willing to pay or sacrifice for this new technology. Instead, this literature typically asks students whether they agree with statements like "do you value iLectures positively?". Students answer such questions under the assumption that they are free since they do not pay directly for their access. Past literature has also suffered from extremely low or non-reported response rates. Phillips et al (2007) invited 13,278 students with access to WBLT across four universities to complete their survey. They reported: Taplin, Low and Brown 179 The exact response rate of the survey cannot be determined as, although all students enrolled in those units were offered the survey technology, it is impossible to know the exact number of students in those units who actually used WBLT (Phillips et al, 2007: 856). Hence their 815 responses could represent a response rate as low as 6% if we wish to make inferences about all students with access to WBLT. While the purpose of Phillips et al (2007) was limited to exploring students who use WBLT, this could be as little as 6% of the student population. This same sample set was used by McNeil et al. (2007), Gosper et al (2007), Woo et al (2008) and Gosper et al (2008), with McNeil et al (2007: 4) providing exactly the same reason for non-determination of response rate given by Phillips et al (2007). Examples of studies not reporting response rates include: 172 students in the Faculty of Commerce at the University of Wollongong (Massingham & Herrington, 2006); 26 part time students in the Master of Science degree at Universiti Malaysia Sarawak (Hong, et al, 2003); 54 students from the Human-Computer Interaction group at the University of Victoria (Storey et al, 2002); and ten students from an entry-level Turkish language course at a mid-west university in the USA (Cagiltay, Yidirim & Aksu, 2006). Since people tend to agree to something that is free, especially if low response rates imply only the few students who see some value in (and therefore use) WBLT are surveyed, this study set out to conduct a utilitarian analysis to evaluate WBLT financially. Financial valuation of WBLT Knowing the value of WBLT serves a range of needs for students, administrators and other stakeholders, including the provision of useful information about acquisition, maintenance, pricing and replacement of WBLT and the improvement of the learning experience. These benefits lead to effective tools for planning of lessons, the prevention of excessive expenditure on infrastructure of WBLT by implementing better design process and mitigations methods, and savings in insurance costs. Standard accounting practices enable the cost of providing WBLT to be estimated. Under the Australian conceptual framework, it is possible to categorise WBLT as an asset: future economic benefits controlled by the entity as a result of past transactions. In the case of WBLT there are elements of asset tangibility (actual hardware infrastructure) and asset intangibility (intellectual property, copyright) which could be recognised by a university. Generally, there are many alternative valuation methods for measuring non-current assets, including historical cost, fair cost, fair market value, deprival value, current cost, net realisable value and replacement cost. These possibilities are summarised in Table 1. The historical cost has a number of shortcomings, including paying no heed to changes in prices and providing irrelevant and unreliable numbers. Current value methods would recognise the increasing value of WBLT by reflecting their performance and accountability in each period of reporting period. However, current value methods require reliable measurement techniques such as independent assessment, historic performance and sampling. Net market values normally refer to assets whose values can be determined on the open market. The service potential of WBLT is not readily captured by a sophisticated existing liquid market. Replacement 180 Australasian Journal of Educational Technology, 2011, 27(2) cost approximates the amount it would cost to replace an asset with a similar assets at current prices. The deprival value method of valuing WBLT could entail three distinct methods of valuation depending on the action taken if a university were deprived of this asset: current replacement cost method if the asset is replaced with the same or a similar asset; market value method if the asset is not replaced and is being held for sale; and net present method if the asset is destroyed and not replaced. These alternative measurement methods open up a great many possibilities for the recording of WBLT as non-current assets, providing important information to key university stakeholders. Table 1: Cost measurement techniques to WBLT (adapted from Brown & Boogaerdt, 2006) Measurement method Issue WBLT application Historical cost Cost of asset are calculated on original historical cost Technological and price changes make this method problematic Current value Recognises the increasing value of assets by incorporating current as distinct from historical value Enables an assessment of the existing service potential or future economic benefits of WBLT Net market value Requires self-generating and regenerating assets to be measured at net market value on the open market Service potential of WBLT is not readily captured by a sophisticated liquid market Replacement cost Calculates a compensatory value of assets Replacement cost is the actual cost to replace an item. The replacement cost is likely to be different from net market values. Deprival value Adopts three distinct methods of valuation if the asset is deprived. Enables possible WBLT evaluations given set scenarios The valuation of the costs associated with providing WBLT by a university in monetary terms is at odds with the non-monetary valuation of WBLT by students and staff in the past literature. We ask how a university can compare the financial dollar cost of providing a service with emotive responses by students to questions like “iLectures made it easier for me to learn”? We do not dispute that WBLT will be useful to some students but seek to compare the total benefit of WBLT to students with the total cost to a university (and therefore indirectly to students) of providing WBLT. Research methods The University of Western Australia’s WBLT known as iLecture from 1999 was superseded by the term Lectopia in 2006 and later rebranded Echo360 (CATL, 2010; Lectopia, 2010). Since WBLT is still commonly referred to as iLectures by students, this paper uses the term iLecture, especially in survey questions. Students enrolled in one of two accounting units were invited to complete a short survey. The first of these was a second year undergraduate unit in financial accounting (enrolment of 600) while the second contained Masters by coursework students (enrolment of 150 students). All these students had access to iLectures. At the end of the last lecture (May 2009) students were invited to participate in the project by voluntarily completing a one page survey (see Appendix B). The survey was Taplin, Low and Brown 181 intentionally kept extremely short so students could complete anonymously within 5 minutes to ensure maximal response rates. As an additional incentive, students who returned the survey were provided with a raffle ticket with one randomly selected student from each unit receiving a bookshop voucher valued at $200. In addition to measuring demographic variables that may explain why some students value iLectures more than other students, we measure students’ valuations of iLectures in eight ways. When the survey questions were pilot tested, a ninth valuation question proved too confusing for some students and so was abandoned. This question involved the discount in fees required as compensation if iLectures were not available, however many students saw this as an opportunity to receive a large amount of money rather than place a value on iLectures. These eight valuations of iLectures fall into three types. Firstly we ask traditional measures, on a five point Likert scale, of the extent to which students agree with statements suggesting iLectures are worthwhile. Secondly, we ask the extent to which iLectures are used. Thirdly, we attempt to gain a monetary value for iLectures. Finally, we also added an open ended question about how the student would be affected if iLectures were no longer provided, to capture more qualitative opinions that students may choose to express. Results The total enrolment in the two surveyed units was 750, however only 530 attended the final lecture when the survey was administered. A total of 211 surveys were completed, giving a response rate of approximately 40% of the students asked to complete the survey. Descriptive measures of student demographics The students from our sample generally rated the unit difficulty as average to moderately difficult (mean = 3.6 on a five point Likert scale from 1 = very easy to 5 = very difficult). On the four point scale of typical academic grades from 1 = pass to 4 = high distinction, 80% of students responded with 2 = credit or 3 = distinction. The average number of units studied was 3.6 with 70% studying the typical full time load of four units and 21% studying three units. 16% of the students were working full time, 52% part time and 32% were not working. 68% of the students indicated English was not their first language. Average attendance was 8.7 lectures, with 58% of students attending at least 10 of the 11 lectures. Australian accounting education typically contains a large number of international students. Curtin University is no exception, with a large number of full fee paying international students from China and south-east Asia. Descriptives of the eight valuations of iLectures In this section we present summary statistics for the eight valuations of iLectures. The first three are traditional measures as used by Gosper et al. (2008), where students responded with a degree of agreement on a 5 point Likert scale indicating whether they found iLectures improved their learning experience and results, and made learning easier. Summary statistics for the mean, standard deviation and percentage of students either agreeing or strongly agreeing to these questions are presented in Table 2. 182 Australasian Journal of Educational Technology, 2011, 27(2) Table 2: Descriptive statistics for three traditional valuations of iLectures (N between 208 and 209) Valuation Mean SD % agree 1. My learning experience in this Unit was made positive overall by iLectures 3.26 1.05 42.5 2. iLectures helped me achieve better results 3.19 1.02 37.5 3. iLectures made it easier for me to learn 3.32 1.08 46.9 5 point Likert scale: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree For each of these three questions almost half the students selected agree or strongly agree and the means are all greater than the midpoint of 3. Although variability is high (extreme responses of 1 and 5 were given for each question) these results indicate a high value was generally attributed to iLectures. These results are, however, weaker than those achieved in Gosper et al. (2008) who found 76% of students agreed that their experience of using iLecture in the unit was positive overall for their learning. This discrepancy in results is likely because Gosper et al. (2008) found only 39% of their students almost always or always attended face to face lectures, confirming their sampled students were those expected to be using and valuing iLectures. Our study only involved two accounting units so may be less generalisable to other disciplines, but captured a higher proportion of our population and hence may be more representative of this student population. In our study face to face lectures were valued even higher, with 81% of students agreeing or strongly agreeing that face to face lectures had helped them achieve better results in assessment, as well as making it easier for them to learn. Hence it is possible the results of Gosper et al. (2008) focus on students less able or willing to attend face to face classes, and this resulted in their higher appreciation of iLectures. Table 3 contains summary statistics for four alternative questions valuing iLectures (valuations 4 to 7). The first two relate to their usage during semester and for exam revision respectively. As expected, students indicate they will use iLectures during revision more than they used them during semester. This is most likely because iLectures are valued for revision purposes, especially by students who have already attended the face to face lectures, although it is possible that the higher mean only reflects an intention to use iLectures. Table 3: Descriptive statistics of alternative valuations for iLectures (all N between 206 and 208) Valuation Mean SD 4. Out of the 11 iLectures how many did you use? 2.97 3.13 5. How many iLectures do you intend to use for exam revision? 4.42 3.68 6. If iLectures required a download fee, how much would you be willing to pay per semester? 14.75 33.2 7. Out of the twelve tutorials, how many would you be willing to cancel to have access to iLectures? 2.09 1.61 The average download fee was $14.75 for the sixth valuation of iLectures (which is akin to a current value, see Table 1). While this may appear a small amount per student, multiplied over a class it attributes a significant monetary value to iLectures. For example, for a large class of 500 students it translates into a total valuation of $7375. Hence iLectures may pass this simple cost-benefit analysis, especially if Taplin, Low and Brown 183 implemented at a university wide level to many students. For a small class of 50 students, the total valuation is only $737.50. In this case a lecturer might argue this benefit is too small to be worthwhile. While the above means can be used to derive a total valuation for iLectures, the high variation in the valuations provided by students is also important. For example, 71% of students responded with $0 indicating they were not willing to pay anything for iLectures. Thus the majority of students place no monetary value on iLectures but a few value them highly. In practice, it is likely the cost of iLectures will be paid by all students equally (if not in higher fees then by a reduction of other services that could be provided). This raises important ethical and equity issues. These issues are discussed further in the Conclusions below. The seventh valuation of iLectures was an attempt to measure the value of iLectures in a fee neutral way by suggesting a substitution to fund iLectures by cancelling tutorials, a traditional service provided to students. This valuation of iLectures is a form of deprival value (see Table 1). On average, students were willing to cancel 2 of the 11 tutorials to make iLectures available. In monetary terms, this represents a valuation similar to the $14.74 download fee. To see this, suppose a tutorial has on average 20 students. Then the saving by cancelling two tutorials is 2/20 = 10% of the cost of one tutorial (the labour cost for a sessional tutor is approximately $100 per tutorial, and although this may be the major cost, it is not the only cost). This distribution is also skewed with 52% of students indicating they would not be willing to cancel any tutorials in order to access iLectures. Thus a majority of students also place no value on iLectures if the valuation is performed in units of tutorials. Our eighth valuation of iLectures involved a simple question to which students could either agree or disagree, “Would you be willing to accept a $50 reduction in unit fees if it meant iLectures were not available?” This is a hybrid form of both replacement cost and deprival cost (see Table 1).While providing less information than the download fee question, it was felt that this simpler question would be easier for students to answer since it did not require a numerical answer. 50% of the students agreed to this proposal, suggesting half the students would accept a $50 reduction in fees but no access to iLectures while the other half would forgo the $50 in order to receive iLectures. This suggests the median valuation of iLectures is $50 but the mean (and hence a total valuation) cannot be estimated from this question. However, it is clear that the students provide a higher valuation on iLectures when possibly receiving a discount on fees, than when asked to pay higher fees to access iLectures. This is unsurprising because most students are cost sensitive: an increase in fees to access a service is worse than a corresponding decrease in fees when access is denied. Increases in fees, like increases in taxes more generally, are unpopular. Correlations between valuations of iLectures Table 4 contains correlations between the eight valuations of iLectures. Questions are abbreviated but appear in the same order as in Tables 2 and 3 with the eighth valuation of iLectures “more$50” equalling 1 for students responding no (and 0 if responding yes) to the question “Would you be willing to accept a $50 reduction in unit fees if it meant iLectures were not available?” This coding ensures all eight valuations of iLectures are measured so that a higher value indicates a higher valuation of iLectures. The term “more$50” indicates that the students who value iLectures at more than $50 receive the higher value of 1 for this variable. Pearson 184 Australasian Journal of Educational Technology, 2011, 27(2) correlations are below the diagonal and Spearman correlations are above the diagonal. Since these are similar we refer to the Pearson correlations below. The first three traditional measures on a five point Likert scale are all highly correlated (r from 0.737 to 0.829, p < 0.001). This suggests these three questions are all measuring a similar notion of value. Although they refer to different aspects (learning experience, results and learning) the essence of these three questions is mostly captured with just one of the three questions. Table 4: Correlations (r) between valuations of iLectures (all N between 202 and 209) positive assess learn use revision fee canceltut more$50 positive 0.763*** 0.703*** 0.565*** 0.504*** 0.048 0.093 0.212** assess 0.790*** 0.814*** 0.504*** 0.485*** 0.041 0.130 0.308*** learn 0.737*** 0.829*** 0.551*** 0.582*** 0.031 0.062 0.365*** use 0.509*** 0.472*** 0.474*** 0.642*** 0.052 0.077 0.296*** revision 0.481*** 0.461*** 0.532*** 0.615*** 0.062 0.079 0.318*** fee 0.133 0.084 0.083 0.094 0.045 0.166* 0.097 canceltut 0.024 0.092 0.049 0.060 -0.033 0.097 0.023 more$50 0.208** 0.293*** 0.335*** 0.270*** 0.302*** 0.113 -0.010 Pearson correlations are below the diagonal and Spearman correlations are above the diagonal. *, ** and *** denotes significant correlations at the 0.05, 0.01 and 0.001 significance levels. The uses of iLectures, both during semester and for revision, are highly correlated with each other (r = 0.615, p < 0.001) and with the three traditional measures of value (r from 0.509 to 0.532, p < 0.001). Thus students who agree with the subjective valuations of iLectures tend to be the students who use, or intend to use, iLectures. In contrast, the monetary download fee and the valuation of iLectures in terms of number of tutorials were not significantly correlated with any of the traditional valuations or usage of iLectures. They therefore measure a different concept of value. In particular, students who agree more strongly with a qualitative statement that iLectures are valuable are not more likely to assign a higher monetary download price. Students will express a favourable opinion about the value of iLectures when they are free, but this does not necessarily translate into them being valued highly enough to be paid for. Students refusing a $50 reduction in fees in exchange for access to iLectures gave higher traditional valuations (r from 0.208 to 0.335) and use iLectures more during semester (r = 0.270) or for revision (r = 0.302). The different results involving the $50 reduction in fees and the extra download fee highlights the different reactions to potential increases and decreases in fees. Correlations between valuations of iLectures and demographics Table 5 shows the Pearson correlations between the eight valuations of iLectures and demographic variables. These demographics are Difficulty (level of difficulty of the material: 1 = very easy to 4 = very difficult), Grade (1 = pass to 4 = high distinction), Units (number of units studied; 1 to 5 = 5 or more), Language (1 = English is first language, 2 = English is not first language), Fulltime (1 = full-time, 2 = part-time, 3 = not working) and attendance (number of face to face lectures attended). The general trend in Table 5 is that the values attributed to iLectures are not significantly different for any demographic group, and this applies to all eight Taplin, Low and Brown 185 valuations of iLectures. The major exceptions are significant (p < 0.01) correlations between finding the material difficult and willingness to cancel tutorials to obtain iLectures (r = -0.193) and reject a $50 reduction in fees (r = 0.209). Interestingly, these correlations are in opposite directions, so students who find the material more difficult value iLectures more in regard to refusing $50 in place of iLectures, but value iLectures less in regard to the number of tutorials they are worth. This can be explained by students who find the material more difficult placing a higher value on tutorials. In particular, these results provide little evidence that iLectures are valued more highly by students who are working or attend class less frequently. Attendance is significantly correlated with only one of the eight valuations (use of iLectures). The small negative correlation (r = -0.141) suggests students who attend fewer face to face lectures use iLectures slightly more during semester. Table 5: Correlations (r) between valuations of iLectures and demographics Demographic positive assess learn use revision fee canceltut less$50 Difficulty (202-205) -0.106 -0.102 -0.015 0.055 0.059 -0.029 -0.193** 0.209** Grade (198-201) -0.023 0.034 -0.043 -0.123 -0.107 -0.066 -0.067 -0.045 Units (205-208) 0.067 0.099 0.045 0.000 0.029 0.001 0.036 0.108 Language (184-186) 0.159* 0.076 0.073 0.101 0.063 0.031 0.186* -0.046 Fulltime (205-208) 0.013 0.025 -0.034 0.046 0.042 0.012 -0.044 0.000 Attendance (205-208) -0.086 -0.077 -0.103 -0.141* -0.025 -0.040 -0.101 -0.004 Values in parentheses are ranges of sample sizes for correlations with the demographic variable. *, ** and *** denotes significant correlations at the 0.05, 0.01 and 0.001 significance level. Conclusion and implications Our results suggest that students, on average, value access to WBLT at $15 per unit per semester. This value enables individual staff and universities to perform a cost-benefit analysis to examine whether investment in WBLT is worthwhile. Since a final decision to implement WBLT depends on several factors, including economies of scale and how many semesters are required to pay for the high initial costs of installing equipment, we leave final decisions concerning to WBLT to readers. We suggest this $15 valuation, or an equivalent valuation using our methodology for the relevant student population, is essential information before reaching any decision concerning the use of WBLT. Our results differ from past literature such as Woo et al (2008) and Gosper et al (2008). We find students use WBLT less frequently and are less likely to agree to positive statements concerning the value of WBLT. We suggest these differences can be explained by differences in study design, types of students surveyed and different response rates. While we capture a more typical sample of students with a higher response rate, our results may not be generalisable beyond accounting or business students. While past literature may over-sample students who value WBLT, we under- sample students unable to attend lectures. We also find low correlations between the traditional questions where students are asked their level of agreement with qualitative statements that WBLT are useful, and questions that attempt to place a more tangible value on WBLT. These low correlations suggest there are different notions of the value of WBLT. We suggest students can agree with statements that WBLT are useful however place no monetary value on them. This brings into question the extent to which these traditional questions valuing WBLT should be used to justify decisions concerning the use of WBLT. We suggest a 186 Australasian Journal of Educational Technology, 2011, 27(2) cost-benefit analysis should be applied to any decision concerning the provision of services to students. Our results suggest WBLT are likely to pass such a test, at least on average when applied to the general student population if not for most individual students. Since WBLT are not usually proposed as a substitute for face to face lectures but as a supplement, our results viewed both within our study and in comparison with past studies raise important ethical and equity considerations concerning the provision of WBLT. In particular, most students appear to place little value on WBLT, however a small minority value them very highly. Further research is necessary to identify more carefully the students who value WBLT highly, and whether the gain to these students warrants widespread use of WBLT. Providing access to education for previously disadvantaged groups has significant value, however decisions to provide this access should be made with appropriate evidence. In particular, whether the benefits of providing WBLT outweigh the costs, and whether there are alternative ways of providing quality education to equity or other minority groups of students in a more effective and less costly way. Our survey also included a final open ended question “If iLecture was not available next semester, how would that affect you?” Most students responded briefly to this question. Out of the 130 comments made by students, 54% responded with sentiments such as “not much” or “no effect” and 26% indicated it would make their revisions harder or difficult if they could not understand the face to face lectures. This provides qualitative confirmation of our quantitative results. Most students place no or little value on WBLT. This is not inconsistent with students agreeing they are useful during the semester or anticipating they will be useful during revision at the end of semester. Services that appear free can be perceived as useful even if they are not used and are not worth purchasing. There are, of course, limitations to this study. First, unlike Gosper et al (2008) the views of staff were not elicited, an emphasis being placed on the perceptions of students rather than staff. The inclusion of staff was beyond the focus of this study because while staff may have qualitative opinions concerning the value of WBLT, they are not in a position to pay for WBLT. Nevertheless, future study might include both academic and professional staff responses to the questions posed in the survey, particularly in terms of comparison and contrast with students’ views. Students voices using qualitative data might also be worthy of future research as this study relies predominantly on the statistical analysis of survey data. Such voices may express both financially and non-financially bound motivations for satisfaction. Second, this paper did not elicit the views of students who did not turn up to the final lecture. A sticking point raised by Gosper et al (2008) was that some staff viewed WBLT as affecting students’ face to face attendance and, indeed, views by attending students on attendance issues were considered in our survey. However, a point emphasised by the respondents’ universities is that attendance was a necessary thing despite the presence of WBLT: It needs to be stressed that the iLecture system should not however be viewed as a replacement for traditional face-to-face lectures. On the contrary, good teaching practice involving engagement and dialog with lecture participants precludes a recording and playback system from being able to ever fully duplicate a live lecture (Curtin iLectures, 2009). Taplin, Low and Brown 187 Third, the proxies presented to students for alternative financial valuations were as simple as the traditional measures. Not only is placing a value on WBLT a complicated task, but placing a monetary value on them is potentially even more difficult for students. Measuring the value students place on WBLT, or any other aspect of the provision of educational services, is a complex problem deserving future research. Nevertheless, we found important patterns, chief of which was that most students do not place a high monetary value on WBLT. References Atkinson, R. (2009). Podcasting and recording revisited: A lecture theatre perspective. HERDSA News, 31(2). http://www.roger-atkinson.id.au/pubs/herdsa-news/31-2.html Baggaley, J. (2008). Where did distance education go wrong? Distance Education, 29(1), 39-51. Brabazon, T. (2002). Digital hemlock. Sydney: University of New South Wales Press. Brown, A. M. & Boogaerdt, H. (2006). Accounting for suburban tree information systems. Corporate Social Responsibility and Environmental Management, 13, 275-285. Bryant, S. M., Kahle, J. B. & Schafer, B. A. (2005). Distance education: A review of the contemporary literature. Issues in Accounting Education, 20(3), 255-272. Cagilitay, N. E. , Yidirim, S. & Aksu, M. (2006). Students’ preferences on web-based instruction: Linear or non-linear. Educational Technology & Society, 9(3), 122-136. http://www.ifets.info/download_pdf.php?j_id=32&a_id=658 CATL (2010). Lecture Capture System (Lectopia). Centre for the Advancement of Teaching and Learning, The University of Western Australia. http://www.catl.uwa.edu.au/elearning/lectopia/ Craig, P., Wozniak, H., Hyde, S., Burn, D. (2009). Student use of web based lecture technologies in blended learning: Do these reflect study patterns? In Same places, different spaces. Proceedings ascilite Auckland 2009. http://www.ascilite.org.au/conferences/auckland09/procs/craig.pdf Curtin iLectures (2009). Description of system. http://ilectures.curtin.edu.au/information/ Daniel, J., Kanwar, A. & Uvalic-Trumbic, S. (2008). Achieving quality in distance education. http://www.col.org/resources/speeches/2008presentations/Pages/2008-04-04.aspx Fardon, M. (2003). Internet streaming of lectures: A matter of style. In Proceedings EDUCAUSE Australasia 2003. http://www.caudit.edu.au/educauseaustralasia/2003/EDUCAUSE/PDF/ AUTHOR/ED031019.PDF Gosper, M., Green, D., McNeil, M., Phillips, R., Preston, G. & Woo, K. (2008). The impact of web- based lecture technologies on current and future practices in learning and teaching. Australian Learning and Teaching Council, Australian Government Department of Education Employment and Workplace Relations. http://www.cpd.mq.edu.au/teaching/wblt/research/report.html Gosper, M., McNeill, M., Woo, K., Phillips, R., Preston, G. & Green, D. (2007). Web-based lecture recording technologies: Do students learn from them? In Proceedings EDUCAUSE Australasia 2007. http://www.caudit.edu.au/educauseaustralasia07/authors_papers/Gosper.pdf Henry, J. & Meadows, J. (2008). An absolutely riveting online course: Nine principles for excellence in web-based teaching. Canadian Journal of Learning and Technology, 34(1). http://www.cjlt.ca/index.php/cjlt/article/view/179/177 Krishnapillai, M. (2010). The just-in-time lectures (Jitl) technology: Implications for small island states. In Third Pan-Commonwealth Forum on Open Learning, Dunedin, New Zealand, July. http://www.col.org/pcf3/Papers/PDFs/Krishnapillai_Murugan.pdf 188 Australasian Journal of Educational Technology, 2011, 27(2) Larreamendy-Joerns, J. & Leinhardt, G. (2006). Going the distance with online education. Review of Educational Research, 76(4), 567-605. Lectopia. (2009). About Lectopia. http://www.lectopia.com.au/ Lectopia (2010). Echo360 in Australia/New Zealand (ANZ). http://www.lectopia.com.au/ Massingham, P. & Herrington, T. (2006). Does attendance matter? An examination of student attitudes, participation, performance and attendance. Journal of University Teaching and Learning Practice, 3(2), 82-103. http://jutlp.uow.edu.au/2006_v03_i02/pdf/massingham_008.pdf McElroy, J. & Blount, Y. (2006). You, me and iLecture. In Who’s learning? Whose technology? Proceedings ascilite Sydney 2006. http://www.ascilite.org.au/conferences/sydney06/proceeding/pdf_papers/p87.pdf McKenzie, W. (2008). Where are audio recordings of lectures in the new educational technology landscape? In Hello! Where are you in the landscape of educational technology? Proceedings ascilite Melbourne 2008. http://www.ascilite.org.au/conferences/melbourne08/procs/mckenzie-w.pdf McKinlay, N. (2007). The vanishing student trick – the trouble with recorded lectures. [verified 18 Mar 2011] http://www.utas.edu.au/arts/flexarts/vanishing.pdf McNeill, M., Woo, K., Gosper, M., Phillips, R., Preston, G. & Green, D. (2007). Using web-based lecture technologies – advice from students. In Enhancing Higher Education, Theory and Scholarship, Proceedings of the 30th HERDSA Annual Conference Adelaide, 8-11 July. http://www.herdsa.org.au/wp-content/uploads/conference/2007/papers/p125.pdf Moss, N. D. (2007). Podcasting of lectures: Learning from download statistics. In Proceedings of the 3rd International Conference on Pedagogies and Learning: Meanings Emerging in Practice, Springfield, Qld. Naidu, S. (2007). Instructional design for optimal learning. In M. G. Moore (Ed.), Handbook of distance education. Mahwah, NJ: Lawrence Erlbaum Associates, pp. 247-258. Okada, A. & Kambayashi, Y. (2002). A web-based lecture video database system with flexible indexing method using action logs, In Advances in Web-Based Learning – Lecture Notes in Computer Science, 2002, Vol 2436/2002, 11-45. Springer, Berlin. DOI: 10.1007/3-540-45689-9_26 Preston, G., Phillips, R., Gosper, M., McNeill, M., Woo, K. & Green, D. (2010). Web-based lecture technologies: Highlighting the changing nature of teaching and learning. Australasian Journal of Educational Technology, 26(6), 717-728. http://www.ascilite.org.au/ajet/ajet26/preston.html Phillips, R., Gosper, M., McNeill, M., Woo, K., Preston, G. & Green, D. (2007). Staff and student perspectives on web based lecture technologies: Insights into the great divide. In ICT: Providing choices for learners and learning. Proceedings ascilite Singapore 2007. http://www.ascilite.org.au/conferences/singapore07/procs/phillips.pdf Rumble, G. (2001). Re-inventing distance education, 1971-2001. International Journal of Lifelong Education, 20(1/2), 31-43. Sammons, M. (2007). Collaborative interaction. In M. G. Moore (Ed.), Handbook of distance education (2nd Ed.). Mahwah, NJ: Lawrence Erlbaum Associates, pp. 311-322. Tallent-Runnels, M. K., Thomas, J. A., Lan, W. Y., Cooper, C., Ahern, T. C., Shaw, S. M. & Liu, X. (2006). Teaching courses online: A review of the research. Review of Educational Research, 76(1), 93-135. Trindade, A. R., Carmo, H. & Bidarra, J. (2000). Current developments and best practice in open and distance learning. International Review of Research in Open and Distance Learning, 1(1). http://www.irrodl.org/index.php/irrodl/article/view/7/342 von Konsky, B. R., Ivins, J. & Gribble, S. J. (2009). Lecture attendance and web based lecture technologies: A comparison of student perceptions and usage patterns. Australasian Journal of Educational Technology, 25(4), 581-595. http://www.ascilite.org.au/ajet/ajet25/vonkonsky.html Taplin, Low and Brown 189 Wiesenberg, F. & Stacey, E. (2005). Reflections on teaching and learning online: Quality program design, delivery and support issues from a cross-global perspective. Distance Education, 26(3), 385-404. Williams, J. & Fardon, M. (2007). Perpetual connectivity: Lecture recordings and portable media players. In ICT: Providing choices for learners and learning. Proceedings ascilite Singapore 2007. http://www.ascilite.org.au/conferences/singapore07/procs/williams-jo.pdf Woo, K., Gosper, M., McNeill, M., Preston, G., Green, D. & Phillips, R. (2008). Web-based lecture technologies: Blurring the boundaries between face-to-face and distance learning. ALT-J: Research in Learning Technology, 16(2), 81-93. http://www.informaworld.com/smpp/ ftinterface~db=all~content=a902152720~fulltext=713240930 Appendix A: iLectures The process of iLectures is described by Curtin iLectures (2009) in the following terms: A lecturer walks into their next lecture, turns the microphone on and delivers a lecture. An hour or so later, without any human intervention, an appropriately titled link automatically appears on the web page of that unit adding the just finished lecture to the list of all the lecture recordings for that unit. A student, logs into that unit's web page from anywhere in the world and clicks on this new link to bring up a window with a streaming high-resolution recording of the lecturer's computer synchronised with the audio from the lecture and watches it while typing notes in Word. Another student jumps on the bus and pulls out their iPod or mobile phone and discovers the latest audio and video podcasts of their lectures already downloaded and waiting for them to view. A video image can also be captured by the document camera in the lecture theatre which, like an OHP, projects a presenter's hand written or printed notes up on the big screen, but also automatically records a video stream as they write or notate those notes. No extra personnel or setup is required for this to occur in an appropriately equipped venue and the only possible change a lecturer needs to make from delivering a traditional lecture is to ensure they use the microphone and use the document camera or a Powerpoint slideshow instead of an OHP or whiteboard.This is now the reality here at Curtin University in 40 lecture theatres which have all been equipped with Automatic Digitisers as part of the Curtin iLectures Project. Staff using other venues can upload or send in an audio or video tape or DVD recording of their lecture or tutorial to Central AudioVisual Support (CAVS) in building 309 where the staff upload the recordings into the iLecture system. Lecture Enhancement not Replacement However, as an extension to traditional lectures and as a mechanism for improving the flexibility and reach of such, this system boasts many advantages. Students with sight, hearing and learning disabilities are also particularly enabled by the system with UWA totally eliminating the need to hire note-takers for their Equity students. Curtin iLecture System Status A team from Central Audio Visual Services is continually expanding the system at Curtin with 40 Lecture venues now equipped. with automatic iLecture systems. Professional development workshops and seminars for staff on incorporating the iLecture system into teaching and learning activities at Curtin have been running for a while and a Staff Guide for using the system is available from the iLectures website. Now that just about all main lecture theatres are equipped with iLecture syatems, smaller lecture rooms and case study rooms are next on the agenda. 190 Australasian Journal of Educational Technology, 2011, 27(2) Appendix B: Survey instrument For each of the following questions, please circle the most appropriate response Over the semester were you working full time, part time or not working? a. Full time b. Part time c. Not working How would you rate the level of difficulty of the material covered in this Unit? a. Very easy b. Moderately easy c. Average d. Moderately difficult e. Very difficult My academic grades are typically a. Pass b. Credit c. Distinction d. High distinction How many units are you studying this semester a. 1 b. 2 c. 3 d. 4 e. 5 or more Is English your first language? a. Yes b. No Out of the 11 iLectures how many did you use? 0-1 2-3 4-5 6-7 8-9 10-11 How many iLectures do you intend to use for exam revision? 0-1 2-3 4-5 6-7 8-9 10-11 Out of the 11 face to face lectures how many did you attend? 0-1 2-3 4-5 6-7 8-9 10-11 Please indicate your level of agreement with SA= strongly agree , A=agree, N=neutral, D=disagree, SD= strongly disagree My learning experience in this Unit was made positive overall by iLectures SA A N D SD iLectures helped me achieve better results SA A N D SD iLectures made it easier for me to learn SA A N D SD Face-to-face lectures helped me achieve better results SA A N D SD Face-to-face lectures made it easier for me to learn SA A N D SD Would you be willing to accept a $50 reduction in unit fees if it meant iLectures were not available? Yes No In order to provide iLectures, below are three alternative suggestions to fund iLectures If iLecture required a download fee, how much would you be willing to pay per semester? $0 $50 $100 $200 $300 $400 Other $ _________ Out of the twelve tutorials, how many would you be willing to cancel to have access to iLectures? 0 1-2 3-4 5-6 7-8 9-10 11-12 You could opt to not have iLectures and receive a discount on your unit fees instead. Circle the very minimum discount you would accept on your unit fees if it meant iLectures were not available. $0 $50 $100 $200 $300 $400 Other $ _________ Taplin, Low and Brown 191 If iLecture was not available next semester, how would that affect you? _____________________________________________________________________________________ _____________________________________________________________________________________ Authors: Ross H. Taplin, Lee Hun Low and Alistair M. Brown School of Accounting, Curtin University GPO Box U1987, Perth, Western Australia 6845 Email: r.taplin@curtin.edu.au, lee.low@cbs.curtin.edu.au, alistair.brown@cbs.curtin.edu.au Please cite as: Taplin, R. H., Low, L. H. & Brown, A. M. (2011). Students’ satisfaction and valuation of web-based lecture recording technologies. Australasian Journal of Educational Technology, 27(2), 175-191. http://www.ascilite.org.au/ajet/ajet27/taplin.html