Does distance e-learning work? A comparison between distance and face-to-face learners using e-learning materials Sara de Freitas and Gerald R Roberts Birkbeck, University of London email: S.deFreitas@bbk.ac.uk; Gerald.roberts@ud.pc.uk This study compares continual assessment data, intake numbers, retention numbers and final examination grades of a mixed cohort of face-to-face and distance learners against similar data from previous years where e-learning materials were not used in order to test whether e-learning materials can support the same quality and quantity of teaching and learning for both face-to-face and distance learners. The results for this cohort of learners demonstrate that: (i) distance e-learners score as well and sometimes better than face-to-face learners; (ii) face-to-face student numbers have increased; (iii) overall, student retention and student attendance have been maintained; (iv) final examination results have been maintained or in some cases improved; (v) lecturer workload was high, but not unmanageable, and it is clear how manageability can be improved. Introduction Birkbeck College was an early advocate of distance learning and the use of ICT (Information and Communications Technology) to support student learning, provide distance learning courses (fourteen courses are currently offered for distance learners including four through the University of London External Programme), provide online learning (Moss and Sansom, 2000), and use e-learning in courses more generally (School of Crystallography; School of Biological and Chemical Sciences and School of Computer Science and Information Systems). Providing e-learning materials are embedded into course work with due consideration of pedagogy, context and learner requirements, the benefits of using e-learning to support 69 Sara de Freitas and Gerald P. Roberts Does distance e-learning work? distance learners - or distance e-learning in this article - may provide greater flexibility for the learner (Cullen, Hadjivassiliou, Hamilton, Kelleher, Sommerlad and Stern, 2002; Reynolds, Caley and Mason, 2002). In particular, electronic course materials may offer benefits including ease of access by students 'anywhere and anytime' (DfES, 2003). There is a range of literature that explores the factors such as motivation levels and retention that affect learning at a distance or for online learning (see Inglis, 2000; Collis and Moonen, 2001; Eisenstadt and Vincent, 2000; Stephenson, 2001). Adding to this work, this paper focuses upon whether greater flexibility is compatible with high quality of learning provision in the absence of face-to-face contact between tutor and students with a particular reference to retention and completion rates. In the face of the growth in the numbers of distance learners in the United Kingdom from 150,000 in 1996/7 to 200,000 in 2000/1 (source: HESA), and indeed the pressure which many institutions are under to increase student numbers, many are turning to e-learning materials to support distance learning. While there is a significant body of work conducted in the United States that has found 'no significant difference' between 'technology-based instruction' and face-to-face learning in terms of learner outcomes (for example, Russell, 2001), there have been few similar studies in the United Kingdom. This article attempts to test the hypothesis that distance e-learning can maintain or increase the quality and quantity of teaching in a United Kingdom institution. The paper provides new quantitative data to allow comparison of the two forms of learning (distance and face-to- face) with a single cohort of students, providing a comparison between: (1) numbers attending, (2) retention numbers, (3) continual assessment grades, and (4) final examination grades. In order to provide a control for the study, wherever possible the paper compares this single cohort with cohorts from previous years where no electronic teaching material was used. The e-content used for distance e-learning at the School of Earth Sciences included a range of digital materials: • full-text documents; • supporting materials; • high-quality graphical materials (including photos, diagrams and movies with sound); • interactive links to course schedules; • links to online resources; and • links to other lecture notes in the series. The distance e-learning materials were created using Adobe Acrobat Writer™ which facilitated authoring of PDF files from other electronic media. In addition, whilst connected to the Web, students can access the electronic library, email and other Web resources using buttons placed on the PDF files. Due to the central mission of Birkbeck to reach people who work during the day, accessibility was a key issue in the mode of delivery of the materials; therefore e-learning materials were made available on CD-ROMs. This facilitated inclusion of high-quality photographs allowing the student to magnify images to study relevant details, just as a face-to-face student might magnify a specimen using a microscope or climb a hill to gain an overview of a geological scene. 70 ALT-] Volume 11 Number 3 The course material was provided to both face-to-face and distance learning students. To ensure that students received weekly feedback, a feature that was thought to be particularly relevant for distance learning students, the final examination requirements for the year 2001-2 were changed from previous years. Previously, final examinations consisted of a theoretical and practical examination. In 2001-2 the practical examinations were replaced by weekly-assessed practical tests - the marks from these were returned within three weeks (except in the case of one course to allow evaluation of the effect of this facility; see below). Three half-course units and one full-course unit from the B.Sc. Geology and B.Sc. Environmental Geology were offered with CD-ROM lecture material for the first time in 2001-2; the courses were also offered for distance learners for the first time. These four courses have been evaluated in this project in order to provide data to find out whether distance e-learning can maintain high-quality learning and whether e-learning materials are beneficial to both distance learners and face-to-face learners. The following courses were examined. • Introduction to Geology; (1.0 course unit: 20 lectures and practicals) complete on five CD-ROMs. Assessed work was marked each week and returned to the students with a provisional mark. • Structural Geology 1; (0.5 course unit: 10 lectures and practicals) complete on one CD- ROM. Assessed work was marked each week and returned to the students with a provisional mark. • Structural Geology 2; (0.5 course unit: 10 lectures and practicals) complete on two CD- ROMs. Assessed work was marked each week and returned to the students with a provisional mark. • Global Tectonics; (0.5 course unit: 10 lectures and practicals) complete on one CD- ROM. Assessed work was returned at the end of the course. The same students studied both this course and Structural Geology 2. Available data for these courses consists of entry numbers, attendance records and marks achieved by students. Method The method used to evaluate the mixed cohort of learners consisted of a graphical analysis of trends in the data. (1) To investigate the quality and quantity of teaching and learning across the cohort, the following is presented: • the number of students for several years of intake is compared; • the retention rates of students for several years of intake are compared; • a general survey of student entry comments made at admissions interview for a sub-set of the 2001-2 cohort is presented and analysed; • average marks for several years of final examination results are compared; 71 Sara de Freitas and Gerald P. Roberts Does distance e-leaming work? • the distribution of marks for several years of final examinations is compared; • mark accumulation during assessed course work is analysed including differentiation between (a) students who had previous degree qualifications at intake, (b) those who did not, and (c) distance learners; • a comparison of mark accumulation during assessed work and final examination results for two course units, where one course included weekly feedback and one did not; the same sub-set of the 2001-2 cohort took both courses. (2) To investigate whether there would be a decrease in face-to-face attendance if high- quality lecture notes and graphics are provided, the following is presented: • attendance records for several years of students are presented and analysed; • attendance records for students admitted in 2001-2 who stated at interview that they may only attend sporadically (using the lecture notes provided on CD-ROM for distance learning) are presented and analysed; • Finally, an analysis of lecturer workload will be presented, commenting on preparation of the e-learning materials and provision of feedback to the learners. Distance learning in Earth Sciences and the adopted method Learning in Earth Sciences specifically requires student access to high-quality visualizations of the processes and products of geological evolution. It also requires the development of practical skills, which allow students to recognize and interact with varied geological materials and geological scenarios. Traditionally, these have been provided by fieldwork and laboratory studies. Of these, fieldwork rightly remains the primary focus of geological teaching (as specified by the benchmark statement for Earth Sciences, Environmental Sciences and Environmental Studies, and the need for 100 days of fieldwork during a B.Sc. Geology course to achieve accreditation and chartered geologist status within the Geological Society of London). Traditional laboratory studies have always been regarded as essential because of the difficulty of delivering interactive practical scenarios with high-quality visualizations by any other method. However, the Web is increasingly providing a viable alternative for the delivery of high- quality materials, for example, the University of Leeds has delivered high-quality materials over the Web (see: http:llearth.leeds.ac.uk/peoplelbutlerlcontents.htm). Due to the nature of these large files, for example, a single photograph that is of sufficient quality to allow examination at a variety of scales (as would be the case with a student increasing the magnification of a microscope or climbing a hill to gain a different perspective on a geological field scene) is at least 10Mb in size when converted to a digital form. In the past, due to the slow download times for 56k modems, we considered that Web delivery impeded attempts at delivering geological laboratory-style studies for distance learning. However, in the near future the Web may provide a valid alternative for delivering Earth Science materials; as the numbers of students with broadband connections at home increase, the materials that are currently offered via CD-ROM can be easily uploaded to the Internet, when ease of access for all students can be assured. 72 ALT-] Volume I I Number 3 A method has been developed whereby large file sizes can be delivered via CD-ROM using a simple and freely available software package. Most computers can use Adobe Acrobat Reader7*1 to read PDF files. This reader software is present on most computers at the time of purchase or can be downloaded free from the Adobe Website. The user does not have to be connected to the Web to access lecture material and could be using a laptop anywhere. However, when connected, links allow easy access to the Web. The file sizes can be as large a several hundred megabytes for multipage lecture notes (the size of CD-ROM) yet access speeds are a few seconds for a page containing four 10Mb photos plus text. Adobe Acrobat Writer™ allows an author to add 'interactive buttons' to lecture notes which permits the student to open Web-links to electronic libraries or staff homepages, open files, go to specific pages in files, play movies and zoom in on photos and diagrams, conduct searches for words and display thumbnails to the pages of lecture notes. When printed the page format is controlled by the author rather than by the particular browser software chosen by the student. This allows the lecturer to control the exact format of printed pages seen by the student. The software allows inclusion of full colour graphics, is easy to use and compresses file-sizes to a minimum. A standard but individual 'house style' can be developed with ease to standardize lecture note styles from individual departments/institutions. The files have been placed onto CD-ROMs using a CD multi- copier purchased for this purpose by the School of Earth Sciences. The CD-ROMs are given to distance learning and face-to-face students as part of the normal course with no extra fee. The practical material for the four courses studied in the project was developed over a considerable period. In total the materials took about 175 days to develop; this was spread over a period of about four years. This broke down into: • twenty days to research and write the text for each 0.5 course unit; • thirty days to draw appropriate diagrams on computer for each 0.5 course unit; • five days to scan appropriate photographs for each 0.5 course unit; • fifteen days to paste the text, diagrams and photographs into the house format PDF file and to make the electronic links, buttons and interactive features for 0.5 course unit. Results Comparison of students intake for the years 1993-2001 To assess whether the introduction of e-learning materials influences student numbers, the number of students entering single honours B.Sc. Geology is plotted in Figure 1. After a decrease in student numbers following 1993 when many oil companies moved offices out of the London area following a change in taxation on oil exploration in the North Sea, student numbers enrolling for Introduction to Geology were between nine and seventeen. CD-ROM lecture material was advertised and offered for the first time in 2001. Intake rose in October 2001 to thirty-six, a number which included seven distance learning students. The total number of students entering the School of Earth Sciences in the year 2001-2 was fifty-three, a number which included students studying for joint honours degrees. 73 Sara de Freitas and Gerald P. Roberts Does distance e-leaming work? : Stuctent intake to Slngfe Honoura Geology 4 0 - 30 4—r 20 10 -ft 1923 1934 [995 1035 ISS7 1398 1933 2000 2001 Year ris^iS dL«jblei sftsi CD_R0M \&ctut£t rl'sstei Ml Retention rates of students for the years 1993-1994 to 2001-2002 To assess whether the introduction of e-learning material influences student retention rates, the number of students not taking the final examination for Introduction to Geology in the years 1994-2002 is compared with the number that enrolled for the course (see Figure 2). There is no clear pattern of increased or decreased retention over the period studied. However, it is noted that retention was worse in the year following introduction of e-learning material than in the previous four years, but similar to that of five years previously. Figure 2: Percentage net taking Intio to Geology Bam c u © Q. 40 - 20 - 0 - n n" 1 d m 1 1993 1994 1995 1935 1997 1993 1959 2CQ0 2001 Year Ma clear paitafn emerges ccncairing stutlsrr.3 who took Std fnal •3eok:-S'y s x For the intake of 2001 into Introduction to Geology, the percentage of students who took the final examination was similar for face-to-face and distance learners (see Figures 3a and b). However, the percentage of face-to-face learners who left the college was about double that of the distance learners (see Figure 3c). Thus, although both face-to-face and distance learners enrol and then do not take the final examination, a larger proportion of distance learning students in this sample remained with the college, deferring examinations to a later time compared with face-to-face students. 74 o it a n c3 . 0 ? rt = J-' ; o o 1 U 1.1 J U. U. a m W> n c 11. o r j ccoooo O CO i£) T fJ C! S d Q Q O on a T N § 1 . Is O O Q O O O O U! irt ^r rn CM »— u •zz-zz LM2 9681 561=: £86- ECO2 7Hv? LCuS •'X-ZZ 6661- assi • 9661 56SL .1? r n Q a o o o o a o »— ~ ii ' ^ .-^ t i w) -r ri -\j — •s - 5 •< = 665!. 366'.. i66'. 966i S66'. *-SS(. £66'. 2C-O2 tCOS CCffi 6661 3SS!. S661. rESt £661 111 "I f ' f — ̂ i ~i ii r I a ??. Ill Ji3 § 3 5 = 55 •A 1U I J a 3 ••y 8 a j il tS - ••y Sara de Freitas and Gerald P. Roberts Does distance e-leaming work? Survey of student entry comments made at admissions interview Students were questioned at interview to assess their backgrounds and reasons for enrolling. The students include occasional, non-degree students and joint honours students in addition to the mixed cohort of single honours face-to-face and distance-learners described above. The results are presented in the Table 1. Course Intake number BSc Geology . 29 Distance learning 10 BSc Environmental Science/Physical Sciences/ Geology and Physics 8 Non degree/occasional 4 BSc Environmental Geology 2 53 in total Table I: Intake number across the courses Summaries of the findings are as follows. • Seventeen students already had degrees out of fifty-three (32 per cent). • The average age of students was thirty-five. • Twenty-five of the students said they had never heard of Birkbeck, but found the college on the Web. • Ten students (who are not distance learners) said they were especially attracted by the use of e-learning material; nine of these would not have come to the School of Earth Sciences without CD-ROM material and all nine of these enrolled for the B.Sc. Geology. The most notable finding is that nine students indicated that they would not have enrolled without the provision of e-learning material. This was due to the expectation that they would use the distance learning option as a safety net if they could not attend all the lectures. Thus, these nine students together with the ten students who enrolled for distance learning, increased the student numbers by nineteen (36 per cent of the total intake of fifty-three) as a direct result of provision of e-learning material. Comparison of average marks at final examination for the years 1994-2002 To assess whether the e-learning material helps to maintain the quality of teaching and learning, as evidenced by final examination results, data for the years 1994-2002 were compared (the results for the year 1995 are not available and Structural Geology 2 and Global Tectonics run every second year, see Figure 4). The examination results for 2002 in Introduction to Geology showed the highest mean percentage on record. The examination results for 2002 in Structural Geology 1 showed the second highest mean percentage on record. The examination results for 2002 in Structural Geology 2 showed the second highest mean percentage on record. Marks for Global Tectonics were similar to previous years. External examiners confirmed there was no difference in the level of examinationss for these years. 76 ALT-J Volume 11 Number 3 Distribution of average marks at final examination for the years 1994-2002 To assess whether the e-learning material helps to maintain the quality of teaching and learning, as evidenced by distribution of final examination results, data for the years 1994-2002 were compared (the results for the year 1995 are not available and Structural Geology 2 and Global Tectonics run every second year, see Figure 5). It was judged that too little data was available for analysis using histograms so the standard deviation of the results was analysed. The examination results for all four courses show no clear pattern of variation in standard deviation. Mark accumulation through time during assessed course work To assess whether continual assessment associated with e-learning material influences achievement as evidenced by accumulated marks, the cumulative marks scored by students through time are presented (see Figure 6). Overall findings of the study on the Introduction to Geology showed: • there was no obvious difference in achievement between those with or without a degree upon entry; • distance learners have a high drop-out rate, but can achieve first-class marks; • the single first-class student improved with time evidenced by a better fit to the data using a second-order polynomial than with a linear equation. However, this highlights one of the few examples of clear improvement associated with continual assessment and weekly feedback; • most students who do not drop out follow linear learning curves, in some cases interrupted by flat portions to the curve caused by not handing in some assess- ments; • the lack of upwardly concave curves, which would indicate improvement in achievement through time, may be due to several factors: i. the early practicals may be too easy compared with abilities of the students (although external examiners state that the assignments are appropriate for this stage of a B.Sc. Geology); ii. the later practicals may be too difficult compared with abilities of the students (although external examiners state that the assignments are appropriate for this stage of a B.Sc. Geology); iii. students were not trying as hard in later practicals; iv. upward concavity may be hidden in stepped curves associated with not handing in practicals. The results of the Structural Geology 1 course are presented in Figure 7. The findings show that: • there is no obvious difference between distance learning and face-to-face students; • distance learners can achieve first-class marks; • most students who do not drop out follow linear learning curves, in some cases interrupted by flat portions to the curve caused by not handing in some assessments; 77 1—:— L "—""^ " .zocz CGSZ 3S6L 3SEL S-S6L — 1 1 L —t—i— I KOZ 8651 I XI- r I y O 'O O zooz i.OO2 'CCOZ 8661 .3661 S66l EEGL I! a § / < _ i ' - ' * ; " * - ' " " * > - - ' • ' • _ - . ' i i ^ ^ tooz ••cooz •66 6 1 • 8 6 6 1 . •iGGl .9661 .S66c KG l £66C 2000 1800 1600 KOO 1200 1Q00 eoo 600 400 200 0 - - JR M *™ 1*1**" r Jj g o * 16 i t or •liSJl'iS&'lV.'CI t Figure c b Inlrodutlbn toceclog1,* siuclenls 'Aillionl iicvj !•>:• Liilr,1 iniKllJjiicelc-jiii.r'iM m 1800 •Jf 1C0D g 1400 ~ 1200 > icon •3 BOO COO 400 —Y-r- • " . > « - . ... - • J- + 10 FKjuie mnoducibn 1000 ,„ 1B00 -K 1600 | MOO QJ 1200 •£ 1M0 S MO P eoo u 200 5 10 15 ixc-s or a i i « s a | viva k There is no otwious different^ in achievemem between those with or uiihout a degres upen entry. (Black symbols shew I st class marks; bls:k crosses shew p3ss marks. CistoncQ lesrners have a high drop-cut rate but can achieve 1st Class Msil.-s. Thosin^lo 1st Oass student improved with time. Other symbols :HDW nia-k zzcumulation for individual students! f l y u i * &\ and e: Inl IOCIIK 1 tan to Gcclogy: c;i uwlhc un» foi si i i : t n l X, wii li P.w dlffci enl hesii fII func I V:mi to li r.t-sl fcjjilo •Aiic-ll ier tl>? sludiMit ;iu: unui!;it a I mai b In o llne.'ii c< .u"t;doi:ii ing p;il! i. > 101X1 - • ^ GOU • "3 ^iJli - L> D. v - a a i 7 a « ' j 5a.»3r«-9.5.*ca uJ - n ^iifi? j f c * * ^ E ^ "'••ca V - 7S.3O73I- 12'X.11 p* - n Qd î7 * * * * * * '11 i 111 1'J i'O 0 i 10 l i 30 The ainglo Isc Class distance learner iniproved ftrcucjri trot: evidenced by 2 better fit to tho data using a 2nd cdor polynomial than tuth a linear ecjjition. Hotvever, this is CUB of the few examples of clear iir.prcvoment associated with continual assessment and weekly feec&sck. Most students who do net drop-oui follcw linear learning curves, in some cases interuptcd by flat portions to tho curve caused by not handng in some assessments. The lack of upwardly-concave curves, which would indicate improvcnicrit in adiiD'/etnent through time, may bo duo to several factcs: I f Tho early practical? may be too easy: 2} The later practiC3ls may he too difficult; 3) Students 3ro not trying as hard in later pra-ticals; 4 i Upward concavity may be hidden in stepped curves issaciatcd with not handing in practicsla. Sara de Freitas and Gerald P. Roberts Does distance e-learning work? Figu re 7a: Structural Geolcgy 1 : Distance learning students 900 aoo 700 600 500 400 3ao zao ICO 0 - ; • : * Y I ! • • - • • • • • " . « i t r - * • • • • : - • • • • - • •• * ••- - : < - : : • • ' : ' • : t • 1 I 5 10 Practials (Symbols represent Individ ual srudenls) Figure 7b: Structural Gsolcgy 1: Face to face students 1200 ^ 1000 C - aoo > n 6QC § 400 u 200 -•'-'; ••> t.. . . " ! t ' . ! i ' - : " " . . C - " : \ • ' - - " r " • ' • • • • ' - " . . •c t • ' - • ' - ' • :'-:;r | 1 • • • • i f -1 2 4 6 Practials (S)Tn bcls leptean t Indhidual sudaitsi 10 There is no obvious difference between the achievements of face to face- and distance learning students. Distance learning students cart achieve 1 st Class Marls. The lack of upwardly-concave curves, which would indicate Improvement in achievement through time, may tie due to several factors: 1) The early practical may be too easy: 2) The later practical may be too difficutt: 3) Students ate not tiying as ha rd in later pratticals; 4) Upward concavity may be hidden in stepped curves associated with not handing in practieals. (Black circles show I st class marks: black erases shew pass marks) • the lack of upwardly concave curves, which would indicate improvement in achievement through time, may be due to several factors, as mentioned above. Comparison of mark accumulation during assessed work and final examination The same students took the courses: Structural Geology 2 and Global Tectonics. However, Structural Geology 2 assessed work each week, which was marked and marks were handed back each week. Global Tectonics assessed work each week, but the marks were only returned at the end of the course. The results (Figures 4 and 8) show that: • ten out of eleven students achieved better marks in Structural Geology 2 during assessed work; • in terms of final marks, the mean mark in 2002 for Structural Geology 2 was 60 per cent (Standard Deviation 14 per cent) with a mean of 57 per cent for Global Tectonics (Standard Deviation 20 per cent); (see Figure 4). The above may indicate that handing back marks each week improves learning evidenced by marks achieved during assessed work. Alternatively, it may indicate that the practieals in Structural Geology 2 were easier than in Global Tectonics - although external examiners did not confirm this. The final examination results were not distinguishable for the two courses, indicating that marking and returning assessed work each week has not had an obvious effect in terms of increasing final marks. Attendance records for the years 2000-2001 and 2001-2002 To test whether provision of e-learning material affected attendance, records for the years 2000-1 and 2001-2 are compared in detail for the course Introduction to Geology (see Figure 9); less detailed analysis of the other three courses is also included (see Figure 10). 80 Figure aa Si 800 % 600 I 4t>0 "§ 200 3 o • Strut 2 a Global Tectonics 5 Praclfcal 10 Flguie Bb 4 0 0 ZQO £l • Etruc 2 « Global Tectonics 0 S Piactlcal 10 Flguta Be 8 0 0 GOO 400 200 0 ! & • • • • Strue 2 t) Global Tectonics 10 Practical Flguiefid 'fe 800 | 600 I 400 00 0 Struc 2 Tectonics 0 5 Practical 10 Figure Se Flguie Bf Figure Figure fill J * Struc 2 s Gicbst Tectonics 600 400 200 0 • Struc 2 u Global Tectonics BOO 600 400 200 0 • Struc 2 EI Global Tectonics • Struc 2 5 Practical 10 5 Piactlcal H 1000 E 800 | 600 J! 400 1 200 3 0 A • Struc 2 H Global Tectonics 800 600 400 200 0 • • • Struc 2 H Global Tectonics 0 5 Practical 10 S Practical 10 Ft] lire sk | 800 « 600 S 400 1 200 3 o • Struc 2 H Global Tectonics 5 Practical The aa/ne students took Structural Geaiogy 2 and Glcfcil TeetuiJes. Structural Cijulcvjry 2 Had assessed work eadi Meek wtilcli was marked and tiaiiderj b i t k nltli a n-iirk eatfi meek. QoLiat Tectaiilfs Juul assessed noik eaai week but the maiks were only returned &i the end of the course. 10 out ot 11 5tu. Mi3 _ _ u, r ' ^*t - K 3Z 3 JC • ^ 2pi ***' v "£ Tt n J » ' • <» ib i i ,u ••> 4& VTf a • * * 10 * * Week 15 20 25 (Symbols represent IndN'IdLQl stLctents) Figure 9b: Introduction to Geology: Patterns of individual student attendance 2301-20Q2 25 20 4 3 15 1 3 2 Sit'* r. ; : • F -*- E 3 -X-il i k e fe f • • * • X pi • 'm •JfrSr r - • • —-+—: • c • • -m-a 10 15 Week 20 25 (S>mbds represent Individual students) There is little difference in the pattern of cumulative weeks attended by individual students between 2000-2001 and 2001-2002. The average week's attendance is 12.2 (61 per cent) for 2000-2001 and 12.5 (63 per cent) for 2001 -2002. 82 ALT-] Volume 11 Number 3 Figure 100 -r a 9 0 " 01 80 - f eo- • ' ' • • : -: -"*:. '•'•'': TJ 2000-2001 2001-2002 Year Figure lGc:GlobalTectonics;'?i attendance • ' 9 0 - Si BO- S' 7 0 - r- so- ft 5 0 ' h! 4 0 - £ 3 0 - 2 0 - 10- n - ;»A:4? r 2000-2001 2001-2002 Year Figure 9 0 - « 80 - 10- Figur- 9 0 - a) 80 - ra 7 0 - ^ 6 0 - i . ' y ' ^ . ' >,-.•...̂ -_.'.:,.-.'-..;.-?, *i:iiS-i.l;~y,''\ ''•'•: }'•''•''-;''-:--- ''•.•• - : - ^ r ; - v " v . - ; ' ' > ' u ; v ^ - " ' - - - ; : - • / ; > • • ' • • • • • " • • ; > f ' ^ J V > • • • * • * ! • • • : • " • " - ' . • * . " ' • • • • . . ' • . ' " - . ; ' - • • . - : ; • • " • * • • . ! ' * M ' " " " • • • • 4 4 » • • • > : - ; - ; T J ; . • • ; - ; ; . : u:;,f« • • 4—4- • : •.',' ' i . . " • • ' • " • • " • • - • • » , , , . . . • , - • ; . . « • « —4—4- ' • ' ' . ' • • ' z ' • ' \ ' * • ' ' • ' - ' : ; ' A ' ' . " • . ' * - ; % * • • • " : - • ; . . - - • • • • . . • . - • . • • ^ : • - . - ; - • . • - ; ; ; • - • h :^ : ' • - : • \ . C - •• • ? " . • • " , • • - --.-. ' • •- -•- ' ( • - . • " " > • • • . . . : - " , • • — • 10 15 20 25 Weeks For the 8 students who anticipated using the distance learning option in part and attended at least part of the course, 2 of the 4 who completed the course show fiat portions to their curves indicating non-attendence. One student attended 17 out of 20 lectures with the other attending 14 out of ZO lectures. 4 students did not complete the course. The drop-out rate of 50% is similar to that of the course overall (42% for face to face; 50% for distance learning). 2 of the 8 students attended all the lectures. overall (42 per cent for face-to-face; 50 per cent for distance learning). Two of the eight students attended all the lectures. Thus, half of the students who indicated at interview that they would attend sporadically due to provision of e-learning material, attended every week. Lecturer workload In the October-December term the following courses ran: Introduction to Geology; Global Tectonics; Structural Geology 2. In the January-March term the following courses ran: Introduction to Geology; Structural Geology 1. Grading the assessment provided a large proportion of the allocated teaching time. It was found that grading and providing student feedback took about one day per week for each course. Thus three days per week were spent marking in the first term and two days per week in the second term. Further time was spent on the enrolment and administration of the distance learners, including setting up a Website for advertising purposes. An email, feedback service for students was run and widely advertised. Although it was expected that an email feedback service would provide the main method of feedback to students and would be very time- consuming, there were only two email queries in the whole year. 84 ALT-] Volume 11 Number 3 Summary of results and discussion The provision of teaching and learning material in an electronic format produced the following features: • an increased intake of students into the B.Sc. Geology course as it both allowed for the provision of distance learners and increased the face-to-face student intake - due to increased flexibility as perceived by the students; • no noticeable effect on the percentage of students who took the final examinations through the years 1994-2002; • sufficient quality ensuring that similar percentages of face-to-face and distance learners took the final examination in the year 2001-2; • sufficient quality ensuring that a higher percentage of distance learning students were retained by the college to enter the following year compared with face-to-face students; • maintenance and in some cases an increase in the achievement of students at final examination. This is supported by the fact that these materials were introduced in the year prior to the highest examination mean percentage on record for Introduction to Geology and second highest for Structural Geology 1 and Structural Geology 2; but clearly longer-term data are desirable; • no noticeable effect on the distribution of final examination results around the mean percentage as evidenced by standard deviations (see Appendix 1); • no noticeable effect on weekly or total attendance evidenced by a comparison of cumulative attendance growth curves for Introduction to Geology and percentage attendance levels for the three other courses (see Appendix 1); • very limited non-attendance in a subset of students who indicated at the initial interview that they anticipated utilizing the distance learning option in part even though they were registered as face-to-face students; • a high workload, as it took 175 days over four years of a lecturers' time to author the e- learning material; • assessed work was marked and returned on a weekly basis; findings from this showed no noticeable difference in terms of final assessed work marks or weekly accumulation rates for marks compared across students who (i) already had a degree prior to entry, (ii) no prior degree on entry, (iii) distance learners; • results from the assessment did not commonly produce concave upwards cumulative mark accumulation curves - indicative of weekly improvement in performance: most students followed linear mark accumulation curves although several other external factors that have not been analysed may have contributed to this; • higher marks and steeper mark accumulation curves compared to a course where marks were not returned each week although several other external factors that have not been analysed may have contributed to this - final examination results were very similar for the two courses; 85 Saru de Freitos and Gerald P. Roberts Does distance e-learning work? • assessment conducted on a weekly basis took up fifty days of a lecturers' time out of a hundred days available (two terms of ten weeks) for research, administration and teaching; • overall, the results are consistent with the hypothesis that the provision of electronic materials to support teaching and learning can maintain the quality and quantity of teaching. Most notably, findings of the pilot project run showed that: • overall student numbers have increased; • student retention and student attendance have been maintained; • final examination results have been maintained or in some cases improved. Other benefits of providing electronic materials for the lecturer included: • better organization of the teaching materials; • time-saving in the long run for lecture preparation time, although assessed work produced a high to unmanageable workload. Other notable aspects of the comparison include: • the benefits of weekly assessment were not borne out by any significant effect on student marks; • although email support was offered to the distance learners this was rarely used and did not provide a significant aspect of the course tutors' interactions. It may be that the large amount of feedback given to students each week cut down on the feedback they required via email. It may be more time-effective to provide email feedback with answers to the assignments provided on a separate CD-ROM. As a consequence practical examinations were reintroduced in the year 2002-3; practical work will now be self-assessed with the answers provided on a separate CD-ROM. The email feedback service will also be made available. Conclusions This study has shown 'no significant difference' in the quality of teaching and learning in the cohort regardless of whether the learner was a face-to-face or a distance learner (Russell, 2001). The introduction of distance learning material has increased student numbers. Our work provides a positive test of the hypothesis that distance e-learning can maintain or increase the quality and quantity of teaching in a United Kingdom institution, providing e-learning materials are well-considered, fit into the tutor's pedagogic approach and are clearly presented. References Collis, B. and Moonen, J. (2001), Flexible Learning in a Digital World: Experiences and Expectations, London: Kogan Page. Cullen, X, Hadjivassiliou, K., Hamilton, E., Kelleher, J., Sommerlad, E. and Stern, E. 86 ALT-] Volume 11 Number 3 (2002), Review of Current Pedagogic Research and Practice in the Fields of Post-Compulsory Education and Lifelong Learning, London: Tavistock Institute. Department for Education and Skills (2003), Towards an E-learning Strategy: Consultation Document, Nottingham: DfES Publications. Eisenstadt, M. and Vincent, T. (eds) (2000), The Knowledge Web: Learning and Collaborating on the Net, London: Kogan Page. Inglis, A., Ling, P. and Joosten, V. (2002), Delivering Digitally: Managing the Transition to the Knowledge Media, London: Kogan Page. Moss, D. and Sansom, C. (2000), 'Principles of protein structure: an established Internet- based course in structural biology', ALY-J, 8, 2, 29-39. Reynolds, 1, Caley, L. and Mason, R. (2002), How do People Learn?, London: Chartered Institute of Personnel Development. Russell, T. (ed.) (2001), The 'No Significant Difference' Phenomenon: A Comparative Research Annotated Bibliography on Technology for Distance Education, North Carolina State University, International Distance Education Certification Center. School of Earth Sciences, University of Leeds Web pages, see http://earth.leeds.ac.uk/ people/but/er/contents.htm, last accessed 13 November 2003. Stephenson, J. (ed.) (2001), Teaching and Learning Online: New Pedagogies for New Technologies, London: Kogan Page. 87