pitman Australian Journal of Educational Technology 1999, 15(2), 167-187 Internet based teaching in geography at Macquarie University: An analysis of student use A. J. Pitman, M. Gosper and D. C. Rich Macquarie University, NSW The design of a first year course with enrolments of about 400 students which is delivered using information technology via the Internet or Intranet is reviewed. The course is composed of a variety of information technology based applications including course materials, bulletin boards, email, quizzes, access to the World Wide Web and multimedia packages linked together using a single Internet browser interface. Access statistics through the entire course were collected hourly over the sixteen weeks of semester and separated on the basis of gender and final grade. Statistical analysis is presented which shows that different groups of students access the course in different ways. Performance in the course is statistically related to the number of times the student accesses the package. We find no evidence that males are advantaged or disadvantaged compared to females through the use of information technology although females appear to use the communications part of the course more than males. Based on the access statistics, we make recommendations on the components which seem to be well received by students and we note some components which will require more careful integration into a teaching program. Overall, the delivery of a first year course via information technology works well provided the components are carefully integrated into the course framework. Introduction Recent advances in communications and information technology (IT) are influencing the design and delivery of educational programs in higher education. The World Wide Web (WWW) has enabled the delivery of virtual learning environments to students in a more cost effective way. IT is capable of facilitating course administration, managing of student learning, delivering content, providing interaction and feedback, and supporting assessment and evaluation. Curricula can be developed incorporating a vast array of different learning resources and experiences designed to match specific 168 Australian Journal of Educational Technology, 1999, 15(2) educational contexts. Students can access databases, research and commercial software programs, computer based instructional programs, models, simulations, games, quizzes, virtual laboratory sessions and field trips, or communicate individually and collaboratively with staff and fellow students. This can all be achieved from both on campus or off campus, thereby providing students with viable choices in how, when and where they wish to study. Owing to the newness of the technology and the infancy of research informing the design and development of IT based teaching and learning resources, the full creative potential of IT and its capacity to enhance student learning has yet to be fully explored. For academic staff, IT based approaches provide the means to develop flexible study programs for both on and off campus students without necessarily increasing workloads. IT can enable the development of a different range of teaching resources to meet the needs of an increasingly diverse student body. IT can provide alternative, efficient and cost effective access to resources that previously were only available through on campus computer laboratories or through the library (eg. online newspapers, data bases, libraries, museums and journals). Finally, IT can provide efficient and effective alternative forms of communication with students (eg. bulletin boards and email). For students, the potential advantages are comparable to those for staff. Firstly, there is the potential for enhanced flexibility in both the timing and location of access to educational programs. Secondly, the diversity in curriculum resources has the potential to accommodate individual learning styles. Lastly, online communication via email means that students are not restricted by on campus consultation hours or face to face contact with staff. While these potential advantages exist for both staff and students, attempts to introduce change must be accompanied by monitoring and evaluation throughout all stages of the design, development and implementation of new programs. At Macquarie University we have been exploring the use of IT in first year geography teaching and we have been monitoring how students interact with IT in order to attempt to realise the potential that we believe IT offers. This paper provides a quantitative assessment of how students in one large first year course interact with a variety of IT components. The course includes a number of different IT based components which have been developed and systematically evaluated over a number of years. These components are delivered electronically via either an Intranet (on campus) or the Internet Pitman, Gosper and Rich 169 (off campus) from a common WWW interface. The impetus for such development has come from a variety of strategic, educational and economic concerns (Rich et al, 1997) including the need: to provide a flexible learning environment for students; to maintain or enhance existing educational standards in the face of declining teaching resources and; to diversify the information and learning resources available to students to accommodate different learning styles. The focus of this paper is to review this integrated approach and provide insights into how a large group of students use the technology in relation to four main themes which we believe have important strategic and educational implications for the University and individuals. The themes relate to: flexible learning; possible gender biases in the use of IT; relationships between the use of IT and student performance; and resource design, in that flexible learning programs are underpinned by a resource based learning approach to curriculum design (NBEET, 1997). In the body of this paper we will review the structure of the course and describe the different IT elements integrated into it. This will be followed by a brief assessment of some of the access statistics and a discussion of what we believe we can learn from these statistics in terms of the design and implementation of these types of teaching and learning packages. The course and teaching package In this paper, we focus on a basic first year course catering for approxi- mately 400 students named Global Environmental Crises (GEOS114). The course is taken by a wide variety of students with backgrounds in physical science, social science, law, education and business. It is available to both on campus students and distance education students who may be physically remote from the University. The course includes four modules which introduce students to environmental issues. There are two weekly lectures and a variety of IT based and non IT based resources which are integrated into single teaching and learning package. The teaching resources include print based materials in the form of a study guide, practical exercises and a textbook written to meet course needs (Aplin et al, 1995) and five IT based components discussed below. All these resources are coordinated through a single web interface and released to the students week by week. With this integrated approach, GEOS114 now supports a variety of learning styles including fully independent learning, and offers fully flexible delivery to both internal and distance students. Further details of the IT components and a rationale for their development is given below. 170 Australian Journal of Educational Technology, 1999, 15(2) Information and content provision Basic administrative information is provided on both the course home page and on paper. The amount of paper based information is gradually being reduced due to budget pressures. One advantage of the electronic version is the ability to update information in real time rather than having to wait for the yearly revisions of print materials. A variety of other material is provided through the WWW including data sets, assignment and lecture support material. Internet collection A collection of links to relevant WWW sites provides an online library which offers some advantages compared with the traditional library including: timeliness (the constant updating of good web sites is particularly beneficial); multiplicity (the ability to serve large numbers of students more or less simultaneously); and diversity (the ability to represent multiple views in contentious areas). The recent development of quality online journals has increased the value of giving students WWW access. Multiple choice quiz With around 400 students and no funding for tutorials it is impossible for the staff to provide formative feedback. We also want to gauge student performance; identify problems arising from poorly understood concepts; and reinforce students' knowledge base before subsequent lectures. To address these requirements we developed an HTML based multiple choice quiz with online feedback and the capacity to record student performance. To address these requirements we developed an HTML based multiple choice quiz with online feedback and the capacity to record student performance. The feedback is mostly informative, that is, it provides results plus reasons why the answer is right or wrong. While all feedback is beneficial, the additional explanations have been found to be particularly useful for independent students (Khine, 1996). The quiz also serves to encourage all students to access the course material electronically at least weekly and in doing so helps students to pace their learning. Pitman, Gosper and Rich 171 Multimedia packages We have developed two multimedia packages which form key components for our course. The two: Forests of Australia and Climate Crises can both be delivered via an HTML browser although Forests of Australia is provided on CD-ROM to off campus students due to its size. These two packages provide a framework to explain, revise and test knowledge and understanding of key issues. Text based explanations are supported by pictures, illustrations and short animations which have the potential to support learning (Burton et al, 1995; Clark and Pavio, 1991; Winn, 1994) specifically when the text and/or graphics and animations are explicative, learners lack prior knowledge of the subject, and the focus of instruction is understanding rather than repetition of factual information (Mayer, 1997). In addition, students can explore in more detail and at their own pace aspects of the course content not fully dealt with elsewhere. Email Email communication is an important application as it provides a means of communication for staff and students. Typically this is used for communicating on a one to one basis with staff and students as well broadcasting messages to all students. This has proven highly beneficial to both on and off campus students in facilitating communication. It permits students to ask questions as and when they arise. For the staff, a higher volume of contact can be handled by dealing with requests at a convenient time, from any networked location. Being able to circumvent problems by broadcasting messages relating to common requests can save valuable time and effort on the part of both students and staff. Bulletin board The bulletin board was intended to serve as a replacement for the standard tutorial session. There is considerable potential with this sort of technology to encourage collaborative learning (Katz and Lesgold, 1993) and involvement and interaction between groups of students who for a variety of reasons are unable to attend on campus, or are marginalised in conventional tutorials (Harasim et al, 1995). We have chosen to moderate all messages sent to the bulletin board because at the first year level there is the occasional incidence of inappropriate messages being sent. At this stage we have only used email and bulletin boards as an alternative means of communication and interaction. 172 Australian Journal of Educational Technology, 1999, 15(2) Summary These electronic resources are all delivered via a common interface (http://www.es.mq.edu.au/courses/GEOS114/) written in HTML and scripts and accessed through the NCSA user authentication protocol using an HTML browser. GEOS114 offers a relatively complete teaching and learning package ranging from a textbook to Internet resources. It is important to note that while different technologies underlie various aspects of GEOS114, to the student, all IT components of the course are accessed via a single web interface. This means that students only have to learn a single piece of software to be able to access all resources. The ability to present an IT based course through a common interface, accessible through the Internet is extremely attractive. The remainder of this paper discusses whether the approach is effective. Evaluation of the GEOS114 teaching package One advantage of using the Internet to deliver an undergraduate course is that the technology enables the capturing of very detailed access statistics. Over the 16 week semester (13 weeks of teaching plus a three weeks mid semester break) we collected hourly statistics from the first day of semester (week 1, July 22,1996) to the Sunday following the last day of semester (week 16, November 9, 1996). For this study we have analysed overall access, as well as weekly and hourly access statistics by final grade and by gender, providing us with an insight into how different groups of students accessed the components of the course. From this, we offer some preliminary analysis of the feasibility of providing flexible learning programs and guidelines for the development of Internet based teaching packages. In undergraduate teaching at Macquarie University, we employ the grading scheme of A (obtained by 33 students in GEOS 114 in 1996, or 9.5%), B (obtained by 49 students, or 14.1%), C (obtained by 205 students, or 58.9%), CQ (obtained by 22, or 6.3%) and F (obtained by 39, or 11.2%). The broad distribution of results is similar year to year. Of the 348 students included in this analysis, 152 (43.7%) were female and 196 (56.3%) were male. Access statistics over the semester Access to the entire course by the students is shown in Figure 1a and 1b disaggregated by grade and gender respectively. Pitman, Gosper and Rich 173 Figure 1(a) Figure 1(b) 174 Australian Journal of Educational Technology, 1999, 15(2) Figure 1(a) and (b) (page 173): Access statistics (individual accesses on Internet based pages) over the sixteen weeks of semester. The Y axis is in units of normalised percent of total accesses. This is the number of accesses for all students obtaining a particular grade or belonging to a particular gender divided by the total accesses for all students multiplied by 100. This is then normalised by the number of students obtaining a particular grade or belonging to a particular gender and smoothed with a seven day moving average), (a) for student grade, (b) for male and female students. There is no overall trend in access over time but there is significant variability through the semester. The major peak half way through the semester is due to the use of the Climate Crises package (the Forests of Australia resource does not cause the same peak since the whole multimedia package is accessed via a single html address). The three week minima in the data most clearly seen in weeks 11 and 12 is due to the mid semester break. In the following analysis, this pattern of use can be broken into three basic periods: the total period, the first seven weeks and the last four weeks. t score A B C CQ F A - 0.11 2.18 3.09 3.16 B - 2.09 3.03 3.09 C - 1.72 1.84 CQ - 0.16 Table 1: t score statistics calculated on the basis of grade anomalies (accesses by all students in grade x divided by the number of students in grade x minus accesses by all students divided by the total number of all students). Those in bold are significant at the 5% level. Figure 1(a) shows some interesting differences between the access statistics of students who eventually obtained different grades. Over the total period, it is not clear that those students who obtained an A grade accessed the Internet more than (say) a C grade students. However, if we analyse these data over the full period using t score statistics calculated on the grade anomaly (see Harrison and Tamaschke, 1984) we find that the access statistics by A students are significantly different (at the 5% level) from C, CQ and F students, but are not significantly different from B students. Likewise, B students access statistics are significantly different from C, CQ and F students. Due to the large sample size of C students compared to other grades, the significance of differences with respect to C grades should be interpreted with care. Table 1 shows the Pitman, Gosper and Rich 175 overall statistics for the total period. Note that there are no significant differences between the access statistics of C, CQ and F students. This separates the students into two distinct groups and indicates that the overall performance in the course is significantly related to effort. From Figure 1(a) it appears that A grade students accessed the material more than any other students during the first seven weeks and the remaining groups of students cannot be separated. A statistical analysis can be performed on just this seven week period, the results of which can be found in Table 2. t score A B C CQ F A - 2.18 3.50 3.80 4.27 B - 0.93 1.93 2.26 C - 1.44 1.80 CQ - 0.13 Table 2: As for Table 1 but for the first seven weeks of semester. This shows that A students do access the Internet more than any other group of student and that the differences in the access statistics are significant at the 5% level. However, the only other statistically significant difference in this seven week period is between B grade and F grade students. We cannot say, for instance, that B students access the course materials more than CQ students. The F grade students accessed the materials more than CQ students in the first seven weeks but both F and CQ grade students noticeably failed to access the material as much through the period of highest access. Whether this related to interest, a lack of patience in waiting for access to a computer during periods of peak load or other factors is unknown. Two further points are noteworthy: first the relationship between grade and access statistics can also be seen at the very beginning of the course where A and B grade students look over the computer based resources immediately, while it takes the students who eventually obtain a lower grade a week or two to get going. Second, during the mid semester break, the A grade students were accessing the material almost as much as the CQ students had in weeks 1 through 5. The final period of interest is the last four weeks of the semester where Figure 1(a) indicates that B students' access is anomalous. This might be interpreted as a group of students successfully "cramming" in the last couple of weeks before the examinations. Table 3 shows the results of the statistical analysis. 176 Australian Journal of Educational Technology, 1999, 15(2) t score A B C CQ F A - 1.98 0.23 0.66 0.49 B - 2.36 1.45 1.65 C - 0.67 0.43 CQ - 0.19 Table 3: As for Table 1 but for the last four weeks of semester. During this period, B students access the Internet significantly more than A students and C students. However, the peak in the access statistics from CQ and F students in week 14 means that there is no statistically significant difference between the B students and the CQ and F students in this four week period. A grade students access the course materials significantly less than B grade students (at the 5% level) and numerically less than CQ and F grade students. They had presumably managed to "get ahead" and were able to revise effectively without accessing the computer resources. B grade students invested a great deal of time in the two weeks just prior to the examinations. Separating the access statistics by gender (Figure 1(b)) shows that in week 1, females access the package twice as much as males. Other than this difference, there are no statistically significant differences (at the 5% level) between the access statistics separated by gender hence we find no statistically significant gender based differences in the use of IT in this course. Access statistics aggregated by day of week We can also view the access statistics by day of week. Figure 2 shows that the differences between the students obtaining different grades are relatively subtle. The A and B grade students show a more normal distribution where as the CQ and F grade students show a flatter pattern of access through the week resulting in significantly lower average access over the semester. An average A or B grade student is accessing the package on a Wednesday and Thursday about twice as much as a CQ or F grade student. Weekend access is about 5% of the total access, but there does not seem to be a relationship between weekend access and final grade with F students accessing the package as much as A students. Pitman, Gosper and Rich 177 Figure 2: Access statistics by day of week. (a) for A students, (b) for B students, (c) for C students, (d) for CQ students, (e) for F students, (f) for male and female students (male right hand side bar, note different scale on Y axis). The scale on the Y axis is as for Figure 1 summed over each day of the week over the sixteen weeks of semester. 178 Australian Journal of Educational Technology, 1999, 15(2) Finally, Figure 2(f) shows aggregated daily access for male and female students. Excluding Monday, where male students access the package rather more than female students, there are no obvious differences between the access statistics. It should be noted that the pattern of access through the week is largely determined by the pattern of lectures (Tuesday and Wednesday). Access statistics aggregated by time of day Finally we can view statistics covering time of day access (Figure 3). Approximately 66.4% of all access occurs between 10 am and 2 pm, while 90.6% of all access occurs between 9 am and 5 pm. While access outside the period 9 am to 5 pm appears quite small at 8.4%, these are students who do not require access to our on site facilities and are able to do the course where and when they chose to and reduce the stress on our facilities at peak times. There are differences between the graphs but most of these result from the generally higher access statistics by students obtaining higher grades. There are no clear differences between male and female students with both groups showing similar after hours access (Figure 3f). Key component utilisation Two key components in the course were the bulletin board and the email facility. We had been disappointed in the number of items posted on the bulletin board (345) and had considered removing it from the package. When we viewed access statistics we found that the information contained in the bulletin board had been accessed 2581 times over the 10 weeks it was available. This access can be broken down by final grade and gender (Figure 4(a)) and shows that A students generally read the bulletin board more than B grade students and that there is a gradual decline through C to CQ students (Figure 4(a)). However, F students access the bulletin board more frequently than CQ students. There are also differences in the number of times the bulletin board is accessed by gender with female students accessing it 9.65 times and male students accessing it 5.91 times on average. The reading of posted items on the bulletin board is encouraging, although we need to encourage more students to submit items (note the large difference seen in Figure 4). This is a new mode of communication for the majority of students and there may be some reluctance to communicate through an electronic means rather than verbally. Harasim et al. (1995) noted that this is a common experience for first time users. Pitman, Gosper and Rich 179 Figure 3: As for Figure 2 but for time of day. 180 Australian Journal of Educational Technology, 1999, 15(2) Figure 4(a) Figure 4(b) Figure 4: (a) the number of messages posted to the bulletin board (left bar) and accessed on the bulletin board (right bar) as a function of grade and gender; (b) the number of email messages sent as a function of grade and gender. Both figures are normalised by the number of students obtaining a particular grade or by the number of students in a gender class. In terms of email use, 420 email messages were read or sent over the semester. Overall, 49% were directed to the course convenor. Module 1 saw the greatest use (24.6%) with a drop off to Module 2 (8.7%), Module Pitman, Gosper and Rich 181 3 (14.1%) and Module 4 (3.82%). While we note that we would probably not have received 420 knocks on the doors of staff, the ability of students to send us questions, and the ability of staff to mail all students with clarifying information before too many asked the same question in person, significantly eased the convening of this course. There were relatively few differences in the use of the email by students obtaining different grades. Figure 4 (b) shows that while there was more utilisation of email by A grade students, B , C , CQ and F grade students all accessed email with a fairly similar frequency. Females sent, on average, 1.45 emails and males sent 1.05 which given the size of the course is a significant difference. Discussion This paper has examined the pattern of use of a first year Geography course delivered via the use of information technology by a large class of students. By collecting access statistics over the full semester and then separating these statistics by student gender and final grade we have identified some interesting changes over the period in question. Flexibility The majority of students still choose to undertake their studies on campus, from Monday to Friday and within the 9 am to 5 pm time frame. In 1996 a significant proportion (about 10%) of students chose to access the course utilising extensive WWW technologies at odd hours and at weekends from on and off campus. These students, wishing to access the course from off campus, were required to make their own Internet access arrangements and while there was some support and advice available, the onus was firmly placed on them to make this option work. We were encouraged by the percentage of students who have chosen to pursue this option. For the University this move towards off campus access could reduce demands on University equipment and space and help offset the development costs of an IT based teaching package. Performance There were statistically significant patterns in the use of the package by students. The total amount of access was an important indicator for success and overall, a student's grade was a function of the number of times they accessed the course. Of equal importance was the way in which access was spread over the semester and our evidence suggests that early access and engagement with course materials led to superior performance. A grade students accessed the materials early and 182 Australian Journal of Educational Technology, 1999, 15(2) consistently throughout the semester whereas F/CQ students were late to start and did not put in the recommended work time throughout the semester. There is quantitative evidence that there is a group of students who work really hard in the last few weeks and achieve a B grade. This group of students, while having the intellectual ability to perform creditably, may lack the organisational skills to plan their work schedules. Gender Although there are a lot of conflicting results about the relationship of gender and computer attitudes, research generally supports the findings that males have a higher self efficacy in relation to computer usage than do females (Busch, 1995; Whitley, 1997). This has implications for learning because the choice of whether to engage in a task, the effort expended in performing it, and the persistence shown in accomplishing it are influenced by the learner's self efficacy (Busch, 1995). Interestingly, our results reveal that females are just as comfortable with using IT as males and display similar access patterns. In fact, in the case of sending email and reading the bulletin board, there is a strong indication that females tend to be more likely to use IT. This is an important result which sheds further light on the issues of gender and technology. A number of studies conducted at school (rather than higher education) level have found that girls tended to use computers differently, focussing more on such activities such as word processing and collaborative work while boys liked game playing and competitive work (Kerr, 1996). Our results, indicate that when access is clearly defined and structured then there is no significant difference between the way males and females use the technology, except in the use of some of the communications where females appear to use the technology more than males. This may be due to a maturity factor with usage changing as students move from the school to the higher education sector or it may be, as Whitely (1997) surmised, that female university students are a self selecting group that are more comfortable with using computers. Use of resources We have used a number of different resources to serve different aims and objectives in this course. While the overall combination of resources meets our particular needs we are aware that this is our initial attempt at creating an online environment and modifications and additions will be needed. In general, modifications must be made in relation to the Pitman, Gosper and Rich 183 emerging body of research on instructional design. Emerging evidence suggests that the acquisition of basic skills, concepts, and factual information can be facilitated by the use of instruments modelled on computer based instruction programs with clear objectives, step by step tutorial sequences, review feedback loops, and program control (Fletcher-Flinn and Gravatt, 1995). In contrast, the development of higher order cognitive skills associated with analysis, critical thinking and problem solving abilities may be more suited to applications utilising hypertext based technologies (Jacobson and Spiro, 1995). The design of hypertext applications is both complex and challenging and the degree of learner control and the structure of the program requires a careful matching of cognitive requirements and student characteristics such as their approaches to learning and their ability to regulate and monitor their own progress. This strongly implies that the development of any online flexible learning programs and individual resources must be informed by a careful analysis of the cognitive requirements of the course as well as the characteristics of the student body. Communications package Email is extremely valuable for informing large classes of common problems, but also in terms of students feeling that they are able to communicate with the staff. Our experience indicates that if email is to be used by students it should be replied to within around 24 hours. Overall, we believe that if only one aspect of IT can be included into a course, email is the best one to chose because of the advantages and the lack of any disadvantages. Students benefit from articulating their knowledge by giving explanations and responses to questions (Katz and Lesgold, 1993; Chi et al. 1989; Chi et al. 1994). Although the Bulletin Board provided an electronic mechanism to do this through its capacity to support students' discussing issues, the expected benefits remain to be proven to be an effective instructional tool in our courses. We find that they are read but not contributed to very often by students (although non-contributors may still be benefiting from passive learning). The relatively low contribution rate may be because we included the bulletin board as a supplementary resource but did not integrate it well into the overall teaching and learning package as effectively as we might. To evaluate the relative merits of bulletin boards fully, they must be integrated into the rest of the course such that they become an integral component which the students see as advantageous to their personal performance. 184 Australian Journal of Educational Technology, 1999, 15(2) Conclusions This paper has described how we have implemented IT into a first year course in geography. We have described how IT is used and integrated into the course and how it is accessed. Access statistics accumulated over a full teaching semester have been evaluated. These demonstrate different patterns of student access when separated by eventual grade, but show no bias when disaggregated by gender. We have shown how the monitoring of access patterns can provide insight into student use. These data can also be used to help to identify problem students, to identify successful work patterns which can then be turned into strategies to assist all students in managing their own learning; to identify the resources that are not being used as intended; and to identify access patterns which can then be used to help plan for the provision of flexible learning infrastructure such as computer laboratories and off campus access. Overall, the kind of integrated approach we have presented here offers advantages over aspects of more traditional teaching. We do not argue that IT can replace all traditional methods in all circumstances, but it is possible to provide an IT based approach without compromising standards. There are also cost efficiencies in staff time resulting from the use of email and the ability to contact all students as soon as one student identified a problem. While there are cost efficiencies, there are obviously also costs in relation to developmental and technical support. Developmental costs are falling dramatically as new software comes onto the market, but hardware and human resource costs remain high. In essence, IT can give students the flexibility they want. IT gives staff better communication lines to the student body and access to all students equally and not just to those with the confidence to come and ask questions. When integrated into more traditional forms of teaching, IT can improve the educational experience and we now see the technology to be a valuable tool in enhancing the quality of teaching and learning. At the same time we recognise that, like a lot of early IT based teaching initiatives, the resources described in this paper were developed on the basis of intuitive beliefs about teaching and the capacities of the latest technologies, rather than being grounded in research on cognition and student learning. The results reported in this paper are encouraging, however, they are focussed on the evaluation of student access and usage. The question of whether and how the resources developed assist students in their learning is still largely unanswered. To answer this, we Pitman, Gosper and Rich 185 are now in the process of undergoing a major review and evaluation of the course to determine the effectiveness of the resources in assisting students to achieve the desired learning outcomes and to further investigate some of the issues that have emerged from our evaluations to date such as relationships between usage, performance and individual differences including motivation, attitudes towards IT, age, and gender. Acknowledgments We gratefully thank Kevin Chan whose expertise was fundamental to getting the technology to work. Sean Noble has also proved invaluable in helping us run and subsequently collate statistics from the course. We thank those who taught in GEOS114 in 1996 (Professor Peter Curson and Dr. Paul Beggs). Finally we thank the students of GEOS114 for their messages of thanks and of complaints which have encouraged us or simply helped point us in a better direction. References Aplin, G., Mitchell, P., Cleugh, H. A., Pitman, A. J. and Rich, D. C. (1995). Global Environmental Crises: An Australian Perspective. Melbourne, Oxford University Press. Burton, J. K., Moore, D. M. and Holmes, G. A. (1995). Hypermedia concepts and research: An overview. Computers in Human Behaviour, 11(3-4), 345-369. Busch, T. (1995). Gender differences in self-efficacy and attitudes towards computers. Journal of Educational Computing Research, 12(2), 147-158. Chi, M. T., Bassok, M., Lewis, M. W., Reimann, P. and Glaser, R. (1989). Self explanations: How students study and use examples in learning to solve problems. Cognitive Science, 13, 145-182. Chi, M. T., De Leeuw, N., Chiu, M-H. and LaVancher, C. (1994). Eliciting self explanations improves learning. Cognitive Science, 18, 439-477. Clark, J. M. and Pavio, A. (1991). Dual coding theory and education. Educational Psychology Review, 3(3), 149-210. Fletcher-Flinn, C. M. and Gravatt, B. (1995). The efficacy of computer assisted instruction (CAI): A meta-analysis. Computing Research, 12, 219-242. Harasim, L., Hiltz, S. R., Teles, L. and Turof, M. (1995). Learning networks: A field guide to teaching and learning online. Cambridge, Mass: MIT Press. Harrison, S. R. and Tamaschke, H. R. (1984). Applied Statistical Analysis. Prentice Hall, Sydney, Australia. 186 Australian Journal of Educational Technology, 1999, 15(2) Jacobson, M. J. and Spiro, R. J. (1995). Hypertext learning environments, cognitive flexibility and the transfer of complex knowledge: An empirical investigation. Journal of Educational Computing Research, 12, 301-333. Katz, S. and Lesgold, A. (1993). The role of the tutor in computer-based collaborative learning situations. In Lajoie, S. P. and Derry, S. J. (eds), Computers as cognitive tools. Hillsdale, New Jersey: Lawrence Erlbaum. Kerr, S. T. (1996). Towards a sociology of educational technology. In Jonassen, D. H. (Ed.), A handbook of research for educational communications and technology. New York: Simon and Schuster Macmillan. Khine, M. S. (1996). The interaction of cognitive styles with varying levels feedback in multimedia presentation. International Journal of Instructional Media, 23(3), 229-237. Mayer, R .E. (1997). Multimedia learning: Are we asking the right questions? Educational Psychologist, 32(1), 1-19. McInnis, C., James, R. and McNaught, C. (1995). First Year on Campus: Diversity in the initial experiences of Australian undergraduates. Canberra: AGPS. NBEET, Higher Education Council (1997). Quality in resource based learning. Canberra: AGPS. NCSA (1997a). http://hoohoo.ncsa.uiuc.edu/docs/tutorials/user.html NCSA (1997b). http://hoohoo.ncsa.uiuc.edu/cgi/ Rich, D. C., Pitman, A. J., Gosper, M. and Jacobson, C. (1997). Restructuring of Australian higher education: Information technology in geography teaching and learning. Australian Geographer, 28, 135-157. Winn, W. (1994). Contributions of perceptual and cognitive processes in the comprehension of graphics. In W. Schnotz and R. W. Kulhavey (Eds.), Comprehension of Graphics. Amsterdam: North Holland. Whitley, B. E. Jr. (1997). Gender differences in computer related attitudes and behaviour: A meta-analysis. Computers in Human Behaviour, 13(1), 1-22. Appendix: Technical provision of IT components There are similarities in the methods we use to deliver the various components of this course. To access the course, students have to be authenticated using the NCSA user authentication protocol (for details see NCSA, 1997a). This protocol is used to provide security for access, quizzes and the bulletin board. In order to use this protocol, we access the online database of students enrolled in the course and, via a Common Gateway Interface (CGI) script (see NCSA, 1997b), automatically create a list of students (via their unique student number) and automatically Pitman, Gosper and Rich 187 assign passwords (the student's surname). This permits access to the course by legitimate students. Accounts with special privileges exist for staff to send email to students and to moderate the bulletin board. Most of the specific course materials are authored in hypertext markup language (html, see http://www.w3.org/MarkUp/MarkUp.html). All materials are written in html and can be read using any major html browser. Navigation between pages is manually created, although in our second and third year courses, all navigation is created automatically. The software we use to allow email to be sent between student and staff relies on NCSA user authentication protocols. We can identify individual students without the need for individual user accounts. All messages are stored in a single location, but each message has an extra tag linking it to a specific student. We also use NCSA user authentication protocols to run the bulletin board. This permits us to identify individual students without the need to set up individual user accounts. All messages are initially stored in a single file and a moderator views and decides whether to accept, edit or delete. If the message is accepted, it becomes accessible to the students. The quiz was written in C and generates html which displays each question onto the screen, receives and processes the student's response. The program logs each quiz attempt by each student. The list of questions, possible answers and responses to each answer are stored as a text file, so that writing questions for the quiz does not require any programming knowledge. Since the students use the quiz via Netscape, it is fully accessible across the Internet. A. J. Pitman, Department of Physical Geography, Macquarie University, North Ryde NSW 2109, Australia. Phone +61 2 9850 8425 Fax +61 2 9850 8428 Email: apitman@penman.es.mq.edu.au M. Gosper and D. C. Rich, Centre for Flexible Learning, Macquarie University, North Ryde NSW 2109, Australia