freeman Australian Journal of Educational Technology Flexibility in access, interaction and assessment: the case for web-based teaching programs Mark Freeman University of Technology, Sydney Potential benefits in re-engineering higher education using Internet-based programs appear promising (Hicks 1997, Geoghegan 1996). The objective of this paper is to report the outcomes of a web-based teaching program trial involving 550 undergraduate business students. The program, called TopClass, provides for private email, public conference discussions, easy and consistent delivery of resources and progress testing via the WWW. Prior internet experience was very low. Students reported the program provided increased support for their learning, improved access and interactivity. 94% of students felt it would have a positive impact on their grade and 72% would like many or all aspects of the program used in other subjects. Academics seeking to utilise web-based teaching technology to enhance the teaching and learning environment would find these results helpful. 1. Introduction The objective of this paper is to report the outcomes of a web-based teaching trial involving 550 undergraduate business students. From this trial a set of critical factors are identified, consideration of which would be of benefit to future academics and academic managers seeking to utilise web-based teaching technology to enhance the learning environment. In the following section we review the relevant literature. Section 3 describes the actual web-based teaching trial. The evaluation results are described and analysed in section 4. Section 5 contains concluding remarks. 2. Literature review Figure 1 depicts an open systems view of the teaching and learning environment (influenced by the work of Ramsden 1992). From this we can infer the likely role of educational technologies like web-based teaching programs. 24 Australian Journal of Educational Technology, 1997, 13(1) A major goal of tertiary institutions and training firms is to ensure quality learning outcomes. Extensive research links quality outcomes with students applying a deep approach to learning. Biggs (1993) and others have shown that students who consider learning as better understanding reality are more likely to adopt a deeper approach. If the key is to have students apply a deep approach to learning and this is a function of their conceptions of learning, what critical factors should educators moderate? Figure 1 shows that deep learning approaches are a function of student conceptions of learning (Marton and Saljo 1997), teacher conceptions of teaching (Trigwell and Prosser 1996), and student and teacher perceptions of the specific teaching environment and general contextual factors (Ramsden 1992). These factors dynamically interact. For example, good student conceptions are a function of positive external support structures, conducive past experiences with them and positive anticipations in the current context. Students with deep approaches to learning in one context may exhibit a surface approach in a different context. Teachers who are aiming to change the way students understand reality, are more likely to design a positive teaching and assessment environment in the current context (Miller and Olson 1994). Good teachers attempt to stimulate students, be considerate of their needs, integrate technology and other learning resources into their instructional strategy and tie assessment to understanding. Like student conceptions, teacher conceptions are influenced by their own past experiences and current expectations with external support structures. The learning context, particularly in regard to external support structures, has changed significantly in the last decade. Pressure to do more and better with less has meant teachers are facing dramatically increased class sizes. As employers and students cope with their own financial pressures, and as the student population becomes more diverse (eg. prior knowledge and language), flexible learning has become a demand to which teachers are trying to respond. Academic managers are considering a range of options to redesign courses to meet these conflicting demands. In the hands of able people, technology can play a major role in enhancing the learning context. Course redesign with technology playing a major role, including the development of virtual courses and universities, amounts to a re- engineering of higher education. The converse is also true though - poorly applied technology solutions result in disastrous outcomes. Freeman 25 Learning outcome quality Approach to a learning task Surface Deep Rote learning Meaningful learning Teacher's approach to design of teaching and assessment context Students' conception of learning Lots of facts. Poor support. Stimulating, considerate teaching. Integrated technology, resources with students' prior conceptions. Assessment tied to understanding. Teacher's conception of teaching Information transmission Changing students' conceptions Le arn i n g e n v i ro n me n t o r c o n t e x t Increased quantity of knowledge or better understanding External support structures (past and present) • Financial support • Physical space, support, distance • Technological support (access, reliability) • Moral support (family, work, teachers, peers) • Other Internal factors (experiences and anticipations) • Language • Cultural experience and attitude • Educational experience and anticipations • Technological familiarity • Other Figure 1: Systems view of the teaching and learning environment. Figure 1 sheds some light on the key factors affecting the potential success of applying technology solutions. Teachers and students who have had positive experiences with technology and whose current technological infrastructure is supportive, are more likely to identify and use technologies that integrate with their teaching or learning context. If students' approaches to learning can vary with the context, then the role of the teacher in designing instructional strategies is imperative. McQuillan's (1994) extensive ethnographic study clearly supports the notion that most benefits of introducing technology are related to the changes in teachers' conceptions and the consequential changes to instructional strategy rather than the introduced technology itself. Positive motivational content, clear goals and expectations, meaningful learner activities and interaction with others are features of learning environments that encourage deep 26 Australian Journal of Educational Technology, 1997, 13(1) approaches to learning (Gibbs 1992). Technological innovations therefore should take these features into account in design and implementation. These potential benefits of educational technology are not always realised. Students or faculties with limited financial resources are less likely to successfully incorporate a technology into their teaching or learning context because they simply can not afford them. Similarly, technology can fail to deliver deeper learning if people have had bad previous experiences for whatever reason (eg. poor on-going support or excessive reliance on huge quantities of facts). As the systems view suggests, even positive experiences with technology cannot guarantee current or future success. Students who have successfully applied technology to achieve a deep approach to learning in the past may be severely discouraged from replicating such an approach if the introduced technology is one that is poorly supported, suffers frequent breakdown or is designed in a way that encourages surface approaches to learning. The Internet and World Wide Web applications are the latest in a string of technological innovations with which educators are experimenting. Figure 1 and the brief discussion above highlight the incredulity of any technology being a cure-all. The benefits and costs of using proprietary-authored web pages in a large business undergraduate context are discussed in Freeman (1996). Students appreciate access to materials and to a lesser extent other participants. Related are various flexible learning benefits - namely that such access can be at any time or place, at the pace of the user and independent of the client platform. Conferencing programs like First Class and Lotus Notes, provide a secure environment for managing an even greater amount of the teaching and learning process in an easier framework. Selinger (1995), Mason and Hart (1997) and others have identified various aspects from computer mediated communication. Positive attributes include better discussions, less isolation and a greater student focus. Negative aspects include reduced motivation from students experiencing technical problems and greater burdens on staff interacting with students. The more recent availability of web-based teaching systems offer the potential of further benefits in re-engineering higher education (Hicks 1997, Geoghegan 1996, Reeser 1997). TopClass and Learning Space are two programs that have recently become available. They provide for private email, public conference discussions, easy and consistent delivery of resources and progress testing via the WWW. Because they use the Freeman 27 WWW and popular browsers (eg. Netscape and Internet Explorer), they are platform-independent, use hotlinks and have an attractive easy-to-use interface. It was these purported benefits we wished to evaluate in a trial. Blackhurst and Hales (1997) describe various features of TopClass in considerable detail. Their trials involved only some TopClass features and few students. Kiser and Toreki (1997) trial with 128 chemistry students highlighted very positive student responses to a number of aspects, in particular progress testing with optional quizzes. Some 86% indicated a preference for introducing the web-based teaching program into other subjects with 96% indicating that it had a positive impact on their grade. 3. Web-based teaching trial The web-based teaching program, called TopClass, was trialed in the autumn 1997 semester at the University of Technology, Sydney. A small cross-disciplinary team from two different faculties was informally established five weeks before the semester start. TopClass had been previously trialed in a small teacher education context in 1996 using a Mac server and a large context trial was considered the next appropriate step. A 200 simultaneous user license Windows NT version was purchased in late 1996. While the two main participants - the teachers from Business and Education - had complementary skills and experience, the uniting factor was an enthusiasm to improve the teaching and learning environment. Over the five week period the system was set up, material designed, support material written and video taped, and academic and support staff trained. Anticipated teacher gains revolved around greater control and further efficiencies arising from utilising a custom-built program for the tertiary environment. Gains to students over the previous arrangement were also anticipated to arise from the custom-built nature of the program. Examples of these further improvements included a web-based email option and the ability to hold public discussions or conferences via the web. The latter were expected to enhance students' ability to communicate, particularly in relation to accomplishing team assessment tasks. Additional learning benefits were expected from the ability to set optional quizzes that could be automatically marked. Kiser and Toreki (1997) found that final exam failure was negatively correlated to students undertaking optional quizzes. A number of other contextual factors led us to believe that students would be positively disposed to a web-based teaching trial. Previous surveys had indicated that 85% had external access to computing facilities. Over 65% had purchased an individual copy of a computer based learning program (called LearnFM) even though it was freely available on the campus 28 Australian Journal of Educational Technology, 1997, 13(1) network. External internet access had grown from 8% to 18% in the three months to June 1996. By the start of semester 1997 (March) this had risen to 62%. By semester end (June) 69% had external access with 10% from their home, 22% from work and 37% from other. Some 550 undergraduate business students enrolled in Business Finance, an on-campus subject that ran for 13 weeks. Students could attend a two hour lecture each week and a one hour tutorial every fortnight. Students on one campus could choose to attend the day or evening lecture. Students enrolled at the other campus in the identical subject could attend the one lecture, available only in the evening, or travel the 20 minutes to the other campus on a different day and evening. Grades were assessed via a final exam (50%), a case study (30%) plus either optional quizzes or a research essay (20%). The majority of students chose the optional quizzes and many completed more than the minimum of two from four. Just over half the students were male and over a quarter were studying on a part time basis. 37% of students were under 20 years of age and another 37% less than 25 years. The majority of the remaining respondents were less than 35 years. 81% had prior experience with group work. By far the main anticipated benefit was the sharing of ideas of group work (50%). Web conferencing was meant to enhance this sharing. The least attractive aspects were controlling unequal work contributions (40%) and difficulties in coordinating meeting times (15%). The lack of student internet experience prior to the trial is described in Table 1. 58% had little or no experience with email, rising to 85% for internet relay chat (IRC). While nearly 30% perceived that their experiences with email or web browsers had resulted in a certain degree of competence, only about 10% perceived their experience and mastery had resulted in expert competence. Email Web browsers Bulletin boards IRC Expert 11% 9% 3% 3% Some competence 29% 28% 10% 7% Little experience 27% 31% 19% 17% No experience 31% 30% 65% 68% Missing 2% 3% 3% 5% Table 1. Internet experience A range of approaches was used to evaluate the outcomes. These included two focus groups, three student surveys, analyses of message threads and student results. The focus groups were the richest source of information Freeman 29 about students' learning. The focus groups and analysis of message threads looked at both student perceptions and at how they approached their learning using the web-based teaching program. The thrust of the student surveys was on students perceptions of the teaching and learning environment, including the web aspects, since, as indicated above these perceptions influence their approach to learning. The two main instruments for evaluating staff perceptions were the reflective diaries and analysis of the message threads. The results of these evaluations are discussed in the following section. 4. Results 4.1 Student perceptions While the richest data about student perceptions was the focus groups, these qualitative results are discussed in the context of the more quantitative results. Student expectations and realisations of web-based teaching can be gauged from the pre-course survey (n=455) which was replicated (n=258) in the final weeks of the course. The lower numbers of students responding to the post-course survey reflected the fact that some students were using learning resources other than lectures and were therefore not present in the lecture when the survey was handed out. The focus groups confirmed that students were using a range of resources to tailor their individual learning needs, and for some this meant not attending lectures. Students commented positively about having a range of resources and activities. Greater opportunities to access information and interact with material or people helped students accept responsibility for their learning and enable them to do it more flexibly. The results of the two surveys are in Table 2. Agree Neutral Disagree Pre Post Pre Post Pre Post Easy to learn 47% 85% 36% 10% 17% 5% A productive use of time 76% 80% 19% 13% 5% 7% Friendly to use 33% 71% 37% 20% 19% 9% Table 2. Expectations and realisations of using the web-based teaching technology. Pre-course survey n=455. Post-course survey n=258 Despite the relative lack of experience with the Internet in Table 1, students expressed positive perceptions of using the technology in Table 2. The majority of students anticipated that the technology would be easy to learn (ie. 47%) and a productive use of time (ie. 76%). These expectations were more than realised in the post-course survey with in excess of 80% agreement to both questions. The large minority that expected the 30 Australian Journal of Educational Technology, 1997, 13(1) technology to be friendly (ie. 33%) had doubled by the post-course survey (to 71%). Another aspect of the post-course survey entailed replication of questions used by Kiser and Toreki (1997). These aimed to isolate various other aspects of the web-based teaching technology. Usage can be gauged from Table 3 below. Section 3A appears to replicate Kiser and Toreki. The main reasons for accessing TopClass were students wanting to do the optional quizzes (26%) and download (23%) past exams (compared to Kiser and Toreki's 24% and 28% respectively). Because the optional quizzes were open-book and the final exam provided formulas, the questions focused on testing common students misconceptions rather than remembering facts. This demonstrates that students were taking a deeper approach to learning. Staff-student interactions appeared an important addition to the learning process in this trial (21%), particularly because the coordinating academic moderated them. By comparison, private email communication with tutor or fellow students was not a popular motivation in this trial or in Kiser and Toreki. Discussions were more valuable because students could gain greater assessment and learning benefits by submitting a question or by lurking on other's questions, and then reading the answers of the one who sets the assessments. Because a significant proportion of questions related to common misconceptions, students were again demonstrating a deeper approach to learning. Sections 3B and 3C focus on frequency and intensity of usage respectively. In the current trial 38% of students used TopClass regularly. While other usage patterns (in 3B) reflect those found by Kiser and Toreki, this feature stands in stark contrast to their reported 8% who used TopClass on a regular basis. Section 3C describes the greater intensity usage in the current trial. 30% of respondents used the program more than 20 times over the semester and 6% less than 5 times. Kiser and Toreki report 55% of their respondents used the program less than 5 times over the semester. High usage, particularly when such usage is largely optional, indicates a cost-benefit perception. Three plausible explanations can be postulated from Figure 1. The high and more regular use in our trial could reflect differences in internal factors. Australians generally respond very quickly to technology innovation as shown in the high penetration of mobile phones, automatic teller machines, video cassette recorders and PCs. Such response may reflect more positive past experiences with technology generally as well as familiarity with web technology specifically. This is probably even more the case with the sample in this trial because they were Freeman 31 business students. Business students would be more likely to anticipate using computers in their course and subsequent profession than say an arts student. 3A. In what ways did you use the website this semester (check all that apply) Frequency % To do the optional quizzes 228 26.4 Download copies of previous exams 196 22.7 To download copies of extra tutorial Q&A 135 15.7 To check out the Q&A in the discussions 185 21.4 To send messages to my tutor 83 9.5 To send messages to my team members 38 4.4 3B. How frequently did you use TopClass? On a regular basis 99 38.4 Occasionally with no set pattern 47 18.2 Whenever I had free time 46 17.8 Only to do part of an assessment 44 17.1 Just before quizzes were due 17 6.6 To communicate with my team members 1 0.4 Missing 4 1.6 3C. How much did you use TopClass over this semester? Less than 5 times 16 6.2 6 to 10 times 63 24.4 11 to 15 times 48 18.6 16 to 20 times 49 19.0 More than 20 times 77 29.8 Table 3: Usage patterns (post-course survey n=258) Second and related to this first reason, higher and more regular usage can arise because students in our trial have better external support structures. Very high external access to PC and internet in our trial (but unavailable in Kiser and Toreki) must make higher and more regular usage possible. This probably relates to the higher number of part-time students who can either use a computer at work or afford to have their own at home. Third, usage differences could be due to influences of the teacher in the design of the teaching and assessment context. Dealing first with the assessment design. In that students were only required to use TopClass once if they chose the research essay assessment option, namely in relation to self and peer assessment of the case study, it is unlikely that assessment per se caused higher usage in this trial. Students choosing the optional 32 Australian Journal of Educational Technology, 1997, 13(1) quizzes over the essay were required to access TopClass a minimum of three times. In as much that students perceived that the web resources helped them understand common misconceptions better and thus perform better in assessment, then we can conclude that TopClass usage was indirectly affected by assessment. But the latter relates to the other aspect over which teachers can influence usage, namely in the design of the teaching environment. Making available and integrating more technological and human learning resources can encourage students to take a deeper approach to learning. In our trial, with a diverse student population and staff teaching on multiple campuses, there is already a disparity in potential access to human and technological resources. For example, financially secure students who live near the campus, have supportive families and study full time at the same campus that the academics are mostly located, have advantages over part time students who have pressure balancing their work life, family commitments and their studies. The former have a much greater chance of accessing lectures, materials in library or closed reserve and interacting with staff. In such an environment, a well designed web-based teaching technology can have a relatively larger effect and encourage a deeper approach to learning. Quizzes Coursework Discussions Frequency Percent Frequency Percent Frequency Percent Useless 5 1.9 0 0.0 13 5.0 Quite useful 75 29.1 39 15.1 104 40.3 Very useful 93 36.0 100 38.8 85 32.9 Extremely useful 78 30.2 112 43.4 42 16.3 Table 4: Perceived usefulness by students of web-conferencing aspects. The post-course survey (n=258) included the phrase 'when they worked'. This was deemed necessary because just prior to the survey, the TopClass database had become corrupted and was unavailable for over two weeks being cleaned by the software developers. The problem was caused by a bug in the TopClass release version 1.2 under high load with the testing feature. The developers were very helpful in cleaning the database and providing a patch for the bug in version 1.2.1 Student perceptions of the relative usefulness of these different TopClass aspects are described in Table 4. Given that the surveys were conducted in the lecture, one could expect a bias away from technology resources in favour of human resources. Table 4 indicates that less than 2% felt that the Freeman 33 quizzes were useless. Not one student indicated the Coursework material was useless and only 5% felt Discussions were useless. Interestingly, a significant majority of students (ie. 67%) felt that the availability of an anonymous account encouraged interactions in the threaded discussions. Given the trial objective was to evaluate effectiveness of the different features of the program in the overall teaching and learning context, a focus group and an overall course evaluation (n=285) were undertaken in the last week of lectures. Another focus group was undertaken five weeks after the end of the semester. The content of the message threads was also examined. Table 5 summarises the overall positive perceptions, supported with data from the course evaluation in the final column. Students appreciated the enhanced access to information most, the improved opportunities to interact with learning material, and the improved opportunities to interact with people who would support their learning. Students in the focus group articulated that these aspects of the learning context aided the development of their understanding of business finance rather than simply assisting them to rote learn facts for an exam. As such, deeper learning was encouraged by each of these. For example, because students had access to exam and quiz answers as well as the questions, they were able to test their own understanding, receive feedback and then subsequently retest their understanding. Given the wide availability of past exams, students could observe that rote learning would be insufficient. The focus groups confirmed that students were using TopClass to learn more deeply. Another example supporting this, provided in the focus groups, related to the lurkers who simply observed others' discussions. They commented that they were able to correct their own misconceptions from such lurking. A fourth benefit arising in the focus groups related more to learning the innovative relevant technology than learning the subject per se. Students commented that it would be beneficial to know about the web because it was apparently so ubiquitous in everyday business and society. The first three positive aspects of the web-based teaching technology which helped students achieve new understandings arose also in Freeman (1996). They were thus part of the overall course evaluation survey. Students rated access to support materials very highly, the Announcements folder in particular (ie. 83% agreement in Table 5). This folder makes it easier for students to access the dynamic administrative information to keep up with what was happening. Students also rated highly the opportunities to use the static learning materials (eg. 82% agreed in Table 5 that the provision online of the extra tutorial Q&A, which were dynamically 34 Australian Journal of Educational Technology, 1997, 13(1) updated in the Coursework folder during the semester, helped their learning). 81% agreed that the dynamic learning opportunities provided by the optional, auto-corrected quizzes helped their learning. To a lesser extent, students appreciated the opportunities to get asynchronous support from their peers as well as their teachers. For example the ability to interact with coordinating staff by private email or via the public discussions was appreciated by students (ie. 68% and 67% respectively in Table 5). Aspect of learning context TopClass folder % Agree Access to support material or information • statically (eg. past exams) Coursework 81 • dynamically (eg. admin) Announcements 83 • dynamically (eg. lurk others' Q&A) Discussions 64 Interaction opportunities with materials to learn • dynamically (eg. quizzes) Coursework 81 • statically (eg. extra Q&A) Coursework 82 Interaction opportunities with people for support • dynamically (eg. email peers) Send/Read Messages 50 • dynamically (eg. email staff) Send/Read Messages 68 • dynamically (eg. learning by asking Q) Discussions 65 • dynamically friendly chat Discussions 67 Table 5: Positive aspects of web-based teaching technology to learning context. Post-course survey n=285. The negative student perceptions of TopClass are reviewed in Table 6. The main problems were the lack of access to a reliable machine for on campus users (58%) and the lack of reliability in the TopClass or NT server software (37%). During the semester, the database had become corrupted. Poor backup procedures meant it was inaccessible for over two weeks. Other NT server problems caused the system to crash more than was preferred. Access problems are clearly critical and must be addressed by academic and support managers because 77% indicated that these negative aspects caused them to lose interest in using TopClass. Frequency Percent TopClass/server seemed to crash often 193 37.1 Slow modem connection 78 3.9 Slow PCs in labs 116 22.3 Difficult getting a lab PC within a reasonable time 133 25.6 Table 6: Negative aspects of web-based teaching technology to learning context. Post-course survey n=258. Freeman 35 From the students' perception, did the positive aspects outweigh the negative? Table 7 indicates a resounding affirmative answer to this question. 94% of students felt TopClass would have a positive impact on their grade (in 7A) and 72% would like many or all aspects of TopClass used in other subjects (in 7B). These results replicate the very positive student response found in Kiser and Toreki (1997). 7A. Given the range of aspects used in TopClass, do you think it will have an impact on your grade? Frequency Percent Significantly negative 2 0.8 Small negative 7 2.7 Small positive 129 50.0 Significantly positive 113 43.8 7B. If it could be guaranteed to be working, would you like other subjects to use TopClass? No Way 13 5.0 Only some aspects 54 20.9 For many aspects 51 19.8 Definitely yes (all aspects) 134 51.9 Table 7: Overall student perceptions. Post-course survey n=258 There are two competing explanations for these results. One explanation is that students' conceptions of learning is confined to rote learning facts and they perceive they will achieve a higher grade, in this subject and others where TopClass may be introduced, because rote learning alone is assessed. The other explanation is that students conceive of learning as better understanding or meaningful learning and that this is ultimately how grades are assessed in this and other subjects. Hence these students believe that their use of TopClass will support such deeper approaches to learning. There are two reasons for accepting the deeper learning explanation in this trial. First, students were able to review past exam papers and aspects of these were referred to throughout the semester. It would be obvious upon such a review that meaningful learning and understanding were being assessed. The second reason for accepting the deeper learning explanation relates to the feedback in focus groups. Students appeared to have a taken a high responsibility for their learning, had made connections between the subject and the real world, and appreciated that a range of resources had been made available to help them learn. The relative value of the main resources can be gauged from three questions using a seven point Likert scale (where 1 = Very poor, up to 7 = Excellent). Students overall rated 36 Australian Journal of Educational Technology, 1997, 13(1) TopClass at 5.0, well above tutorials (at 3.9) but less than lectures (at 5.6) in helping them learn. 4.2 Staff perceptions Coordinating staff experienced a reduced burden of time wasting administrative queries. These could be more easily dealt with via web announcements and the public discussion. With such large student numbers, staff had to be disciplined though in discouraging the use of private email and phone. Students prefer these because they are more individualised and instant (unlike the web which one has to choose to log into). There were several other positive aspects of the public discussions and announcements. First, using TopClass during lectures to recall announcements and identify good questions meant students could see they were getting the same product and hints on all campuses. This reduced student concerns for equity and freed up lecture time. A further benefit of the discussions is that staff can point to times and dates when information became public. With large classes, this serves to reduce students excuses for avoiding doing work and doing it properly because they could otherwise plead 'I didn't know'. Coursework was also of benefit because staff could reduce the time spent replacing print material sabotaged in library closed reserve, reduce the time spent creating and marking questions for formative feedback, and increase the speed of the submission-mark-return cycle. External links (eg. to a stockbroker's website) gave case studies a more real world application that is of benefit to the learning context. Negative aspects for staff also exist. First, the time and date stamp on discussions gave an accountability pressure.. Students expect answers to be forthcoming within a reasonable time, including the weekend! Considerable time is spent familiarising and in developmental work. Future subsequent uses of a web-based teaching tool should yield returns on that investment. Considerable frustrations were experienced from the software instability faced during the semester. These arose from two sources. First the released version TopClass 2.1 had several bugs. Second, some instabilities were introduced from using data/coursework developed in the beta version. Almost all of these were addressed in maintenance release TopClass 1.2.1. Being an early adopter added further problems well documented by Geoghegan (1996) and others. For example, students use academic staff for technical queries. Staff are exposed to risks because preventative procedures are still being developed. Poor reliability, whether caused by malfunctioning desktops, networks or servers, becomes an issue with which teachers must now deal. Freeman 37 5. Concluding remarks The objective of this paper is to report the outcomes of a web-based teaching program trial involving 550 undergraduate business students. The program was integrated into the teaching and learning environment. This trial was undertaken over the 13 weeks to mid-June 1997. Despite very low prior experience with the internet, students appeared to value web-based teaching in terms of its increased support for their learning. A range of evaluation methods were used to ascertain outcomes. The focus groups were a particularly rich source of information about students learning. The results confirm that students largely had positive perceptions of the interactive features, self testing and monitoring facilities, in the web-based teaching program, and appreciated the ready access to online information. The focus groups suggested that students perceived that these features of the learning environment encouraged them to understand and thus take a deeper approach to learning. And as shown earlier in Figure 1, if students take a deeper approach to leaning then that is more likely to be the outcome. The survey results confirm these findings for this sample. The ease of use and asynchronicity suggests students in this sample value flexible learning. 94% of students felt TopClass would have a positive impact on their grade and 72% would like many or all aspects of TopClass used in other subjects. These positive aspects of web-based teaching replicate those of Kiser and Toreki (1997). Care should be taken in generalising these results which are based on one learning context and one particular web-based teaching program to deliver computer mediated communication. A reflection on the evaluation results reveals four critical factors. Geoghegan (1996) and others have identified many of these before as major determinants of whether a technology 'crosses the chasm' from the early adopters to mainstream faculty. They fit also into Figure 1. Academics and academic managers seeking to utilise web-based teaching technology to enhance the teaching and learning environment would do well to consider these issues. • Pick the right people, namely committed academic and support staff. Enthusiastic academics are prepared to make the environment conducive to learning and in particular, integrate the technology into the teaching and assessment context. Considerate student communications are still critical to ensuring positive students learning conceptions. Moral support from friends, supervisors and colleagues can go a long way in maintaining enthusiasm as does committed technical support staff who are prepared to minimise downtime. 38 Australian Journal of Educational Technology, 1997, 13(1) • Undertake staff and student training. All teachers and related support staff (technical and non-technical) need to be aware of what is happening so they can respond appropriately. Train in advance where possible and have backup hard copy, online or video instructions for staff and students. Demonstrate its use at every face-to-face session with students so they become familiar with it. • Ensure efficient, 24-hour, seven-days-a-week access to reliable technology. Early adopters may tolerate risks of poor service and backup but mainstream users will not. Various spillover effects were identified because of the large group, remote access aspects of the trial. Bugs like rotating proxy servers, shared phone lines and 'call waiting' caused major problems for some remote access students. While these are unrelated to the specific software, such spillovers need identifying before web-based teaching is used in a major way for assessment purposes. • Evaluate what works and what does not. Publishing evaluations for critical review will enhance the likelihood that educators facing similar financial pressures can make better-informed decisions on value-adding technologies. The evaluation of the current trial is being extended to evaluate student gender, age or study pattern biases. If the aim of introducing technology is to enhance student learning outcomes, then the technology must be designed so as to encourage students to take a deep approach to learning. For this to be possible, teachers' conceptions may need to change from one of information transmitter towards facilitating changes in student conceptions of their subjects. Educational technologies therefore do not remove the role of the teacher as the primary facilitator of student learning and as the designer of the learning environment. Academic managers who perceive technology only as providing cost efficiencies by removing the teacher as information transmitter, are using a limiting conception of what it means to be a teacher in higher education. References Biggs, J. B. (1993). From theory to practice: A cognitive systems approach. Higher Education Research and Development, 12, 73-85. Blackhurst, A. E. & Hales, R. M. (1997). Using an education server software system to deliver special education coursework via the World Wide Web, http://.serc.gws.uky.edu/www/ukat/topclass/tc.html Freeman, M. A. (1996). The role of the Internet in teaching large undergraduate classes, Flexible Online Learning, http://.www.ee.uts.edu.au/folj Freeman 39 Geoghegan, W. H. (1996). Instructional technology and the mainstream: the risks of success. Mayhum Distinguished Lecture, SUNY College at Fredonia. Gibbs, G. (1992). Improving the quality of student learning. Oxford: Technical and Educational Services Limited. Hicks, P. J. (1997). Re-engineering higher education. http://.www.umist.ac.uk/future/re-he.htm Kiser, R. & Toreki, R. (1997). Efficacy of WEST Software in CHE 105-001. http://www.chem.uky.edu/misc/105westsurvey.html Marton, R. & Saljo, R. (1997). Approaches to learning. In F. Marton, D. Hounsell and N.J. Entwistle (Eds), The Experience of Learning. Edinburgh: Scottish Academic Press (second edition). Mason, J. & Hart, G. (1997). Effective use of asynchronous virtual learning communities. http://www.arch.usyd.edu.au/kcdc/conferences/VC97/papers/mason.html McQuillan, P. (1994). Computers and pedagogy: the invisible presence. Journal of Curriculum Studies, 26(6), 631-653. Miller, L. & Olson, J. (1994). Putting the computer in its place: A study of teaching with technology, Journal of Curriculum Studies, 26(2), 121-141. Ramsden, P. (1992). Learning to teach in higher education, London: Routledge. Reeser, S. (1997). A look in to the "How To's" of web-based education. http://.www.ilstu.edu/~sdreese/article.htm Selinger, M. (1995). The role of electronic communication in supporting beginning teachers. Trigwell, K. And Prosser, M. (1996). Changing approaches to teaching: A relational perspective, Studies in Higher Education, 21, 275-284. Acknowledgments Financial support for this project was provided via curriculum development initiatives for flexible learning from the Faculty of Business and the University of Technology, Sydney. The author gratefully acknowledges technical support provided by James Sawers and assistance from Shirley Alexander, Simon Housego and Jo McKenzie in preparing this paper. Mark Freeman, School of Finance & Economics, Faculty of Business, University of Technology, Sydney. PO Box 222 Lindfield NSW 2070 AUSTRALIA Email: Mark.Freeman@uts.edu.au Phone: +612 9514 5425 Fax: +612 9514 5515