http://www.sajim.co.za/peer17.3nr1.asp?print=1 Peer Reviewed Article Vol.3(1) June 2001 Longitudinal comparison of two presentations of an Internet-based course for adult learners: a university case study G.J. de Villiers and J.C. Cronjé Department of Information Science University of Pretoria bda@mweb.co.za Contents 1. Introduction 2. Background to the problem 3. Research question 4. Outline of the study 5. Components of the two digital classrooms 6. Instructional strategy 7. Results 8. Conclusion 9. Recommendation 10. References 1. Introduction Nothing before has captured the imagination and interest of educators simultaneously around the globe more than the World Wide Web (Owston 1997). Digital classrooms, supplemented with telecommunications media, are currently being used in tertiary institutions world-wide, in the form of Web-based courses for adult learners (Gunawardena 1999; McMahon 1997). Tertiary education faculties, matric teachers, and business trainers alike are turning to the World-Wide Web as a vehicle for implementing instructional innovations (Khan 1997; Owston 1997). The Internet is claimed to be one of the most powerful tools for providing lecturers and learners with necessary conditions for independent and interactive learning (Le 1999). It provides an educational discourse in which learners can interact widely with other members of a learning community and, at the same time, be in control of their own learning. Their interaction for learning can be immediate, prompt, widely shared and resource supportive (Owston 1997), which may not be possible in a traditional mode of teaching, confined by the classroom's physical condition (Le 1999). Moreover the Web can help us refocus our institutions from teaching to learning, from teacher to learner (Owston 1997). In the rush for universities to market Internet-based teaching, the traditional teacher-centred knowledge transmission metaphor (efficient delivery to individual passive absorbers) prevails, even though the new technology offers unique possibilities for promoting reflective and collaborative learning (Taylor and Maor, 2000). It must be kept in mind that the Internet and the Web are simply resources which must be designed to support effective instructional dimensions (Reeves and Reeves 1997). It is thus imperative for the instructor to consider the possibilities and constraints of this medium, in order to exploit the possibilities and work around its constraints (Reiser 1994). This article reports on a longitudinal comparison of two presentations of a Internet-based course for adult learners, which was presented in two consecutive years. The purpose of the study was to compare which aspects of Internet-based learning operate and function successfully, and which do not. The main contribution of this study was that it provided an appropriate model for future digital classrooms. 2. Background to the problem This study investigates the technology-based delivery of a Master’s level course in the domain of Internet-based learning. This course initially required learners to travel to the University of Pretoria from their various home regions, spread across South Africa, for face- to-face interaction and contact time between learners and the course presenter and among the learners themselves. From 1997 to the year 2000, the course (RBO880) was presented solely by means of a 'digital classroom', defined by Clarke (1998) as a computer accessible, on-line learning environment attached to the World-Wide Web, and which delivers material to adult learners at locations other than the course delivery centre. In 1997 Patsy Clark, in her MEd thesis, investigated the feasibility of using the World-Wide Web for the telematic teaching of adults. The focus of her investigation was the same Master's level course (namely, Internet-based learning) presented via a digital classroom, to facilitate individual and collaborative projects. The Web site was supported by a dedicated electronic mail server that re-distributed messages to all participants, enabling on-line discussions. The results of her study indicated that, provided learners have access to stable and reliable telecommunication links, telematic methodologies can be effective facilitators of course delivery for both individual and collaborative projects. Moreover, e-mail interaction and communication among course participants, as well as between course participants and the instructor, can be invaluable enhancements to Web-delivered learning material. This was confirmed by the instructor, who considered the course to be the best he had ever presented. The identical course was presented in 1999. During this presentation, certain problems occurred. In looking back over the course, the instructor refers to the RBO digital classroom of 1999 as the 'worst course he's ever presented', and which also had a high attrition rate. The instructor, despondent about the 1999 course, made modifications in 2000 to improve it, so as to make it as successful as 1997. Therefore, this study focused on a longitudinal comparison of the successes and weaknesses of two RBO courses, that is, the presentations of 1999 and 2000. 3. Research question The major research topic to be addressed by this research was: How did the RBO digital classroom of 2000 compare with the RBO digital classroom of 1999? The question can be refined by the following sub-questions: 1. Which aspects of the RBO digital classroom of 1999 worked, and which did not? top top 2. Why did these aspects of the RBO digital classroom of 1999 work or not work? 3. What modifications were brought about in the RBO digital classroom of 2000? 4. Which aspects of the RBO digital classroom of 2000 worked, and which did not? 5. Why did these aspects of the RBO digital classroom of 2000 work or not work? 6. What was successful/not successful in both of the digital classrooms? The following tools were used to collect the data: interviews, questionnaires, participant observation and an overview of the messages sent to the listserver. 4. Outline of the study In this section, the target population, objectives, components and the instructional strategy of the two classrooms are described and discussed. 4.1 Target population The target population differed considerably in 1999 and 2000, as evidenced by Table 1. Some of the learners had inadequate Internet skills but, in contrast, others were very experienced or skilled learners, able to move at a rapid pace. Some of the learners were not the instructor’s typical students; some of them were guest-students, having been invited by the instructor, since there were only a few official enrolments. In the 1999 group, a considerable number of these learners discontinued the course. The RBO course of 1999 incorporated a group of learners from Rhodes University. Their instructor was investigating to what extent universities could collaborate by presenting joint courses. In fact it turned out that the collaboration was unsuccessful, possibly due to different standards of expertise at the entry level. Table 1 Description of the learners on the RBO course of 1999 and 2000 top 1999 2000 Number of learners registered for the RBO course 23 learners registered 11 learners registered Student status 7 registered M.Ed. learners 1 registered for MA (Information Science) 15 enrolled for non- degree purposes 9 registered M.Ed. learners 2 enrolled for non-degree purposes Gender 11 female 12 male 8 female 3 male Age range From early twenties to mid-fifties From mid-twenties to mid-fifties Employment status 22 in full-time employment 1 in part-time employment 9 in full-time employment 2 in part-time employment 4.2 Objectives The aim of the course, in both 1999 and 2000, was to: Present learners with a map and compass and leave them to learn the skills of navigating through cyberspace; sharpen their Internet literacy; and provide learners with the theoretical and practical know-how to use the Internet effectively in teaching/training. The latter gave learners a unique meta-learning experience, that is, teaching 'teaching on the Internet' on the Internet. The course content was divided into two aspects: educational aspects and physical aspects. Educational aspects covered such themes as network-based learning, aspects of distance learning and the Internet, and the management of network-based learning. Physical aspects were concerned with the practical competencies learners should display with the following technologies: e-mail, listservers, chat groups, FTP, zipping and news groups. Learners also had to find their own free Web space and write elementary Hypertext Markup Language (HTML). 5. Components of the two digital classrooms In both 1999 and 2000 the digital classroom had two components: the Web site and the listserver. The Web site used the metaphor of a real physical classroom, while the listserver was set up to facilitate interaction and communication among the learners in the course. Each year the course is run a new communications tool is added. In 1999 a bulletin board was added. This proved to be remarkably effective in how it was used: The instructor used it for posting learners’ work, and then referring learners to it; posting follow-ups on different topics/assignments and for threaded discussions; and a backup for static information. Previous contact Some learners had worked together on projects the previous year. Most learners, however, had not met face-to-face Most learners had met face-to-face, except the two learners external to the course Intellectual/academic ability of the registered RBO learners in terms of how homogenous or mixed they were Very diverse Reasonably homogeneous Levels of: Computer literacy/experience Internet, WWW literacy/experience Motivational levels Very diverse Extensive background to no background Low, medium and a few highly motivated learners Reasonably homogeneous Extensive background to a basic background Medium to highly motivated learners Location of learners Highly dispersed Highly dispersed top In the year 2000, eGroups (a free group service) was added. eGroups was a remarkably effective improvement, because not only did it send messages to all class members, but it also kept an archive of the messages on the eGroups Web site. It therefore had an automatic bulletin board, where messages could either be read in order of date or viewed by thread. eGroups also includes other sophisticated features, such as a list of members, chat room, poll, links, calendar and a database, of which the instructor put the chat room and calendar to use. The Web site for the two digital classrooms of 1999 and 2000 can be viewed at http://hagar.up.ac.za/rbo/1999/classroom.html and http://hagar.up.ac.za/rbo/2000/index.htm, respectively. The digital classroom of 1999 was divided into four sections: 1. Blackboard and the notice board. The blackboard was a graphic file that learners could access and edit, making it the equivalent of graffiti one would find in a physical classroom. Clicking on the bulletin board gave comprehensive on-line study guides. Clicking on the blackboard revealed the timetable and tasks, similar to what one would find in a physical classroom. 2. Instructor’s desk and resource cupboard. The instructor’s desk was linked to his home page, with his curriculum vitae and links of general interest. To the left of his desk was a link to the collaborative tasks and, to the right, the resource cupboard, which offered links to subject matter and Web site construction programs such as graphic tools and HTML editors. 3. Poster wall. The poster wall had links to projects by learners from previous years, as well as to the collaborative 'posters' created by the learners as the course progressed. 4. Space for the learner's desks. Each learner was assigned a WWW directory that was linked to a desk graphic. The learners had to replace this graphic with a personalized one, and manipulate it in any way they liked. They then had to 'fill the desk' with things typically found in a school desk (RBO Curriculum 1999): Your ears (mailto: ...) Your utility bag (links to handy stuff such as HTML editors, search engines, clipart libraries, etc.) Your textbooks (links to useful sites) Your work (interesting stuff you have done in other M.Ed. modules) Your hobbies (personal info and/or links to sites of special interest to you) Your class work (your answers to all the objectives of the course) Your portfolio (a link to the portfolio of your examination project). The digital classroom of 2000 had similar sections except, instead of clicking on images, learners clicked on wooden buttons. These buttons took learners to the course outcomes, tasks, timetable, references, instructor, classmates and the lifeline. In an effort to get the whole classroom on the screen, the learners’ desks were drastically reduced in size and situated right in the middle of the classroom. To the left of the learners' desks was the poster wall and to the right the blackboard, which learners were to scribble on. 6. Instructional strategy top The learning theory the instructor adopted for RBO was both constructivist and collaborative. Constructivism is based on learners actively constructing their own knowledge (Tam 2000). Constructivist aspects include real-world situated learning, anchored instruction, discovery learning, integrated testing and transfer (applying known skills to new tasks) (De Villiers 1999). The active learner participation required in constructivistic models can lead to long-term results and real-world performance. Web-based teaching and training facilitate learner initiative, knowledge construction and real-world exposure via browsing (De Villiers 1999). In the RBO course of 1999 and 2000, learners not only had to construct their own meaning; in the process they were also required to construct individual and collaborative Web pages around certain real-world situations and problems. The advantage of this was that learners could benefit from the knowledge contributed by the other learners. The course began with an introductory session on the bulletin board (in 1999) and the listserver (in 2000), where learners introduced themselves and gave their motivation for doing the course. Learners then worked on their desks. The collaborative period then followed, where learners built 'posters'. In 1999, there was only one collaborative group task. Learners worked in groups of three to four learners on a task. In 2000, the whole class was instructed to collaboratively build a 'virtual opera', after which they worked in two-person teams, reporting on a Web-based learning initiative. The collaborative tasks met Johnson and Johnson’s (1999) five prerequisites for effective co-operative learning, namely positive interdependence, individual accountability, a mutual goal, face-to-face promotive interaction and social skills. Finally, learners had to submit proposals for their own three-month Web-based project. These had to be discussed on the bulletin board or the listserver (in 1999 and 2000, respectively), executed and published in their own desk. The instructor deliberately avoided supplementing the digital classroom with face-to-face contact, which would have been at the expense of pure distance-learning. Distance-learning was an inherent objective of the course – the idea being to study distance education by means of distance education. In a face-to-face contact session, the instructor could have passed more information on to them in tangible format, but it would have lost out on constructivism. 'I wanted them to experience what it feels like when they are actually at a distance without contact. In fact, when some things don’t work, that’s also OK, because then they also learn how NOT to do things' (RBO instructor). The RBO digital classroom was therefore the means through which learners would actively learn via the Internet and the Web and, in the process, experience the strengths and limitations of teaching, learning and collaborating via the Internet and the Web (Clarke 1998). 7. Results The results of the five research questions are now discussed with the intention of solving the main research question, that is, how did the RBO digital classroom of 2000 compare with the RBO digital classroom of 1999? 7.1 Which aspects of the RBO digital classroom of 1999 worked, and which did not? Aspects that did not work In the 1999 group, there was a significant attrition rate among the outsiders who registered for the M.Ed. Only two of 15 outsiders who registered for the course completed it. However, top only one of the M.Ed. learners did not complete the course. These learners discontinued the course primarily due to busy time schedules, but also due to frustrations and difficulties with technology and inadequate Internet skills. The overall attrition rates were 78% in 1999 and 27% in 2000. The instructor also did not give enough feedback to learners. In 1999, 62% of the learners did not agree that the instructor supplied feedback to and assessment of their various tasks, as evidenced in the following statement: I wanted feedback to know that what I was doing was good or bad – that I was on track or off track. Although 75% of the learners agreed that they did benefit from the listserver, 67% of the learners did not feel a sense of community building, and 62% of the learners did not find the environment to be supportive. As a result, some learners experienced problems on the affective level. Although 92% of the learners found the course a real learning experience, 35% of the learners did not enjoy the course. One of the reasons for this was due to the conflict that arose between certain learners in the course, as evident in the following comments from learners: That group identity is not merely an issue of people seemingly working towards similar goals. That one person (or a clash of personalities between two people) could sabotage the whole process. I questioned myself in relation to interactions with other people and realized that I had taken for granted the rapport I have developed with colleagues in a face-to-face environment. This is an important learning curve for on-line learning so the experience is appreciated. I did not have a nice experience with the group of people. Aspects that did work On the whole, the collaborative group projects were effective. One group project was particularly successful, in the way in which the instructor handled a difficult situation. Early on in the course a controversy arose between two participants, who were both outsiders and therefore did not know each other. The conflict between them continued, resulting in different reactions from the rest of the group. Some learners took sides and others felt intimidated and remained silent. To resolve the conflict between these two learners, the instructor placed them together in a co-operative learning group. Their task was to 'build a Web site dedicated to conflict and conflict management in virtual learning environments'. One of the instructor’s graduates, a calm and serene individual, was used as the intermediary to 'pour oil on troubled waters'. They called their task inFLAMation, and came up with a brilliant metaphor of Calvin, Hobbes and Lucy. 7.2 Why did these aspects of the RBO880 digital classroom of 1999, work or not work? The group was very diverse in terms of background. Learners did not know each other upfront, and the instructor had not realized that he still needed to integrate those who did not know each other. A further problem was that some of the outsiders dominated the listserver, as evident in the following statement from two of the M.Ed. learners: I felt hostile towards these new people, and felt they were imposing on our group and outsiders did not really belong on the course. The skills of learners ranged from learners with an extensive background to learners with no background knowledge of the Internet. Learners with an extensive background knowledge were slowed down, while the instructor tried to keep the others up to date. 7.3 What modifications were brought about in the RBO digital classroom of 2000? The skills gap was smaller in the class of 2000. This enabled learners to be closer together in terms of their Internet skills. The number of outsiders allowed on the course were drastically reduced. In the year 2000, there were only two outsiders, and they were allowed on the course because they were strongly motivated to participate, and their skills were on a par with the other M.Ed. learners. The instructor drastically improved on the feedback he gave learners. In 2000, 88% of the learners agreed that the instructor supplied feedback to and assessment of their various tasks. The design of the classroom changed in 2000 to make the design cognitively comprehensible, that is, consistent, predictable and one that enabled user control. 7.4 Which aspects of the RBO digital classroom of 2000 worked, and which did not? Aspects that worked The listserver proved to be effective in that 78% of the learners benefited by it and 67% of the learners felt both a sense of community building and experienced the environment as supportive. The individual and collaborative tasks were both very effective in 2000, particularly the task that involved the entire group of learners. The instructor gave the learners the task of composing a virtual opera. The specific learning outcomes were to (RBO Curriculum, 2000): Work together as a team; produce Web pages in a frames environment; obtain and produce suitable graphics and animated graphics in gif or jpg format and use them to enhance the Web page; edit MIDI files and insert them as background objects in Web pages; present at least 20 Internet terms with their definitions in a de-contextualised format; and use an HTML editor. The task worked remarkably well in terms of what the learners gained from the exercise. Learners learnt a great deal about collaborative learning (its benefits and its pitfalls) over a distance and also enhanced their technical skills. The following comments from learners indicate this: I learnt that co-ordinating a project over the Internet is possible but far from optimal. It was especially frustrating as simple decisions took forever to be made. Things could drag on for days. What amazed me was that a group of students that live far apart, with different interests and viewpoints could put together this opera. The opera was therefore very successful in creating a sense of unity and a strong bond between the learners. Aspects that did not work Of the learners, 63% found the interaction of the listserver both frustrating and useful. The more active learners in the course, became discouraged when their e-mail messages were ignored or answered at a much later stage. I experienced the interaction with other learners as frustrating due to the fact that one does not always receive answers or get a reaction on e-mail. I found e-mail to be a frustrating communication method if an immediate response was required. Nothing beats direct face-to-face communication, or a phone call. The chat room was available for synchronous communication, yet learners did not get it right and 56% of the learners experienced technical problems with the chat room during the two sessions arranged. Overall, all the learners felt their second collaborative group task was a success. There was however a few aspects that hindered group cooperation, such as: Frustration of one learner having more time than the other and, therefore, delaying his/her team-mates; electronic disappearance/loss of one’s partner in cyberspace. One learner was very honest about his input: working collaboratively with no face-to-face contact was a recipe to encourage irresponsibility on my part; lack of personal contact; and one learner’s Internet connection was washed away during the floods of March 2000. In 2000, 88% of learners supplemented their on-line collaboration with other means of communication during the course, such as phone and face-to-face contact. In 1999, this percentage was 26% less. 7.5 Why did these aspects of the RBO digital classroom of 2000 work or not work? In 2000 the class was more integrated and homogeneous. The collaborative tasks helped the learners to get to know each other, and kept learners from feeling isolated from the rest of the group. The opera worked particularly well to bring a sense of unity to the group. Learners were on the whole more motivated than the learners in 1999. The learners were more carefully monitored, and received feedback regularly. The schedule was not as tight in 2000. Learners were given much less work and more time to complete their tasks. 7.6 What was successful/not successful in both the digital classrooms? In both digital classrooms, the course was successful in: Adequately guiding, facilitating, and enhancing the learning of its learners, as well as meeting the needs of adult learners, without the provision of face-to-face interaction and contact time; providing learners with materials and models that motivated them to learn; and the way the instructor grouped his learners in their collaborative group tasks. The instructor takes on the following approach: I agonise very much. I really work on an outcomes-based approach depending on what I want, and then I apply groups. The point is, it is not an opportunist thing. I don’t let the learners choose. I decide very clearly what I want and I MAKE GROUPS, and that is why my groups work. In both digital classrooms, the course was not entirely successful in providing learners with useful and relevant communication that met their needs. In 1999 and 2000, 45% and 63% of the learners, respectively found the interaction frustrating. 8. Conclusion The results of this research indicate that the RBO digital classroom of 1999 was inadequate in integrating learners, providing feedback to learners and in creating a supportive environment for learners. The answers to the six sub-questions indicate that the digital classroom of 2000 was a major improvement, with learners, in general, being more positive and enjoying the learning experience far more than the 1999 group. The lower attrition rate in 2000 also attested to this. However, despite the problems that arose in 1999, the learners who remained in the course learnt a great deal about group processes in the on-line environment and developed a sophisticated understanding of the interactive potentials, pitfalls and issues faced in on-line collaboration. With the modifications made in 2000, the study proves that the model works but that the detail of the model needs fine-tuning. The following challenges arise from the study: Maintain a sense of the group throughout the course; keep interaction on listservers useful to prevent learners from feeling frustrated by the e-mail communication. Also, keep listservers running, and not dying out after the course is over; and prevent learners, in the collaborative group assignment, from resorting to other means of communication, such as face-to-face contact and telephone calls. This research has helped to identify some of the strengths and pitfalls that can occur in digital classrooms, based on a comparison of two digital classrooms, specifically looking at which aspects of Internet-based learning operate successfully and which aspects do not operate or function successfully. It is hoped that this research will expand instructors’ abilities to effectively educate and train via the Internet to attain greater success in future digital classrooms. 9. Recommendations At the outset of the course, encourage learners to post introductions along with their fears and expectations for the course or, when possible, to create a homepage that other learners in the group can visit. top top Ask learners to report weekly on their progress or attempts/successes/failures on the tasks given to them. This may keep discussions more constructive, making the instructor more aware of where learners could be experiencing problems and learners may find that they share the same difficulties. Learners can establish clear norms and guidelines and agree to them right at the outset of the course (Fisher, 2000). Learners then abide by these norms and guidelines for the duration of the course. The home network and the remote network must be stable. To avoid learners meeting face-to-face, place learners in teams with learners who live far apart. Include many sub-deadlines and place pressure on learners to keep to them. Schedule inter-learner debates on controversial but relevant topics. 10. References Clarke, P.A. 1998. Telematic teaching of adults via the World Wide Web: a university case study. Unpublished M.Ed. dissertation, University of Pretoria: Pretoria. De Villiers, M.R. 1999. Applying the hexa-C metamodel of instructional theory and design to educational web applications. In: P. de Bra and J. Leggett (eds.) Proceedings of Web Net 99 – world conference on the WWW and Internet. Honolulu: Association for the Advancement of Computing in Education. Fisher, K. Phelps, R. and Ellis, A. 2000. Group processes online: teaching collaboration through collaborative processes. Educational Technology and Society, 3(3). [On-line]. Available WWW: http://ifets.ieee.org/periodical/vol_3_2000/v_3_2000.html Gunawardena, C. 1999. The challenge of designing and evaluating 'interaction' in Web-based distance education. In: P. de Bra and J. Leggett (eds.) Proceedings of Web Net 99 – World Conference on the WWW and Internet. Honolulu: Association for the Advancement of Computing in Education. Johnson, D.W. and Johnson, R.T. 1999. Making co-operative learning work. Theory into Practice, 38(2):70–71. Khan, B.H. 1997. Web-based instruction. Englewood Cliffs, NJ: Educational Technology. Le, T. 1999. A Web-based study of students' attitudes towards the Web. In: B. Collis and R. Oliver (eds.) Ed-Media 1999: world conference on educational multimedia, hypermedia and telecommunications. Washington: Association for the Advancement of Computing in Education. McMahon, M. 1997. Social constructivism and the World Wide Web – a paradigm for learning. [On-line]. Available WWW: http://www.curtin.edu.au/conference/ASCILITE97/papers/Mcmahon/Mcmahon.html. Owston, R.D. 1997. The World Wide Web: a technology to enhance teaching and learning? Educational Researcher, 26(2): 27-33. RBO880 Curriculum 1999. Course notes for M.Ed. Computer-Assisted Education, top University of Pretoria. [Online]. Available WWW: http://hagar.up.ac.za/rbo/1999/rbotasks99.html. RBO880 Curriculum 2000. Course notes for M.Ed. Computer-Assisted Education, University of Pretoria. [Online]. Available WWW: http://hagar.up.ac.za/catts/learner/2000/opera/index.htm. Reeves, T.C. and Reeves, P.M. 1997. The effective dimensions of interactive learning on the WWW. In B.H. Khan (ed.). Web-based instruction. Englewood Cliffs, NJ: Educational Technology. Reiser, R.A. 1994. Clark’s invitation to the dance: an instructional designer’s response. Educational Technology Research and Development, 42(2):45:48. Tam, M. 2000. Constructivism, instructional design, and technology: implications for transforming distance learning. Educational Technology and Society, 3(2). [On-line]. Available WWW: http://ifets.ieee.org/periodical/vol_2_2000/tam.html. Taylor, P. and Maor. 2000. Assessing the efficacy of online teaching with the constructivist on-line learning environment survey. Teaching and Learning Forum 2000. [On-line]. Available WWW: http://cleo.murdoch.edu.au/confs/tlf/tlf2000/taylor.html. Disclaimer Articles published in SAJIM are the opinions of the authors and do not necessarily reflect the opinion of the Editor, Board, Publisher, Webmaster or the Rand Afrikaans University. The user hereby waives any claim he/she/they may have or acquire against the publisher, its suppliers, licensees and sub licensees and indemnifies all said persons from any claims, lawsuits, proceedings, costs, special, incidental, consequential or indirect damages, including damages for loss of profits, loss of business or downtime arising out of or relating to the user’s use of the Website. ISSN 1560-683X Published by InterWord Communications for the Centre for Research in Web-based Applications, Rand Afrikaans University