Online learning for design students Zhengmai Zhao, John Cook and Nick Higgen De Montfort University, Leicester This paper describes the development of a Web-based learning resource for Design students at De Montfort University, and presents the results of a study to investigate the effectiveness of this system. Some issues regarding further improvements to the online resource are also discussed. Introduction During the last five years, many changes have taken place in undergraduate Design education in the UK. The main force behind these changes has been the dramatic increase in the number of students without a commensurate increase in teaching staff. Typically, many Design departments are now teaching at student-staff ratios of up to 50:1 compared with the 10:1 typical of Design courses only a few years ago. With the more recent reductions in higher- education funding, combined with the added pressures on staff to raise their research profile and also become income generators, the problems relating to increased student numbers have been further exacerbated. The authors of this paper believe that in the current educational climate, the only way to maintain and promote the quality of the educational provision is to make teaching and learning more productive and efficient by harnessing modern technology. Earlier this year, we developed an online learning resource on a lecture entitled Contemporary Crafts, based on a platform provided by a TLTP project, STILE (Students' and Teachers' Integrated Learning Environment), and put it into use for an undergraduate introductory design module, Contextual Basis of Design. The work was initiated with two objectives: 1. It was expected that an intensive short-term effort to build the online resource would bring about significant long-term benefits by reducing both the costs and staff time involved in the management, delivery and updating of the course materials. 2. It was intended that student support would be improved by providing them with flexible access to learning resources which included lecture notes, bibliography, lecturer information, and a large number of images with explanations. These visual materials 69 Zhengmai Zhao et al Online learning for design students were regarded to be particularly important for students to better understand the subject matter, but students had previously had only a limited access to them. In order to improve the learning resource itself, and to find the most appropriate way to incorporate it into existing pedagogical practice, an exploratory study was carried out. In this paper, a description of the online resource will be provided, along with a report on results from the study, and a discussion of some issues raised from the initial experience. The online resource The World Wide Web provides an excellent platform for developing, organizing and disseminating learning resources due to its unique combination of features: universal readership, hypertext and multimedia capabilities (Berners-Lee et al, 1994). Once information is made available on the Web, it is accessible from any computer connected to the Internet world-wide, which allows a highly flexible access to the learning resource. Documents on the Web can be organized in hypertext form. This means two things: (1) the provider can build a resource in a non-linear fashion, and expand the resource by integrating new material using hypertext links; and (2) the reader can escape from the sequential organization of the pages to pursue a thread of his or her own, something which offers unusually high levels of learner control. The multimedia capability of the Web makes it possible to incorporate resources in a variety of forms including text, visuals, sound, and graphics. It is due to these attractive features that the educational community has been putting great efforts to make learning resources available through the Web (Blumberg, 1994; Rebelsky, 1994). STILE (Ruggles et al, 1995; Zhao et al, 1995), one of TLTP Phase I projects funded jointly by the four UK higher- education funding bodies (HEFCE, HEFCW, SHEFC, and DENI), is more ambitious in making educational resources available through the Web. It involves four UK universities, initially bringing together the expertise of the University of Leicester and Loughborough University of Technology, and then being extended in its second year to De Montfort University and the Open University. STILE aims to establish a powerful Web-based learning environment with three main features: • the provision of a large, extensible resource-base with special reference to images covering a broad range of disciplines; • the means to locate and access relevant information efficiently; and • the means to facilitate a more flexible approach to learning, including group work and distance and collaborative learning. We decided to develop an online resource within the STILE framework due to its simple model, consistent interface and range of tools which makes both the access to and the provision of information relatively easy. Additional important reasons for adopting STILE were that the STILE software is freely available to all UK higher-educational institutions, and that the project provided the necessary technical and financial support. The conceptual model of a STILE resource-base can be thought of as a network of pieces of information. Each piece of information is either a resource (something to be found and used) or indexing information (something to assist in finding the desired resource). The indexing information is also referred to as topics. A topic represents a particular subject area. Subject 70 Volume 4 Number I areas can be anything general, specific, material or abstract. In terms of this particular study, topics include Contemporary Crafts, Craftperson, Slides of Contemporary Crafts, and Glass. The resources are actual teaching and learning materials. In this study, images and bibliographies are considered as primary resources. Figure 1 illustrates the STILE abstract model, where the shaded circles represent topics which form a topic net, the boxes represent resources which hang on the topic net, and the dashed arrows represent links to external relevant information. The resource-base holds a hierarchical structure, with more general topics being placed at a higher level in the hierarchy. Figure I: The STILE abstract model It is rather natural that the STILE model is realized on the Web because both STELE and the Web consist of networks of linked information. Each STILE topic is represented as an HTML (HyperText Markup Language) document, each resource is represented as a file the Web can handle, and links of information are represented using hypertext links. Navigation of the STELE resource-base can be performed by following the hypertext links. This method of locating information is simple, but can be slow since it proceeds only one step at a time through the network. More effective navigation of the topic net is possible using the STILE navigation tools. One such tool allows navigation at the topic-title level rather than the actual topic document level. The major advantage of this functionality is that it makes parallel navigation possible, i.e., more than one link can be followed at a time, thus speeding up the process of locating information. Another useful navigation tool is the topic-title search engine. Given a word or phrase, the search engine will gather all topics in the current resource-base which include the given word or phrase in their titles, and display them in the order of relevance. The process of building the STILE resource-base featured in this study began with design of the structure. This work included the identification of the indexing information (topics), the 71 Zhengmai Zhao et al Online learning for design students actual information the students would require (resources), and the structure of the topic network. Then topic information, in plain-text format, was marked up using an extremely simple but function-limited markup language called Trivial MetaLanguage, and converted into the topic network using the STILE batch conversion tool. Following that, the STILE browser- based editor was used to link resources (mainly images in GIF format) into the topic net, and to make necessary modifications to the resultant resource-base. The combination of batch conversion tool and interactive editor enabled a standard STILE resource-base to be built without the need to learn how to write HTML documents. Evaluation Objectives The main purpose of the evaluation was to see how the online resource on the lecture entitled Contemporary Crafts worked in a real educational situation so as to improve both the system itself and the way it is integrated into pedagogical practice. Due to the ill-defined goal of the evaluation, an exploratory evaluation strategy was employed. More specifically, an attempt was made to examine the effectiveness of the resource-base in terms of the product of using the resource-base, the process of interacting with the system, and the attitude towards the innovation. Participants Twelve out of 168 undergraduate students who were taking the introductory design course Contextual Basis of Design were requested to participate in the study. They were chosen because they were assigned the seminar presentation topic concerned with the subject of Contemporary Crafts (all other students were allotted seminar presentations against lectures not on the Web). It must be noted that the allocation of seminar topics was conducted randomly. Each chosen student used the resource-base, completed questionnaires, and attended interviews. Procedure Shortly before the lecture on Contemporary Crafts, the chosen students were introduced to the online resource as a study reference in addition to the lecture. They were each provided with flexible access to the resource through about 20 PCs with high bandwidth connection to the Internet in the university library. Since the online material was intended as a flexible learning resource, the students were supposed to access it whenever they wanted. Obviously because of this, it was impractical to observe in person how the students exactly interacted-with the system. As an alternative, an attempt to elicit such information was made using a combination of self- reporting and an automated user-access log. At the very beginning of the study, a questionnaire developed by Draper et al (1994) was used to measure the students' previous experience with computers. The students were also given another questionnaire that they were required to hand in by the end of the evaluation. This questionnaire, adapted from Shneiderman's QUIS tool (Shneiderman, 1992), covered a range of issues including use of the system, overall reactions to the system, system capabilities, process of learning to use the system, and efficiency of using the system. Every student who took the course was requested to give a seminar presentation on an allotted topic about a month after the related lecture was given. The assessment data of all 72 ALT-J Volume 4 Number I students' presentations was available for the subsequent comparison. After all the chosen students made their presentations, structured interviews were conducted. Results and discussion According to the students' self-report, they used the online resource twice on average, and the average length of each session was 25 minutes. These figures were consistent with the data automatically captured in the server log file. The students reported that they used the resource for reviewing the lecture, preparing for seminar presentation, and seeking new ideas. They also reported that curiosity of this new learning technology served as an important initial driving force for them to start using the system. Analysis of questionnaires and interview protocols showed that the overall reaction to the system was generally positive. The system was regarded by the students as being useful, enjoyable to use, and easy to use once they had mastered the basic skills needed to use the system. Most of the students found it easy to find the information they wanted, and to navigate around the system. However, their responses towards how easy it was to learn to operate the system were rather diverse. A further investigation of the students' computer experiences suggests a link exists between their skills of using computers and their response to this issue. It would appear that those students who felt more confident with computers found it easier to learn to operate the system. The students listed a range of benefits from the system. First, the system served as a backup to the lecture, which gave the students an opportunity to review the lecture in more detail. More interestingly, some of the students said that having known the lecture was online, they could concentrate better on the interaction with the tutor during the lecture itself without being worried about taking detailed notes, and that it had the potential to significantly change the way in which they would respond to lectures in the future. Second, the system provided greater flexibility of access to the material, with respect to time, place, pace, and sequencing. The students liked the possibility of accessing the material in their own time and at a place preferable to them. They did not have much choice of place, since none of them had access to the Web anywhere except in the university library. However, they saw this as a great potential benefit of the system. They appreciated the control provided by the system in terms of the pace and sequencing when browsing through the material, especially in the investigation of the visual material. Access to primary materials, images of crafts for instance, is vital for design students to obtain artistic feeling, critical thinking, and creative inspiration. In this particular case, the lecturer had a large collection of slides and only one hour to show them. With the system, the students were now happier to have more flexible access to them. As one of students commented during the interview: 'It's just good to get down to a little bit and study it and then go on to the next when you're ready, rather than have a quick look and just walk out'. In addition, some of the students felt that they became more engaged in the learning process when interacting with the system than when sitting in lectures. Finally, the students, especially those with little computer experience, felt more confident in learning with technology-based systems after using the system. The overwhelming dissatisfaction expressed by the students was with the slow response rate of the system. The length of time to display an image, with an average size of 50KBytes, sometimes became intolerable due largely to the low capacity of the university network 73 Zhengmai Zhao et al Online learning for design students service. In addition, a richer resource-base was strongly demanded for future use. The system did not include links to any materials by others because the subject-author could not find any relevant external resources. A number of improvements were also suggested by the students including the provision of more explicit online help, a full-text search engine, a printing-out facility, and a multiple-windows capability so that comparisons between images could be made more easily. Every student's seminar presentation was monitored by using a common mark sheet, and a group discussion about its positive and negative aspects was carried out immediately after each presentation. The presentations by the students chosen to participate in our study seemed better than those of the remaining students. Ninety-one per cent of the chosen students obtained a grade of A or B, while only 82% of the remaining students reached such a grade. In contrast to 2 1 % of the remaining students receiving an A grade, the percentage of the chosen students receiving a top grade was as high as 36%. However, the results of the Mann-Whitney U test show that there is not a significant difference in the grades of seminar presentations between the chosen students and the remaining students (NA= 11, NB= 144, U = 592, z = 1.39<1.96). As mentioned early in this section, an attempt was made to obtain the information in relation to the students' interaction with the system by using a combination of self-report and automated user-access log. It was found that the students could explain the basic strategy with which they dealt with the system, e.g., systematically covering all contents or selectively picking up points which looked interesting to them. However, they were not able to recall the detail of their behaviour. For example, they had difficulty detailing where they got lost even though they reported disorientation in their navigation, which could be very useful information in terms of improving the structure of the material. Moreover, it was not realized until quite late that the information captured in the Web server access-log file was not adequate to determine the students' actual navigational tracks because of Web client-side cacheing. Simply speaking, a Web browser, which is the interface to the system, can store in its local cache a certain number of Web pages previously requested by its users. Every time a request for a Web page is made by a user, the browser checks to see if the page is in its cache. If not, the browser sends a request over the network and then stores the received page in the cache. Otherwise, it picks up the page from the cache without requesting the server to provide that page. In such a case, the server access-log file will not include this user request. Obviously, the cacheing function is able to save time and cut down traffic over the network. However, it posed a problem for this study in its attempt to find how the material on a server is accessed by a user. The evaluation effort described so far concentrated on only one, but probably the most important one, of our objectives for using the online resource, that is, to facilitate student learning. The other longer-term objectives were to reduce the cost and staff time spent on the management of course materials. The related questions concerned how easy it was for the tutor to put original resources into the system and to update them afterwards. Compared to the evaluation of student use of the system, the assessment of ease of use for the tutor to deliver resources through the STILE platform was more straightforward. Experience from this study showed that the maintenance of the resource-base was relatively easy through the use of the STILE browser-based editor. However, it was still not easy, especially for a non-UNIX user to build a STILE resource-base even though HTML programming skills were not necessary. As described earlier, to build the resource-base, a bulk conversion of our topic network was carried out from a marked-up file using the STILE batch conversion tool. For a non-UNIX 74 ALT-J Volume 4 Number 1 user, extra skills such as file transfer over the network and a basic working knowledge of UNIX are needed to fulfil this task. This is because the current STELE conversion tool works only under the UNIX environment. Once the topic network was established, the resources (mainly images of crafts) were mounted onto the topic-net using the STELE browser-based editor. This task was easy but tedious because there were a quite large number of images which had to be linked into the topic-net one by one. There are two suggestions to ease this difficulty. First, to improve STELE's authoring facility to allow platform-independent mass conversion enabling the system to convert not only pure topic networks but also to include references to resources from bulk data. Second, for potential STELE resource providers who are computer illiterate, to improve their computing skills simply because it is unrealistic to expect to gain benefit from a new technology without a basic technical understanding. Conclusions and further work The online resource-base has proved to be successful in demonstrating its great potential in meeting our proposed objectives, that is, improvement of the support for Design teaching and learning. The students revealed in the study a great appreciation of the high degree of learner control provided by the system, and expressed a strong demand for more information online concerning the subject area. From the tutor's perspective, the maintenance of the existing resource-base has been made relatively easy by the use of the STELE browser-based editor although the current conversion tool still needs real improvement. Nevertheless, this experience has also raised a range of issues that need to be thoroughly addressed before the innovation can be fully applied to existing teaching and learning activities. Some of these are highlighted below, and might be of general interest to those in a similar situation. The university's IT infrastructure particularly for online access to learning resources needs to be urgently improved, which includes updating the network capacity and the establishment of specialized computer labs for online resource delivery. This has an implication for the way in which the university allocates its resources. A shift from teaching to learning, and a consequent shift in resources from teaching resources to learning resources, might be essential. New strategies and skills need to be developed for both students and tutors. For students, consulting information from an online system is different from consulting information from conventional printed materials. Likewise, for tutors, delivering information in hypertext form requires different abilities from those they have already developed. Teaching styles need to change in order to make effective use of such innovations. In this case, it might be possible to rely heavily on the online system for information delivery so that more staff time can be devoted to actual student contact in helping students develop the practical skills of designing and making things. Guidance on the use of copyright materials should be made available to the academic staff who want to provide educational resources online. Unless the materials are owned by the provider, copyright clearance is necessary for their use or even their adaptation. The lack of knowledge of dealing with copyright appears to be a restraining force in making educational resources available online. Further work on the use of the online learning resource will concentrate on three areas. The first area of work will be to include more information in the resource-base. The contents of the 75 Zhengmai Zhao et al Online learning for design students resource-base will gradually expand to include the whole module of Contextual Basis of Design; links to relevant external resources will be added; materials such as previous students' seminar presentations and transcriptions of group discussions may be integrated; and the use of digital video, animation, and 3D objects will also be explored. The second area will involve encouraging students to become not only information consumers but also information providers, and to make use of communication facilities supplied by the STILE platform. Finally, the possibility of replacing facts-based lectures with a well-built online resource system will be further investigated. Acknowledgements The authors wish to express their gratitude to all members of STILE project team and those students who participated in the study described here. References Berners-Lee, T., Cailliau, R., Luotonen, A. Nielsen, F. and Secret, A. (1994), 'The World- Wide Web', Communications of the ACM, 37 (8), 76-82. Blumberg, R. B. (1994), 'An electronic science/math/history resource for the WWW', The Second International WWW Conference: Mosaic and Web, Chicago, Illinois, USA, October 17-20, 1994, http://www.ncsa.uiuc.edu/SDG/IT94/proceedings/educ.html. Draper, S. W., Brown, M. I., Edgerton, E., Henderson, F. P., McAteer, E., Smith, E. D. and Watt, H. D. (1994), 'Observing and measuring the performance of educational technology', Technical report, TILT project, University of Glasgow. Rebelsky, S. A. (1994), 'A web of resources for introductory computer science', The Second International WWW Conference: Mosaic and Web, Chicago, Illinois, USA, October 17-20, 1994, http://www.ncsa.uiuc.edu/SDG/IT94/proceedings/educ.html. Ruggles, C., Underwood, J. and Walker, D. (1995), 'STILE: a hypermedia approach to the provision of a flexible, interdisciplinary resource base for teaching and learning', Innovations in Education and Training International, 32 (3), 209-19. Shneiderman, B. (1992), Designing the User Interface: Strategies for Effective Human- Computer Interaction (2nd edition), Reading MA, Addison-Wesley. Zhao, Z. and Patel, D. (1995), 'An online resource-based learning environment' in Jonassen, D. and McCalla, G. (eds), Proceedings of ICCE 95 - International Conference on Computers in Education, Singapore, 5-8 December 1995, 315-23. 76