phillips Australian Journal of Educational Technology 2001, 17(1), 96-114 The use of a QTVR image database for teaching veterinary radiology and diagnostic ultrasound to distance education students Rob Phillips Murdoch University Romana Pospisil Edith Cowan University Jennifer L Richardson Murdoch University The result of the project described in this paper is an innovative use of Quicktime Virtual Reality (QTVR) for display and manipulation of veterinary radiographs and ultrasound images, within a database developed for use in the Veterinary Diagnostic Imaging unit of the Master of Veterinary Studies at Murdoch University. One of the aims of this project was to find an alternative for the bulky sets of radiographs used by external students for their case based coursework and assessment activities. The QTVR solution provides a means by which students can move and zoom within images, resize images and compare images side by side. QTVR also allows important areas of images to be highlighted by hotspots, allowing annotation of images, which is helpful for assisting external students. Some of the priorities of this project were to maintain the detail and the depth of the hard copy radiographs in the QTVR images, to simulate the problem solving process used in reading radiographs, and to improve the learning outcomes by highlighting and annotating important areas of images. It was hoped that this solution would provide a more cost effective and convenient method of delivery of large numbers of images to external students. In the subsequent cost benefit analysis, it was found that the innovation described here offers many economic advantages to the School of Veterinary Clinical Science. Introduction This paper reports on the innovative use of QuickTime Virtual Reality (QTVR) in a CD-ROM based database of radiographs and ultrasound Phillips, Pospisil and Richardson 97 images, for use in the Veterinary Diagnostic Imaging unit in the Master of Veterinary Studies offered by Murdoch University. The Master of Veterinary Studies is an advanced coursework Masters degree available internationally through distance education, and targeted at professional veterinarians in practice. This project was initiated by the need to find solutions for several problems inherent in such external practical based units. In this case, an alternative was sought for bulky study materials and teaching aids sent to students in remote locations, as well as a means of improving communication with students working on imaging based cases experienced in the unit and in their professional practice. The Masters degree enrolment is small (49 students in both 1999 and 2000) and it is offered in traditional print based mode, with some online and technology mediated components. Individual units include the use of email, some online tutorials and discussion forums, in conjunction with print based materials, video, radiographs, ultrasound images and microscope slides. In some units, a custom built computer program is used for case based problem solving exercises. Currently, enrolment numbers are restricted by the cost of producing sets of unit materials such as radiographs and pathology slides. The Masters unit V620, Veterinary Diagnostic Imaging, is typical of a unit where the number of sets of radiographs limits the enrolment numbers, even though there is a growing demand for places. In 1998, the number of students enrolled was 25, but in 2000 it was limited to 22, because damaged and lost materials had limited the amount of available resource materials. It was thought that Masters degree enrolment numbers could increase if appropriate use was made of online and Interactive Multimedia (IMM) technologies. Background V620 includes four sections covering: radiographic technique and ultrasound principles; and the imaging and interpretation of the skeletal system; the abdomen; and the thorax, in small animals. In this unit the major teaching strategy is for students to work on case studies in their coursework and assessment, using large numbers of radiographs and ultrasound images demonstrating different patterns of disease in the various anatomical regions. The case studies are supported by print based materials. By using the radiographs and ultrasound images, students learn the principles of recognising normal and abnormal structures and learn to define the patterns of change present in disease processes, and make diagnoses from these. If students experience difficulties in recognising radiographic or ultrasound changes or diagnosing an abnormality, they can 98 Australian Journal of Educational Technology, 2001, 17(1) discuss the case with their lecturer via telephone. Once the non-assessable worked case studies component is complete, students undertake case evaluations using radiographs or ultrasound images and complete case related assignment questions. The teaching and learning problem In offering V620 in external mode, there are several ongoing practical problems, which are either logistical or educational: • logistical issues of high costs and administrative load in producing and distributing radiographs • the logistics and inconvenience to students of handling and transporting bulky radiograph files • teaching problems caused by lack of interaction between teacher and student Logistical problems The preparation and delivery of the materials to students is both labour intensive and costly, with the need for multiple sets of radiographs to be copied, labelled, packaged and mailed to students each semester. Table 1 lists the various cost components for offerings of the unit in 1998 and 2000. If sets are lost or damaged in transit, it is too expensive for the School to replace them, which is why only 22 students were accepted in 2000. In addition, new cases cannot be readily added. Even the ongoing expense of providing materials to students is hard to sustain. Table 1: The cost of providing copies of radiographs to students. Averages per student in parentheses. Fixed costs Cost of new radiograph sets (200 hard copies) $2,748 Cost per new hard copy image $14 Marginal costs 1998 Unit 2000 Unit Number of students 25 22 Cost of maintaining radiograph sets $1,030 ($41) $1,000* ($45) Postage $3,510 ($140) $5,340 ($243) Administrative handling $1,000 ($40) $1,000* ($45) Total marginal cost $5,540 ($222) $7,340.00 ($334) * estimate Phillips, Pospisil and Richardson 99 Since 1998, the average cost of providing radiographs to students has increased from $222 per student to $334 per student, mainly due to postage costs for overseas enrolments. It is clear that these high costs limit the potential of the unit to increase enrolments. Studying with the hard copy radiographs can also be inconvenient. Students enrolled in the Masters degree are mostly practising veterinarians seeking further to develop their skills. Using hard copies of images means that students require access to a viewbox, usually located at their workplace, while most of their coursework can be done from home. Students therefore tend to be transporting radiographs in bulky files from one location to another, which can be a considerable inconvenience. Teaching problems Apart from tackling the practical issues involved in the supply and handling of radiograph and ultrasound images, there was clearly a need for improved methods of communication and provision of feedback to students. The external mode presents difficulties in achieving a satisfactory level of interaction between teacher and student. There is an unavoidable delay in students receiving feedback on assignments sent by post. In previous years, attempts have been made to improve contact between teachers and students by using email and individual telephone discussions. Teleconferencing has been used for group based discussion of case study radiographs. However, this proved to be costly and often unsatisfactory, as it was difficult to discuss images without being able to pinpoint or highlight areas of interest. Online learning environments, such as WebCT (WebCT Educational Technologies, 1999), can facilitate communication between the student and tutors, as well as communication with other students (Collis, 1996; Harasim et al., 1995, summarised in Phillips & Luca, 2000). At this stage, we have included a WebCT discussion forum to improve teacher-student and peer- peer communication. However, it was recognised that other approaches needed to be used to provide for: • opportunities for interaction with learning resources and the tutor in the absence of face to face technique sessions • opportunities for dynamic discussion of images • student feedback on specific image problems 100 Australian Journal of Educational Technology, 2001, 17(1) • teaching subtle recognition without visuals These requirements were seen as being beyond the scope of the initial project, but we will return to them in the Conclusion. Before expending effort on designing interactive activities, it was first necessary to establish the feasibility of the general approach to providing electronic images to students. The rest of this paper is about the implementation of a radiographic database and CD-ROM and describes the technical design and formative evaluation of the two trial CD-ROMs produced for the unit in first semester 2000. Requirements for an electronic solution It should be clear from the preceding sections that educational issues were paramount and that the technology was seen as a tool to support the educational needs of students. In particular, the focus was on addressing a specific area of student need, using a considered learning design with innovation being integrated into the learning experience. These factors were identified as essential elements of projects contributing to successful learning outcomes as described by (Alexander & McKenzie, 1998) in their report evaluating information technology projects for university teaching. In line with their recommendations, the project development team included a skilled project manager and used a proven project management methodology. The brief for the project was to make the radiographs and ultrasound images available to off campus external students through the creation of an image database delivered on CD-ROM. We sought to decrease the mailing and reproduction costs, as well as reduce the inconvenience of handling large files of radiographs. There was also the potential to add value to the educational experience by providing greater flexibility in selecting cases to be included in the unit, and adding tools for highlighting or annotating images. In our explorations of alternatives to the use of hard copies of radiographs and ultrasound images, we started with a detailed analysis of how radiographs are actually used by student veterinarians in case evaluations. In practice, the radiographs are examined with a general light source (viewbox), initially from a distance, subsequently at close range, and finally in fine detail using a focal bright light source. Images and anatomical regions can be displayed side by side for comparison. Students learn to Phillips, Pospisil and Richardson 101 identify an abnormal region, which then requires closer examination, and then locate appropriate items for comparison. From this analysis, we were able to identify several key requirements that had to be met when presenting images in the new format: 1. images should be able to be used in a manner that mimics the actual processes followed when handling hard copies of radiographs 2. the required diagnostic detail within the image must be maintained 3. all students need to be able to easily access and view the images (taking into consideration the large numbers of images used in the unit (more than 200) and the remote locations of some of these students) 4. the normal viewing process should be simulated at multiple levels including the viewing of images from a distance and close up, without restricting the decision making process involved in determining the regions that require further examination in fine detail 5. students need to be able to compare several images at the same time in a similar manner to displaying them side by side on a viewbox 6. the flexibility of viewing the images of their choice should be maintained 7. images should be clearly labelled to facilitate referencing Implementation In the previous section we have described the shortcomings of the traditional, external mode of study in achieving the desired learning outcomes in this unit, and indicated that a technological solution could overcome some of these difficulties. The aim was to replace hard copies with scanned images of radiographs or ultrasound images and provide them to students electronically. We have also specified the educational requirements of the solution. The rest of this section describes technical details of the implementation, starting with the scanning and storage of the individual radiographs. 102 Australian Journal of Educational Technology, 2001, 17(1) Scanning radiographs Hard copy radiographs are physically large, of the order of 43 cm x 35 cm (17 inches x 14 inches). When viewed at full size, they are larger than a normal computer screen, so some mechanism of moving the image around on the screen is necessary. As described earlier, it is professionally necessary for veterinarians to observe the whole image first, then to focus in on areas of interest. When attempting to apply these techniques with scanned images, it is necessary to be able to zoom out to view the entire image, and zoom in for fine detail. The interpretation of X-ray images requires fine discrimination between diffuse areas of grey. To replicate this on the computer requires high resolution scanning at high colour depth, and careful attention to brightness and contrast settings to ensure relevant areas are visible. The combination of large physical size and high resolution implied that scanned images would take up large amounts of storage space on the computer. A 17 in x 14 in radiograph scanned at 300 dpi (dots per inch) in millions of colours would occupy 63 megabytes of disk storage, uncompressed. Only a limited number of such images could be stored on a CD-ROM, and loading a single image would place significant load on even the most up to date computer. In V620, students are provided with approximately 200 images and are required to view several side by side for comparison. At first sight, it appeared that it would be impossible to achieve the requirements (large size, high image quality, moving within the image, zooming in and out) with file sizes which were usable on standard personal computers. However, we then considered Apple Computer Inc.’s QuickTime Virtual Reality (QTVR) technology (Apple Computer Inc., 2000). QTVR is a variant of the QuickTime digital video format, allowing the user to experience a form of virtual reality on their computer screen. There are two types of QTVR movies, panoramas and objects. A QTVR panorama is where the camera is looking outward, and 'panning' through a 360 degree arc. Objects are where the camera is looking inward, and give the impression that the object being viewed is turning around, as if it is held in your hand. A QTVR object is often created by standing the object on a turntable with a fixed camera location. One frame of the object is taken at increments, as the turntable is turned. Phillips, Pospisil and Richardson 103 In both panoramas and objects, the QTVR system software enables the user to move and zoom within the image. QTVR also allows important areas of images to be highlighted by hotspots, which enable annotation of images or the inclusion of hyperlinks to other images. This was seen as particularly helpful for external students. QuickTime VR offered the image manipulation features needed for the radiographs and provided inbuilt compression algorithms to reduce image file sizes. However, the question remained whether QTVR could work with a single radiograph or ultrasound image. After some research, we determined we could achieve this by creating a virtual QTVR object, with only one frame (setting the horizontal sweep angle to 0 degrees). We were then able to perform a feasibility study to determine whether the image quality was adequate and how great the image compression would be. An obsolete XRS RSU1 X-ray Scanner (with a resolution of 146 dpi) and the Ray Vision software was used for scanning most large images. The resolution was marginal for some images, particularly for bone detail and ultrasound images. Consequently, some were scanned with a smaller UMAX A4 flatbed scanner at 300 dpi resolution. This gave a higher resolution, but with the limitation that insufficient light was able to pass through the image to provide the depth of grey scale across the image, to make the features visible. The radiologist needed to perform a quality check on each scanned image and, frequently, several different scans with different contrasts had to be made, to ensure that clinical features were visible. Eventually, the radiologist was satisfied that the scanning and QTVR technique was suitable for all but the abdominal and some thoracic images. However, a higher resolution X-ray scanner would have increased quality and reduced the amount of image preparation time. Scanned images were ‘cleaned up’ in Adobe Photoshop, then imported into the VR Worx application to create the QTVR files. Once it had been established that the approach described here was feasible, the remainder of the images was scanned. Although there were originally 200 hard copies, not all of these were scanned, but some new images were added. In total, 250 images, of varying type, quality and size, were scanned. In some cases, multiple images were displayed on the one X-ray film, but these were produced as separate QTVR files. In total, 319 individual QTVR movies were produced. 104 Australian Journal of Educational Technology, 2001, 17(1) The compression offered by QuickTime also resulted in considerable reductions in file sizes. Given that there was a large range of image sizes, the average size of the scanned images was approximately 2MB, while the average size of the QTVR images was 300kB. This size makes it feasible to deliver images over the web, as well as on CD-ROM. The authors are unaware of any previous use of QTVR to display X-rays for learning purposes. The Anatomedia project (Kennedy, Kennedy, & Eizenberg, 2000) contains radiographic images, but these are of low resolution, intended for the learning of anatomy. The particular advantage of the approach described here is that the QTVR radiographic images are of sufficient detail to enable diagnosis. Kraft et al. (1998) reported on a multimedia program for teaching veterinary orthopaedic radiology. Relatively low resolution images were displayed, and specific areas of interest were displayed as magnified images when clicked on. This program took a didactic approach, leading students linearly through explanations and annotations of the radiographs. Instead of treating the computer as a teacher, in our case we have seen it as a tool to assist advanced students to work through the process of determining the area of importance themselves, using a problem solving approach. Software development Development proceeded according to a project management methodology described in (Phillips, 1997; Phillips, 2001; Phillips et al., 1999). Comprehensive documentation and quality assurance procedures were followed and finely detailed timesheet records were kept. A project plan and timeline were developed. However, the timeline was very tight. The project started in November, in the context of a looming deadline. The unit commenced in mid-January, and students had to have access to their materials by early February. The tight timeline restricted the amount of prototyping and bug testing that was possible. It was anticipated that any unexpected delays would impact severely on the ability to meet the deadline. An initial version, containing approximately one quarter of the images, was distributed to students by the deadline. As more information became available, a second version was developed, building on student experiences with the first. Phillips, Pospisil and Richardson 105 Version 1 Once it became clear that the QTVR images were suitable for teaching purposes, effort turned to finding the most effective way to deliver the material to students. The typical way by which students engage with the printed materials is through case studies. An image or images are presented, together with history and clinical examination information and students are guided through the radiographic features, leading to a radiographic diagnosis of the condition. Because text information is associated with each image and case, a database solution was indicated. The database would enable related information to be kept together, simplifying maintenance, and enabling further cases to be added or deleted in subsequent years. Because of the file sizes and internet bandwidth issues, it was decided to provide the images on CD-ROM. The requirement was therefore for a database which could be distributed on CD-ROM. FileMaker Pro was chosen as the most appropriate database for this purpose, due to its potential for use on the web, and prototyping of the CD- ROM commenced. The database structure was built, but the process of entering data into the database was delayed for non-technical reasons. The administrative staff member responsible for managing the case data was on leave, and, because of version control problems, it was difficult to identify the most accurate case data for inclusion in the database. Eventually, data for the first set of case studies was finalised and written to CD-ROM. The intention was that the rest of the images could be sent out subsequently in version 2. When testing the first prototype of the CD-ROM, it was discovered that the database would not run directly from a CD-ROM. The short term solution was to install the database from the CD-ROM onto the user’s computer, and run it from there. The impact of the delay in entering database content was that the CD-ROM was not able to be tested on a wide range of computers and hardware configurations before being sent out to students. Subsequently, a number of students reported difficulties in being able to use the CD-ROM on their existing hardware. The CD-ROM provided to students was relatively resource intensive, compared to IT requirements associated with previous units in the Masters degree, and those disseminated to students at the start of the project. In particular, it was very time consuming on some computers to install the images onto the hard disk. In addition, some students could 106 Australian Journal of Educational Technology, 2001, 17(1) not get the QTVR images to display, either because there was insufficient RAM available for the QuickTime software, or because it was not installed. The initial problems clearly affected the views of some students towards the unit. However, students having computers with the specifications defined for the unit reported few difficulties with the first CD-ROM. Nevertheless, there were clear perceptions from a number of students that the technical difficulties impeded their learning. The provision of technical support by the development team may have ameliorated these feelings. The development team anticipated problems in implementing a new piece of software at short notice, and provided students with complete hard copy materials for the first section of the unit. In addition, a total of 69 cases using 152 images was used for the second section and provided in QTVR format on the CD-ROM. Unfortunately, due to financial constraints, only 34 hard copy radiographs were provided for this second section. Version 2 The development of the second version was also pursued in the context of tight timelines. The slow installation process and inability to work from the CD-ROM resulted in the FileMaker Pro database being dropped for the time being. Instead, all images were stored on the CD-ROM, and a simple navigational shell was written in Macromedia Director to enable students to locate and open QTVR images. This solution had the advantage of speed and ease of use, but at the expense of the contextual material stored previously in the database. In this version, students referred to paper based information about the cases, and most of the images were also supplied as hard copy. We always envisaged that version 2 would be a short term solution to an immediate problem, and that a more robust solution would be developed based on student feedback. Formative evaluation It was essential to discover the views of students towards the use of the innovations described in this paper. The evaluation framework proposed in the ASCILITE Evaluation Handbook (Phillips, Bain, McNaught, Rice, & Tripp, 2000), derived from Bain (1999), was used to inform the evaluation strategy. In particular, the evaluation focussed on two aspects of the framework: Phillips, Pospisil and Richardson 107 • formative monitoring of learning environment (to determine whether the innovation is functional in its context and accessible/attractive to students) • formative monitoring of learning process (to determine whether the innovation is influencing the learning process as intended) The 22 students were sent an email message seeking feedback about their experiences with the unit. We wanted to find out what they felt about the use of the QTVR images and how they used them in their learning. The questions were designed specifically to elicit descriptive and deep responses from the students. Responses were received from 7 students, and the benefits and disadvantages of using both the CD-ROM and hard copy methods of studying are summarised in Table 2. Major issues are discussed in more detail below. Table 2: Characteristics of the CD-ROM and hard copy methods of studying Veterinary Diagnostic Imaging. CD-ROMs Radiographs Useful for studying and self tests Better for marked assignments Better image quality Good for gross detail Good for fine detail Could be viewed at home or work - preference for home Had to be viewed at work Students unfamiliar with this approach Students familiar with this approach Convenient in terms of time and place Cumbersome and time consuming Initial technical problems Good for ultrasound images Need relatively powerful computer Relatively cheap to distribute Very expensive to distribute Hotspots very helpful Hard to identify abnormalities from written descriptions Zooming capability helpful, but not as good as viewboxes CD-ROMs versus hard copies The technical issues referred to earlier clearly affected some students’ perceptions: 108 Australian Journal of Educational Technology, 2001, 17(1) I hate the CD-ROM's. Maybe my views are tainted by the problems I had originally with viewing the CD-ROM's and the fact that as a result I am behind and struggling to get up to date again. However, other students overcame their early poor perceptions: Initially when I was having computer problems I was very frustrated with the CD-ROM, however once the problem was rectified I found it very useful and an efficient use of time. I think it would be good for future students to utilise. Students tended to use the CD-ROMs for studying and self tests. However, all respondents reported that they used the hard copy radiographs for the marked assignments. This was because the students felt that the radiographs had a better image quality. I found the hard copy radiographs easier and better to look at as these are more realistic and easier to image - particularly found the orthopaedics difficult to really look at due to the small size. However, the CD-ROMs were seen as more convenient in terms of time and place, and some students enjoyed the flexibility of having both methods available. I felt that both methods were important and used both at different times and in different places. The radiographs had to be viewed at work and these I tried to do at quiet times in between consults and in breaks so that I didn't have to go in to work too often on days off. Two students reflected that their preference for the radiographs may have been due to their familiarity with this technique. I probably feel comfier with the radiographs as that is what we are all used to the most. In addition, the radiographs had the drawbacks of being cumbersome and time consuming. Students had to transport books and notes to a vet treatment room with a viewer, and then try to write their reports from there, when the room was not suitable for that purpose. The feedback from students indicates that they would like to receive images in both modes. The CD-ROM images were found to be suitable for study purposes, because of the flexibility and hotspots. However, students wanted hardcopies for the assignments, because of a perceived higher image quality. A mixed approach will significantly reduce unit costs the next time it is offered. Phillips, Pospisil and Richardson 109 Unit structure, external studies and communication Issues Students generally found the unit to be well structured and a positive experience: I have enjoyed the unit and found it informative and have learnt a lot from both hard copy and CD-ROM X-rays - a well structured and well written unit. I have found the course very informative and have enjoyed the teaching material that you have sent… It has been a great opportunity for people like me who have been out in practice for over 20 years to catch up with all aspects of their subject and to enjoy the benefits of new teaching methods. Students reported some difficulties in studying externally, but recognised that there were few alternatives to them if they wanted to continue their veterinary practice. They had found that the most restrictive part of studying other external units in the Masters degree was the lack of practical work. They found that the multitude of radiographic and ultrasound images provided in this unit met some of their practical needs: I think the practical component of this unit is great. Several students found that the workload in this unit was greater than in previous units in the Masters degree, and one student also found it more difficult: This course would be wonderful if we had the luxury of twice the amount of time to do it in! IT literacy issues Students’ levels of IT literacy were also important. Some students reported an initial lack of confidence and hesitancy at engaging with the technology. However, several students reported that they have overcome their anxieties and are engaging with technology and the tools it provides. This is my second year on this course and therefore my second year at using a computer so I am getting over the shock of it all now and coping much better. I feel quite happy about it now... Students saw important advantages in working in a technology supported environment. The use of email and the web for communicating with lecturers was seen as very positive and better than telephone contact. The asynchronous nature of email was identified as important given the widespread, and busy, student cohort. 110 Australian Journal of Educational Technology, 2001, 17(1) I think the major benefit of the technology is being able to work independently in a variety of situations - it allows for greater flexibility. I think it is essential in allowing for continuous feedback and access to tutors. Despite the range of problems experienced by the students, they felt that future students would benefit from the use of the CD-ROM based approach. The technology is definitely broadening the horizons for the future. I think the growth of your ideas with the computer programs will make the teaching even better in the future. It will probably enable more information to be sent to us. Cost-benefit analysis This project was funded through Murdoch University’s Teaching Development Fund, with contributions of $19000 from the Division of Veterinary and Biological Sciences, for academic staff release and hardware/software requirements, and $20,000 for work to be undertaken by the Teaching and Learning Centre's Educational Design Group. The Educational Design Group keeps comprehensive timesheet records in order to account for effort expended and to calculate metrics for future software development projects (Phillips, 2001). The hours expended in each phase of the project described in this paper are shown in Table 3. It can be seen that the majority of the effort in the project went into the scanning of the images, followed by the development of the version 1 database. The project management component is consistent with other relatively sophisticated multimedia development projects (Canale & Wills, 1995; Phillips, 1997, p57) Table 3: Hours expended in each phase of the project. Phase Hours Percentage of effort Project Management 75.5 16% Prototyping 36 8% Scanning 213.5 45% Version 1 101 21% Version 2 22.25 5% Website 24.75 5% Total 473 100% Phillips, Pospisil and Richardson 111 Table 4 summarises the cost of production of the 319 QTVR images. It can be seen that the cost metric for the production of a new QTVR image is $27. This contrasts with a cost of $14 for each hard copy image. However, from Table 1, each subsequent hard copy also costs $14, while the marginal cost of production of new CD-ROMs is quite low. This argument is carried further in Table 5, which compares the fixed and marginal costs of producing the teaching resources in both hard copy and CD-ROM formats. The figures in the hard copy column are derived on a per student basis from the rightmost column of Table 1. The up front electronic cost is derived from the 473 hours worked by the development team at $40/hour. The other figures are best guess estimates. Using these figures, the CD- ROM method becomes more cost effective at enrolments of 7 students, and each additional student returns a benefit of approximately $4000. Table 4: Cost metrics for production of QTVR images. Total scanning expenditure $8,540 Number of original hard copy images 250 Cost per hard copy image $34 Number of QTVR images 319 Cost per QTVR image $27 Table 5: Fixed and marginal costs of each method of production. Hard copy Electronic Up front costs (1 set) $2,748 $18,920 Reproduction costs for extra copies $2,748 $10 Postage $243 $8 Administrative handling $45 $1 Maintenance costs $45 ? From Table 1, the average cost of providing hard copy radiographs to students in 2000 (postage and handling) was $334 per student ($7340 in total). However, in 1998, three sets of materials were damaged or lost. These should have been replaced, at $2748 per set. The real cost of maintaining full sets of materials for 25 students is $7340 + 3 x $2748 (=$15,583). This equates to $623 per student. On the other hand, the cost of burning and distributing CD-ROMs is estimated as $19 per student, plus an undetermined maintenance cost. 112 Australian Journal of Educational Technology, 2001, 17(1) There is a benefit to the school of $12100 ($604 per student) each year in using the electronic images. This could be spent each year in maintaining the electronic images without costing the School any more than it does at present. At the same time, extra student income can be earned at a marginal cost of $19 per student (assuming that extra students do not cause extra teaching load). The CD-ROM solution offers the school more flexibility and opportunities for income generation. Conclusion It is clear from the cost-benefit analysis that the innovation described here offers many economic advantages to the School of Veterinary Clinical Science. The formative evaluation of the project, however, indicated that students had some concerns about the suitability of the images, in their current form, for assessment, but were satisfied with them for learning activities. One option is to provide students with hard copies of assessment materials, with other material on CD-ROM. This will reduce the economic benefits to some extent, but maintain educational quality. A second option is to purchase a 'state of the art' X-ray scanner to produce higher resolution images. There are acknowledged problems with both versions of the CD-ROM, and work is proceeding on development of a third solution. In this version, HTML pages will be generated automatically from a database, and stored in a static form, which allows the images to be annotated with contextual information. These pages may be delivered entirely on CD-ROM, or using the WebCT CD-ROM tool. In the section entitled Teaching Problems we identified problems experienced by students working alone with hard copies. Work has begun to circumvent some of these problems, by including interactive tutorials and quizzes. It is intended that interactive activities will replace print based worked case studies and teleconferences, thereby: • easing the burden of marking whilst enriching the learning experience by providing more opportunities for communication and discussion of diagnosis • enabling case studies to become more challenging and educationally effective • providing immediate feedback to students • enabling dynamic annotation of images Phillips, Pospisil and Richardson 113 The approaches being developed in this project have wider applicability in the Master of Veterinary Studies programme, in which many units are case based. The image database and interactive tutorials are designed to be modifiable for other image based units. We are confident that the approach described here will lead to increased enrolments in the Masters degree, as enrolment numbers will no longer be limited to the sets of physical resources available. Acknowledgements This project was funded by a Murdoch University 1999 Teaching Development Grant designed to promote innovation and research in the theory and practice of teaching and learning. The project team members from the Educational Design Group of the Teaching and Learning Centre at Murdoch University were: Nick Castle - Multimedia Programmer Eleanor Chaos - Scanning and QTVR Christine Bailey - Project Coordinator Lisa Masiello - Graphic Designer Terri Sheehan - HTML Author Carol Adair - HTML Author References Alexander, S. & McKenzie, J. (1998). An Evaluation of Information Technology Projects for University Learning. Canberra, Australia: Committee for University Teaching and Staff Development and the Department of Employment, Education, Training and Youth Affairs. Bain, J. D. (1999). Introduction. Higher Education Research and Development, 18(2), 165- 172. Canale, R., & Wills, S. (1995). Producing professional interactive multimedia: Project management issues. British Journal of Educational Technology, 26(2), 84-93. Collis, B. (1996). Tele-learning in a Digital World: The Future of Distance Learning: International Thomson Computer Press. Harasim, L., Hiltz, S. R., Teles, L., & Turoff, M. (1995). Learning Networks: A Field Guide to Teaching and Learning Online. Cambridge Massachusetts: The MIT Press. Apple Computer Inc. (2000). QTVR Home Page. Apple Computer Inc. [viewed 6 Jun 2000, verified 3 Mar 2001] http://www.apple.com/quicktime/qtvr/ 114 Australian Journal of Educational Technology, 2001, 17(1) Kennedy, D., Eizenberg, N., Kennedy, G. (2000). An evaluation of the use of multiple perspectives in the design of computer facilitated learning. Australian Journal of Educational Technology, 16(1), 13-25. http://www.ascilite.org.au/ajet/ajet16/kennedy.html Kraft, S., Hoskinson, J. J., Mussman, J. M., Michaels, W. E., McLaughlin, R., Gaughan, E. M., & Roush, J. K. (1998). Development of interactive patient-based multimedia computer programs in veterinary orthopaedic radiology. Veterinary Radiology and Ultrasound, 39(2), 98-104. Phillips, R. (1997). The Developer's Handbook to Interactive Multimedia - A Practical Guide for Educational Applications. London: Kogan Page. Phillips, R., Bain, J., McNaught, C., Rice, M., & Tripp, D. (2000). Handbook for Learning-centred Evaluation of Computer-facilitated Learning Projects in Higher Education (1.0). Australasian Society for Computers in Learning in Tertiary Education. [viewed 17 Apr 2000, verified 3 Mar 2001] http:/cleo.murdoch.edu.au/projects/cutsd99/handbook/handbook.htm Phillips, R. A. (2001). A Case Study of the Development and Project Management of a Web/CD Hybrid Application. Journal of Interactive Learning Research, in press. Phillips, R. A. & Luca, J. (2000). Issues involved in developing a project based online unit which enhances teamwork and collaboration. Australian Journal of Educational Technology, 16(2), 147-160. http://www.ascilite.org.au/ajet/ajet16/phillips.html Phillips, R. A., Pospisil, R., Bell, J., & Patterson, A. (1999). Meeting Needs: A staff development resource for redesigning sociology courses according to an outcomes-based model. Proceedings Australasian Society for Computers in Learning in Tertiary Education Conference, Brisbane, Australia. http://www.ascilite.org.au/conferences/brisbane99/papers/phillipspospisil.pdf WebCT Educational Technologies. (1999). Web CT: World Wide Web Course Tools. http://www.webct.com/ Rob Phillips Teaching and Learning Centre Murdoch University, Murdoch WA 6150 r.phillips@murdoch.edu.au Romana Pospisil Faculty of Business and Public Management Edith Cowan University, Churchlands WA 6018 r.pospisil@ecu.edu.au Jennifer L Richardson School of Veterinary Clinical Science Murdoch University, Murdoch WA 6150 jennyr@numbat.murdoch.edu.au