kennedy Australian Journal of Educational Technology 2000, 16(1), 13-25 An evaluation of the use of multiple perspectives in the design of computer facilitated learning David Kennedy Monash University Norm Eizenberg and Gregor Kennedy The University of Melbourne In the medical sciences, there is a well documented problem that involves the separation of discipline based knowledge from clinical practice. In order to address this issue in anatomy teaching, the CD-ROM An@tomedia has been developed - integrating anatomical knowledge with clinical applications. The structure, and evaluation of An@tomedia are described. Introduction The teaching of anatomy at The University of Melbourne has, over the past 10 years, integrated the structure of the human body with its clinical applications. The process was initially supported with the development of an innovative textbook. With the advent of new computer based technologies the innovation has extended to the development of an interactive CD-ROM to enrich and engage the student more actively in the learning experience. The software, An@tomedia, provides greater interactivity for the student by providing multiple perspectives (e.g., systemic and regional views) of anatomical knowledge linked to clinical applications in an interactive computer facilitated learning (CFL) environment. The perspectives provide a framework for students to integrate anatomical knowledge with clinical application and procedural knowledge. At all times, the students have the ability to switch between perspectives in order to facilitate their knowledge construction. In section 1 of this paper, the history of the development of An@tomedia is elucidated, followed by an brief examination of what we know about student learning and how this has influenced the educational design of An@tomedia. In section three, the initial evaluation of An@tomedia by 14 Australian Journal of Educational Technology, 2000, 16(1) students, tutors, and content experts is presented. The final part of the paper discussed the resulting changes that have been made to the final release version of the software, currently in use at The University of Melbourne by (first year) undergraduate medical students. An@tomedia: A history An innovative anatomy teaching program was initially developed for medical students at The University of Melbourne (Eizenberg, 1988). This program (primarily print based) aimed at encouraging an understanding of anatomical principles which could be utilised in future clinical contexts. Although students found the new program highly successful, its value was limited by the primarily descriptive teaching materials then available. It was apparent that even if a student intended to search for meaning, (ie., adopt a deep approach to learning) this endeavour may be restricted by the content or organisation of the prescribed text. A dilemma emerged— should some of the innovations be abandoned in order to accommodate existing textbooks or should new learning materials be developed? Further study (Driver & Eizenberg, 1989) provided a clear direction—students strongly recommended that the learning materials be designed around the course rather than the reverse. New learning materials compatible with the improvements in the teaching program were commenced. Thus, the development of the An@tomedia CD-ROM has evolved from a series of action research cycles aimed at improving the learning of anatomy (Eizenberg, 1991). The first phase involved the development of an innovative text book design. The structure of this text was based upon research that investigated the principle factors in a textbook that encouraged students to adopt a deep approach to learning (Driver & Eizenberg, 1995). The conclusions of that study included the: • learning approach used while studying from a text book is influenced by the design of that text book, • text can influence the approaches adopted in other learning activities, • text can influence student’s intentions and perceptions regarding the subject, • referential and structural aspects of the learning context (including the text) are critical in order to encourage the designed student learning approach. (after Driver & Eizenberg, 1995) Kennedy, Eizenberg and Kennedy 15 The critical design factors in the content and organisation of the textbook, linking anatomical details with the clinical perspective has been extended and enhanced in the An@tomedia CD-ROM. This has been done by the use of: • high quality images (e.g., radiographs, detailed dissection images from real human bodies, and graphics with coloured overlays highlighting key anatomical details), • integration of clinical questions and answers, and • the facility for the student to examine anatomical concepts from multiple perspectives using multiple pathways. Bowden and Marton (1998) strongly support the need for integrating the knowledge from multiple perspectives and a range of clinical contexts to enable students to function successfully as professional doctors. When medical graduates enter the workplace, they are not faced with situations labelled ‘anatomy’, ‘microbiology’, or the like. They are faced with patients with illnesses... (p. 129) Student learning and An@tomedia A student’s approach to learning is a qualitative aspect of learning—that is, how the subject matter is experienced and how she or he subsequently organises and integrates knowledge. The student’s perception of the learning context affects the approach adopted (Ramsden, 1984). Students may adopt a surface approach (e.g., learning by rote) or a deep approach (e.g., a holistic approach, actively searching for meaning) (Biggs, 1989; Marton & Säljö, 1984). The student’s learning process may be supported by the application of appropriate learning opportunities by the teacher, including selection of a text book, or the use of computer facilitated learning (CFL) within the context of a subject. The use of multiple perspectives (and multiple pathways) in computer facilitated learning (CFL) environments has been advocated in order to: • allow learners to access the material in a manner more suited to her or his needs (Reeves, 1992), • support higher levels of cognition (Ramsden, 1992), and • support a more reflective approach to learning (Laurillard, 1993). In traditional models of teaching anatomy, either a systemic or a regional approach was adopted. Initially, An@tomedia developed with the intention of integrating these two perspectives—the systemic components 16 Australian Journal of Educational Technology, 2000, 16(1) within regions, and regions that were part of (wider) systems. However, in order to provide a richer learning environment, there was a need to embed clinical, procedures, imaging, surface anatomy, and dissection components within the software and integrate these with the first two perspectives. Figure 1 is representative of the final educational plan of An@tomedia. In Figure 1, students may deconstruct the body using Dissection and Imaging, or construct the body via Systems and Regions. In Figure 1, an example involving the back (the first module to be completed) is highlighted. The figure shows a model of the educational design of An@tomedia in a three dimensional space. The X-axis represents the anatomical regions (only three regions of the body are shown), the Y-axis represents anatomical systems (only four are shown), and the vertical axis represents students ability to construct or deconstruct the body by dissection or imaging. Students may move freely between perspectives, answer questions, and investigate clinical problems about the back. Figure 2 shows the view seen by a student examining anatomy from one perspective, dissection. The remaining 3 perspectives are listed on the lower right—reached with a simple mouse click. There are questions for the student to answer (e.g., ‘Can you identify the thorocolumbar fascia’), and overlays of anatomical structures that may be highlighted by the student (the ‘n’ symbol highlights the nerves, shown in white in Figure 2). The student may also search for specific items using the index feature, and in the future, move between specific regions by clicking on the regions on the human symbol on the lower right. An@tomedia has the potential to be utilised by a range of health professionals in a variety of contexts. However, this paper focuses on initial evaluations by undergraduate medical students and anatomy tutors at The University of Melbourne, and by content experts. The evaluation The evaluation of An@tomedia has adopted a case study approach. Data was generated from: • a questionnaire for students (five point Likert questions and open ended free text responses); • a questionnaire for tutors—all medical doctors undertaking postgraduate study (five point Likert questions and open ended free text responses); Kennedy, Eizenberg and Kennedy 17 Fi gu re 1 : G ra ph ic al r ep re se nt at io n of th e A n@ to m ed ia p er sp ec ti ve s 18 Australian Journal of Educational Technology, 2000, 16(1) Fi gu re 2 : T he d is se ct io n pe rs pe ct iv e w it h qu es ti on s an d ov er la ys Kennedy, Eizenberg and Kennedy 19 • a questionnaire for content experts (primarily medical educators in higher education); • observer notes during the evaluation with the students by the first and third authors; and • a focus group with the tutors. The student evaluations were undertaken with undergraduate medical students over two days in the final week of their first year medical course. Each student used the application for approximately one hour (in some cases longer) and then completed an online survey (Kennedy & Ip, 1998). The tutors engaged in the process during the following week (including the focus group). At the time of writing, not all responses from the content experts had been received. The questionnaires for the students and tutors had an approximately 60% overlap for the Likert style questions. This was the case because some questions are more appropriate to students (e.g., It helped me understand anatomy) or tutors and content experts (e.g., I found the four perspectives were well balanced and integrated). Anatomy tutors and content experts also answered specific questions on how they might use the CD-ROM as part of their own teaching program. An@tomedia is not intended as a replacement for an anatomy teaching program, instead it is intended to support integration of student knowledge, the anatomical with the clinical, and systemic with regional. Results The evaluation results highlight the strengths and potential weaknesses of the educational design of An@tomedia. Trends in the results suggest directions for improvement as well as the value of the multiple perspectives. Table 1 presents the mean scores of both students and tutors responses (responses were completed on five point Likert scales where a score of ‘5’ was ‘strongly agree’, ‘4’ was ‘agree’, ‘3’ was ‘neutral’, ‘2’ was ‘disagree’, and ‘1’ was ‘strongly disagree’). The low number of tutor and content expert responses prevented t-tests from being employed. As this evaluation was part of an initial formative evaluation, this was not perceived to be a serious problem and an analysis of trends was relied upon. 20 Australian Journal of Educational Technology, 2000, 16(1) Table 1: Mean and standard deviation (SD) scores for first year medical students, tutors, and content experts Question Mean (SD) Score The content on the Anatomedia program. Students (n=132) Tutors (n=8) Experts (n=9) Anatomedia complements the anatomy teaching program 4.5 (0.71) 4.6 (0.48) 4.6 (0.73) Concepts were addressed and well explained. 4.2 (0.74) 4.1 (0.93) 3.9 (1.17) The content on the back was comprehensive. 4.3 (0.84) 4.9 (0.33) 4.0 (1.00) There was strong correlation between structure and function. 3.8 (0.99) 3.6 (1.11) 3.6 (1.51) The clinical applications seemed relevant and appropriate. 3.9 (0.83) 3.9 (0.83) 3.8 (1.48) While using Anatomedia: it helped me understand anatomy 4.2 (0.77) it helped me organise my knowledge of anatomy 3.8 (0.94) I found the ability to highlight structures in the images valuable 4.5 (0.67) 5.0 (0) 4.6 (0.73) I found the layering of images valuable 4.6 (0.69) 5.0 (0) 4.8 (0.44) I found the content to be engaging 4.0 (0.81) 4.4 (0.70) 3.8 (1.20) Regarding learning the content: An@tomedia helped me understand anatomy from multiple perspectives 4.0 (0.78) An@tomedia enhanced my ability to interpret anatomy rather than merely describe it 4.0 (0.94) I found the four perspectives were well balanced and integrated 4.3 (0.43) 4.1 (0.99) As a result of using An@tomedia students are likely to improve their knowledge of the anatomical basis of clinical procedures 4.1 (0.60) 4.0 (1.41) application of anatomical knowledge to solve clinical problems 4.1 (0.64) 3.6 (1.41) Kennedy, Eizenberg and Kennedy 21 The students were also asked how they would prefer to use the An@tomedia CD-ROM (Table 2). It is significant to note that while over 90% wanted their own copy, 53% indicated they would use it in study groups. The observations made by the first author during the evaluation process supported this response. Pairs of adjacent students engaged in discussion of the key aspects of the different representations of anatomical structures (e.g., one student may have been looking at the radiological view while the other the dissection). Students were also asked a number of open ended questions. Student comments about the best aspects of An@tomedia included: • Anatomical content was superb. Images (dissection, imaging etc.) were beautiful. • Explanations are very clear and highly comprehensive. • Love the clinical relevance and concise exam style questions. • It was simple yet complex. The diagrams to help answer the questions were fantastic and very helpful (helps you understand) • It was a combination of an atlas and text book which is fantastic. It saves you having to flip (between an anatomical atlas and anatomical text)... Table 2: Students preferred use of An@tomedia for learning: Frequency and percentage How would you best like to see Anatomedia used? You may have more than one option. F % Own copy 122 92.4 Tutorials 43 32.6 Lectures 36 27.3 Study groups 70 53.0 The student comments about the aspects that needed improvement included: • Absence of a “search” mechanism for “random access” into a particular concept or principle. • (add an) adequate index from which to link into appropriate slides illustrating the concept or principle. The comments from the students regarding the need for an index and search engine were expected. However, it was necessary to undertake an evaluation before the first release of the program and integration into the 22 Australian Journal of Educational Technology, 2000, 16(1) new problem based learning (PBL) curriculum in order to refine and improve the initial design. The index and search engine are now complete. The tutors were engaged in the role of supervisors, teachers and mentors, particularly in the activities in the dissection room. Comments from tutors included: • [the best feature was the] Combination of aspects of atlas, textbook, and clinical text in one resource. • The radiology component was fantastic – the resolution of the images very clear. The dissections were also impressive. Tutors were also interested in An@tomedia as a teaching resource. They perceived two potential uses for the CD-ROM—as a precursor for students prior to dissections, and access during dissection classes to clarify and illustrate anatomical relationships. The latter has implications for resources—there would be a need to install video projection and computer facilities in the dissection laboratories and provide training for tutors to use the equipment. Responses from the content experts were more diverse (e.g., the values of the standard deviation, Table 1), with both very positive and very negative comments. The content experts included practitioners (experts in the content) who were occasional lectures, to full time medical educators. Some comments were very positive. • [the best feature was the] Integration of in vivo, in vitro eg., dissections/ imaging. Excellent dissections and graphics to highlight structures. Interactive format and quantity of information. • [the best feature was the] Anatomical detail very good, especially the imaging component Less favourable comments included: • Everything is too static. It is a portfolio of drawings and pictures with (some) accompanying text. The CD is used as a filing system but under used for constructive education. In reflecting on the comments from content experts it is important to recall that An@tomedia is designed to be implemented as part of a PBL curriculum—it is not intended that An@tomedia become the sole resource for the teaching of anatomy. One outlook on teaching involves the view that a lecturer constructs learning opportunities for students—some of Kennedy, Eizenberg and Kennedy 23 which may be facilitated by a computer. The teaching methods chosen as the focus of the learning opportunities, teacher centred or student centred or a mixture of both, are strongly influenced by the educational beliefs held by the lecturer (Kennedy & McNaught, 1997). For example, in response to the question 1(l) “Which of the factors above do you believe are the most important in influencing student learning?”, one content expert replied: • None of these factors are strong determinants. The cardinal determinate is what is in the exam. Wherefore, assuming that is what is shown is all that needs to be known, the most relevant qualities are: locate information, graphics clear, how much information, concise. In all discussions with the An@tomedia development team, the need to provide a learning environment that addressed the traditional divide between discipline based knowledge and clinical practice was paramount. An@tomedia has been designed to be one component within a particular educational context—one which is student centred, rather than teacher centred. In Table 2, only 27.3% of students perceived that An@tomedia would be appropriate for use in lectures—a teacher centred learning environment. However, over 50% wished to use the CD-ROM as part of group activities, and over 90% wanted their own copy for personal study and revision. There may be a need in the future to include with the CD- ROM the background perspective that guided its development, and suggestions (for students and teachers) for ways in which it might be utilised, particularly if An@tomedia is to be used in other contexts. Summary, conclusions and future directions Research has shown that many of the design factors incorporated into An@tomedia are likely to encourage and support a deep approach to learning. The evaluation of perceptions of these key design factors in An@tomedia were, with a few exceptions, very positive. The use of multiple perspectives has received strong support from students, tutors and content experts. The comments regarding the comprehensive nature of the four perspectives, allowing the student to move from one representation of knowledge (e.g., imaging and dissection) to the clinical applications has been highlighted (by students and tutors) as one of the most important strengths of An@tomedia. More than one student indicated the most serious problem with the software was that only the Module on the Back had been completed. 24 Australian Journal of Educational Technology, 2000, 16(1) The tutors (in the focus group) also highlighted the usefulness of An@tomedia for their own study, indicating that An@tomedia has the potential to be used at both undergraduate and postgraduate levels. The content experts, in particular, were asked specific questions that focused on the use of An@tomedia for teaching. These responses have highlighted the need to address the curriculum and assessment in which the CD-ROM is used. As indicated in the discussion, there are some issues of implementation and educational perspectives to be resolved if An@tomedia is to be used outside the context of the current medical program at The University of Melbourne. It is anticipated that more compelling conclusions regarding student learning will be drawn from the summative evaluation studies (focusing on student learning) in 1999. While An@tomedia is focused on the content area of anatomy, the results of the evaluation indicate there are potential lessons to be learnt from the use of multiple perspectives to link structure and application in other subject areas. Since the evaluation, the search engine, index, and the Abdomen (second module) have been completed and a number of typographical and graphical errors rectified. An@tomedia is now a part of the first year undergraduate medical program at The University of Melbourne. In 1999, two more modules, the Thorax and the Neck will be completed. Sample images from the CD-ROM may be viewed at http://www.anatomedia.com/ Acknowledgements The authors wish to thank the content writers, Dr Norm Eizenberg, Dr Chris Briggs, Dr Ivica Grkovic and Ms Priscilla Barker, and the Committee for the Advancement of University Teaching (CAUT) and the Teaching and Learning (Multimedia and Educational Technologies) Committee (The University of Melbourne), for funding. References Biggs, J. (1989). Approaches to the enhancement of tertiary teaching. Higher Education Research and Development, 8(1), 7-25. Bowden, J., & Marton, F. (1998). The University of Learning: Beyond quality and competence. London: Kogan Page Limited. Driver, C., & Eizenberg, N. (1995). Constructing and deconstructing the human body: Applying interactive multimedia in the learning of anatomy. In J. M. Pearce, A. Ellis, C. McNaught, & G. Hart (Eds.), Learning with Technology: ASCILITE95, Proceedings of the Australian Society for Computers in Learning Kennedy, Eizenberg and Kennedy 25 in Tertiary Education Conference. The University of Melbourne: The Science Multimedia Teaching Unit. Driver, S. C., & Eizenberg, N. (1989). How is a medical textbook chosen? Australian and New Zealand Association for Medical Education Bulletin, 16(4), 36-44. Eizenberg, N. (1988). Approaches to learning anatomy: Developing a program for pre-clinical medical students. In P. Ramsden (Ed.), Improving learning: New perspectives (pp. 178-198). London: Kogan Page. Eizenberg, N. (1991). Action research in medical education: Improving teaching via investigating learning. In O. Zuber-Skerrit (Ed.), Action research for change and development (pp. 179-206). Avebury: Aldershot. Kennedy, D. M., & Ip, A. (1998). Learning Evaluation On-line (LEO): A customisable Web-based evaluation tool. In C. Alvegård (Ed.), CALISCE '98, Proceedings of the Fourth International Conference on Computer-aided Learning and Instruction in Science and Engineering. Chalmers University of Technology, Göteborg, Sweden: Chalmers University of Technology. Kennedy, D. M., & McNaught, C. (1997). Design elements for interactive multimedia. Australian Journal of Educational Technology, 13(1), 1-22. Laurillard, D. (1993). Rethinking university teaching: A framework for the effective use of educational technology. London: Routledge. Marton, F., & Säljö, R. (1984). Approaches to learning. In F. Marton, D. Hounsell, & N. Entwistle (Eds.), The Experience of Learning (pp. 242). Edinburgh: Scottish Academic Press. Ramsden, P. (1984). The context of learning. In F. Marton, D. Hounsell, & N. Entwistle (Eds.), The Experience of Learning . Edinburgh: Scottish Academic Press. Ramsden, P. (1992). Learning to teach in higher education. London: Routledge. Reeves, T. (1992). Effective dimensions of interactive learning systems. In A. Holzl & D. Robb (Eds.), Finding the future: ITTE '92, Proceedings of the Information Technology for Training and Education Conference. Lucia, Brisbane: University of Queensland. Kennedy, D.,.Centre for Learning and Teaching Support, Monash University Email: david.kennedy@CeLTS.monash.edu.au Eizenberg, N., Department of Anatomy and Cell Biology, The University of Melbourne Email: n.eizenberg@anatomy.unimelb.edu.au Kennedy, G., Biomedical Multimedia Unit, The University of Melbourne Email: g.kennedy@med.unimelb.edu.au