seddon.pdf Australasian Journal of Educational Technology 2012, 28(2), 214-231 ICT-supported, scenario-based learning in preclinical veterinary science education: Quantifying learning outcomes and facilitating the novice-expert transition Jennifer M. Seddon, Brenda McDonald and Adele L. Schmidt The University of Queensland Problem and/or scenario-based learning is often deployed in preclinical education and training as a means of: (a) developing students’ capacity to respond to authentic, real-world problems; (b) facilitating integration of knowledge across subject areas, and; (c) increasing motivation for learning. Six information and communication technology (ICT) supported, scenario-based learning (SBL) problems using case studies that integrated information across subject areas were implemented in a second-year genetics course for undergraduate veterinary science students and linked to educational outcomes. On a post-implementation questionnaire, students appreciated the use of authentic scenarios but login records indicated variable engagement among students. Comparison of learning outcomes from SBL-supported and non-SBL-supported content (within and across student cohorts) indicated that exposure to SBL generated quantifiable improvements in learning in both high and low ability students. Despite this, students did not perceive that the SBL activities improved their learning. Thus, ICT-supported SBL have the potential to reinforce connectivity of content across a range of pre-clinical courses, but to facilitate a genuine novice to expert transition may require consideration of students’ perceptions of scenario relevance, their confidence, and how students of differing learning styles engage with such activities. Introduction Inquiry-based learning has a long history in science education (Barrow, 2006) but there is significant debate regarding the most effective methods of implementation. There is little doubt that exposure to rich, complex tasks based on challenging, authentic questions or problems fosters higher-order cognitive skills (Barak & Shachar, 2008; Barrow, 2006; Endler & Bond, 2008; Foote & Fitzpatrick 2004), generates positive learning outcomes (Barrow, 2006; McWilliam et al. 2008) and has particular utility in pre-clinical education programs (Clyman et al., 2007; Harlow & Sportsman, 2007; Hem-Stokroos et al., 2003; Klein, 2006; Wilson et al., 2006). Designing and implementing effective pre-clinical courses represents an educational challenge because development of the cognitive processes that underpin transition from novice to expert commences before students have access to clinical settings. Traditional approaches have emphasised the acquisition of content knowledge, but transition from novice to expert also depends on development of the ability to organise information around core concepts, identify patterns and implement cognitive procedures that allow various strands of knowledge to be retrieved effectively and Seddon, McDonald and Schmidt 215 efficiently and applied in both familiar and novel scenarios (Bruning et al., 2004; Meier et al., 2008). Given evidence that domain knowledge and higher-order procedural or strategic knowledge develop in tandem (Haigh, 2007; Shen & Confrey, 2007; Skrok, 2007), it is recognised that traditional approaches to the teaching and learning of science subjects, with an emphasis on rote learning and rigid, inflexible notions of scientific thought, have limited utility in the education of competent professionals capable of transferring skills and knowledge to situations radically different from those in which they were learnt or developed. Recognition that technical competence is not an end point, but rather a stepping stone in the journey toward proficiency and expertise (Hager, Gonczi & Oliver, 1990; Kelly & Bell, 2000; Kerka, 1998; Klein, 2006; Ringsted et al., 2006) has led to increased interest in instructional methods that emphasise general analytical and problem-solving skills, rather than details of techniques and methodologies that are likely to be superseded (e.g. Barker, 2008). Parallels between professional practice and problem and/or scenario based learning (PBL, SBL) approaches to teaching and learning have led to widespread adoption in clinical training programs. In a broader, educational sense, PBL/SBL is a manifestation of constructivist pedagogies that emphasise intrinsic motivation as the only truly effective stimulus for quality learning. The contextualised, constructivist approach positions the learner as an independent, autonomous agent who must take ownership of their own learning experiences in order to solve authentic, ‘real world’ problems (Jacobs & Newstead, 2000; McMeniman, 1989; Nunan, 1988; Sigelman, 1999). Consequently, it meets both the general educational requirements for engagement and cultivation of positive attitudes and orientations to learning (Basu 2008; Nunan 1988), as well as the development of higher order reasoning and complex, systematic thinking (Barak & Shachar, 2008; Barrow, 2006; Black, 2005). The nature and extent of authentic learning experiences that can be embedded within the early stages of preclinical learning programs are, however, limited. Although clinical placement is a necessary and fundamental element of preclinical programs (Baker et al., 2007; Hem-Stokroos et al., 2003), ethical standards necessitate at least some simulated learning prior to performance of key tasks in clinical settings. As a result, the potential of electronic media and resources to overcome practical and ethical constraints, and augment learning in the early stages of preclinical training, has attracted significant interest (Downes 2002; Gordon 2006). This study reports on development and implementation of a program of tailored, pre- clinical, ICT-supported SBL experiences within a second year undergraduate veterinary science course in animal genetics. Prior intervention in the form of introducing video-based assessment methods highlighting links between genetic concepts and the clinical profession led to an enthusiastic response and increased motivation among the students (Seddon, 2008). This study was undertaken with the aim of extending the contextualised approach to enhance perceived relevance of course content to professional practice and increase learning gain. The specific objectives of the study were to quantify: (i) the nature and extent of student engagement with SBL; (ii) student perceptions of the SBL problems as a part of their overall learning experiences; and (iii) the effects of SBL on learning outcomes. 216 Australasian Journal of Educational Technology, 2012, 28(2) Methods Design and implementation of ICT-supported SBL problems A series of eight novel SBL activities were designed around genetic concepts central to a second year Bachelor of Veterinary Science (BVSc) subject (Animal Breeding and Molecular Genetics) (Table 1). In addition to learning objectives specific to the genetics content, the course lists the graduate attribute of “An understanding of how other disciplines relate to the field of study”. Hence, the SBL development team included education and genetics professionals, as well as research and teaching staff in related clinical and pre-clinical subjects (microbiology, clinical pathology, veterinary public health, veterinary medicine and virology). Working collaboratively, the team identified case studies that integrated knowledge and concepts across affiliated BVSc subjects (Table 1) with clear linkage to career trajectories of veterinary science students. Case studies were then developed into ICT-supported scenarios using the S B L Interactive version 1.0 (SBLi, Centre of Biological Information Technology, The University of Queensland, http://www.sblinteractive.org/) software suite. Each scenario presented the case study and allowed students to move among locations (e.g. a veterinary clinic consultation room, an office, a laboratory, a farm) to gather information, to collect virtual samples and receive laboratory reports if relevant, and to follow links to further reading, including published research articles. Multiple choice and open-ended questions were embedded, which either provided instant feedback on correct responses with explanations or directed students to bring their responses to the linked tutorial session. In some scenarios, key tasks had to be completed or questions correctly answered to allow progression through the scenario. Students were introduced to the SBL in a lecture setting and provided with information about the reasons for their introduction into the course and guidelines and instructions for use of the software prior to their integration into the course in Semester 2, 2008 (Table 1). Scenarios were integrated into the course in a variety of ways: two were self-directed (no tutorial engagement); three were completed independently by students prior to attendance at a traditional in-class tutorial, which included reference to the case study presented in the scenario. Students completed one scenario within tutorial sessions to introduce the task (interpretation of phylogenetic trees), with intervening tutor-led discussion that aimed to ensure students could transfer the concepts to new situations presented by the tutor. The remaining two units were not incorporated into the learning program in 2008, but held in reserve to allow variation of the overall programme in future years (Table 1). All scenarios were available to all students; attendance at tutorials was required. Responses to questions within scenarios were not graded. Student cohort Learning scenarios were trialled with eighty-three second-year (25 male, 58 female) undergraduate veterinary science students from The University of Queensland. Students were drawn from diverse socio-cultural backgrounds, as indicated by their recorded ‘permanent country’ of Australia (81.7%), Asian nations (12.2%), South Pacific Island nations (4.9%), and European nations (1.2%). The group also included Seddon, McDonald and Schmidt 217 students of varying ages (school leavers through to those who had undertaken prior tertiary study). As is standard within the BVSc course structure, all students were undertaking the same foundational course of study, passing through the degree program as a single cohort. Table 1: SBL activities developed for veterinary science undergraduates Name Scenario Geneticcontent Non-genetic content Delivery 1. Plasmid resistance Veterinary clinic; drug resistant bacterial infection post-surgery Selection, selection pressure Microbiology Initially independent completion, followed by discussion of results and concepts in tutorial 2. A2 milk Farm visit; selective breeding of dairy cattle to produce A2 milk Allele frequency calculations Biochemistry Self-directed 3. Quantitat- ive genetics (Bull breeding) Farm visit; resolution of infertility issues in breeding stock Inbreeding including calculation of inbreeding coefficients Reproduction Self-directed 4a. Copper toxicosis I Veterinary clinic; clinical case of copper toxicosis in dog (single gene model) Genetic testing, PCR, testing through linkage with markers Veterinary medicine, clinical pathology, histopathology 4b. Copper toxicosis II Veterinary clinic; clinical case of copper toxicosis in dog (multiple gene model) Genetic hetero- geneity, real time PCR Veterinary medicine Initially independent completion, followed by discussion of results and concepts combined from both SBL in a tutorial 5. Phylo- genetics Series of clinical cases (foot and mouth disease on farm; giardia infection in dog; FIV infection in cat; assessment of marine turtle for release) Interpretation of phylo- genetic trees Virology, biosecurity, microbiology, veterinary public health, ecology. Undertaken during tutorial; each case performed by students alone or in small groups interspersed with tutor- led discussion and extension 6. Populat- ion genetics Wildlife park: commenting on impact of proposed freeway on koala population Population genetics, including gene flow, genetic drift, bottlenecks. Ecology, viro- logy, wildlife disease. Broader professional practice Initially independent completion, followed by discussion of results and concepts in tutorial 7. Quantitat- ive trait locus (QTL) analysis International consultancy; breeding tick resistant cattle Genome mapping, QTL analysis, marker-based selection Veterinary pathology, parasitology. * Not implemented in 2008 (scenario reserved to allow variation to SBLi program in future years) 8. Immuno- logical genetics Veterinary clinic: complement protein deficiencies in dog Genetic testing, PCR, mutation Veterinary pathology, immunology * Not implemented in 2008 (scenario reserved to allow variation to SBLi program in future years) 218 Australasian Journal of Educational Technology, 2012, 28(2) Evaluation: Engagement and perceived learning outcomes Student perceptions of engagement and learning outcomes were assessed through analysis of login data (access/completion and duration of use data) and a student feedback survey. Access records were analysed to determine the time spent on each SBL per student, and the number and timing of visits (note that scenarios 4a and 4b had independent login data). Login records of all students enrolled in the class were available. A small number of students (<10% of records) showed excessive duration of activity on a single occasion (e.g. one student’s record for a single SBLi problem was more than 23 hours). Data from these sessions were excluded from further analysis. The student feedback questionnaire (Table 2) was comprised of twenty questions relating to: (a) engagement with and/or enjoyment of the learning scenarios; (b) implementation strategies; and (c) perceived impact on learning. The questionnaire was completed during the final lecture session of the course and included 20 questions with a minimum of three questions addressing each of the core themes (engagement or enjoyment, implementation preferences and learning outcomes). Responses graded along a five-point Likert scale (1 = strongly agree; 5 = strongly disagree). Quantification of learning outcomes The overall contribution to learning outcomes was evaluated through integrated analysis of SBLi login data and quantitative measures of student performance in assessment tasks. To control for bias due to individual ability, responses to questions on the mid- semester examination were categorised according to whether or not the topics were supported by SBLi tutorials, creating a dataset where each student served as their own control in a comparison of marks for SBL-supported and unsupported questions. To verify the utility of this approach, login data for each individual was also compared with their second semester GPA from the previous year (allowing improvement or decline in performance to be assessed relative to prior effort/engagement). Continuity of course content across years (prior to and following implementation of the SBLi problems), allowed direct comparison of performance in two key assessment tasks: 1. A group task requiring interpretation of a genetic test and production of a short vet-client interaction video implemented with (2008) and without (2007) a supporting SBLi-based tutorial that allowed students to practice interpretation of genetic tests and vet-client interactions (Appendix 1). 2. An exam question designed to test student application of phylogenetic concepts to a novel case (Appendix 1), after phylogenetic skills were taught via exposure to an SBLi problem with practised transferance (2008) or via large, non-SBLi supported tutorials (2007). Seddon, McDonald and Schmidt 219 Table 2: Student responses to questionnaire Values shown as percentages of all responses. SA, strongly agree; A, agree; N, neither agree nor disagree; D, disagree; SD, strongly disagree. No. Question SA A N D SD A+SA D +SD 1 The SBLi problems helped me to consolidate my understanding of course content. 15 48 19 15 3 63 18 2 The SBLi problems helped me to see connections between lecture content and veterinary practice. 18 68 10 2 3 85 5 3 I enjoyed the SBLi problems because they had a ‘real life’ component. 13 44 24 13 6 56 19 4 The SBLi problems were used in appropriate ways within this course. 6 58 26 8 2 65 10 5 SBLi problems were most helpful when used before associated tutorials. 23 42 19 16 0 65 16 6 SBLi problems were most helpful when used within tutorials. 18 41 16 25 0 59 25 7 SBLi problems were most helpful when used independently of tutorials. 2 18 35 39 6 19 45 8 The SBLi problems required greater proficiency with computing skills than I possessed prior to this course. 0 3 3 55 39 3 94 9 More information about how to use the SBLi problems should have been provided before we undertook them. 3 23 19 40 15 26 55 10 Connections between material used in the SBLi problems and material presented in other BVSc courses were clear. 5 40 32 21 2 45 23 11 My learning was enhanced by the connections between material used in the SBLi problems and material presented in other BVSc courses. 8 31 39 18 5 39 23 12 Inclusion of non-genetic data in the SBLi problems distracted from the core content. 10 13 23 50 5 23 55 13 When doing the SBLi problems, I tried to relate what I learnt to other subjects. 0 44 23 29 5 44 34 14 As a result of a SBLi problem, I spent some time find- ing out more about the topic or other genetics topics. 0 21 27 44 8 21 52 15 I would prefer to only do tasks that relate to the basic genetics topics. 11 34 19 31 5 45 35 16 The SBLi problems enhanced my learning in this course. 13 50 19 16 2 63 18 17 The SBLi problems enhanced my learning in other courses. 0 15 42 35 8 15 44 18 My problem solving skills were improved by the SBLi. 8 21 40 29 2 29 31 19 SBLi problems were over utilised in this course. 3 18 32 42 5 21 47 20 SBLi problems should be retained as part of this course. 27 45 16 8 3 73 11 Results Evaluation Sixty-two of the 83 (75%) students enrolled in the genetics course in semester 2 2008 completed questionnaires upon conclusion of the course (Table 2). Of these, 85% responded positively to linkage of genetic problems with professional practice (Q2) and 56% indicated that they enjoyed the ‘real life’ approach adopted in design of scenarios (Q3). 220 Australasian Journal of Educational Technology, 2012, 28(2) Responses to questions relating to implementation strategies indicated that 80% of students preferred problems to be undertaken before, or within, formal tutorial sessions. Login records (for all problems except the phylogenetics scenario) showed substantial variation in the number of SBLi problems accessed by each student (8% accessed one scenario; 20% accessed two; 19% accessed three; 21% accessed four; 13% accessed five; 11% accessed six). It should be noted however, that the methodology also did not allow for identification of students accessing SBLi problems together with other students during collaborative study sessions. Completion rates tended to be highest for problems that incorporated subsequent tutorial participation. This was particularly noticeable in the case of the first problem implemented (plasmid resistance), which was accessed by 87% of students. The lowest participation rate (39%) was recorded for a quantitative genetics problem completed independently of tutorial support. The amount of time spent on each problem varied substantially both among students and among scenarios. This is reflected in a mean completion time of 47 ± 45 minutes. Completion times were higher at the start of semester (Figure 1), with mean completion time for the first two scenarios at 61 (Plasmid resistance) and 54 minutes (A2 milk). The lowest mean time spent on a scenario was 36 min (Copper toxicosis). 0 10 20 30 40 50 60 70 80 90 100 1 2 3 4 5 6 7 8 9 10 Mid- term break 11 12 13 Study break Exam week Week in semester No. students 0 1 2 3 4 6 Average time on SBL (hr) A2 Milk Plasmid Resistance Quantitative Genetics Copper Toxicosis I Copper Toxicosis II Population Genetics Figure 1: Use of SBLi over the semester Access records were used to determine the number of students (bar graph, left axis) accessing ICT-supported SBLi each week of Semester 2, 2008. Access was recorded at the level of the individual scenario. The SBL Copper Toxicosis II had a supporting Seddon, McDonald and Schmidt 221 tutorial in week 9 or 10 and SBL Population Genetics had a supporting tutorial in week 13. The average duration of access (line graph, right axis) was determined after removal of outliers (sessions >5 hours of access recorded). Note that not all scenarios were available to students from the beginning of semester. For the two SBLi with follow up tutorials, the majority of students (60.7% and 65.1%) accessed the SBLi on the day prior to their scheduled tutorial. However, students tended to access scenarios further in advance of tutorials at the start of semester, with a total of 23.6% of students accessing a SBL three or more days prior to the tutorial in Week 3 compared with 3.6% in a Week 13. Perceived learning outcomes A clear majority (63%) of students indicated that their understanding of course content was consolidated via SBLi problems (Table 2, Q1) and that this enhanced the overall quality of their learning experiences (Q16). Despite efforts to explicitly integrate content from other veterinary science courses, only 45% of students indicated that these connections were clear (Q10). Less than 40% reported that connectivity of content enhanced learning (Q11) or general problem solving abilities (Q18), but 55% of students indicated that non-genetic content was not a distraction from core (genetic) content (Q12). Approximately half of the students indicated that their experiences with the SBLi problems helped them to make connections across subjects (Q13), but only 21% (Q14) made significant use of reference materials (e.g. PDF copies of original research articles) embedded within SBLi problems. Although 47% of students expressed a view that SBLi problems were over-utilised within the overall learning program (Q19), these students showed a clear preference for more traditional teaching strategies limited to core genetics concepts (Q15). Overall, 73% of respondents (Q20) indicated that ICT-supported problems should be retained. Although most students indicated that they possessed sufficient computer skills to complete SBLi problems with no additional training (Q8), only about half felt they received sufficient initial instruction (Q9). Quantification of learning outcomes Evaluation of mid-semester examination results indicated that performance on SBL supported questions (average grade 67 ± 16%) was significantly enhanced relative to non-SBL supported questions (average grade 58 ± 15%; paired t-test p<0.01). For the majority of individuals, performance on SBL-supported questions was higher than on non-SBL supported questions (Figure 2). The diagonal line in Figure 2 indicates the line of equality of grades for SBL and non- SBL supported questions. Overall performance in the course (course grade), however, was not related to either the number of SBLi problems accessed nor the cumulative time spent on the problems (Figure 3) and the use of SBLi problems was unrelated to prior GPA (Figure 4). 222 Australasian Journal of Educational Technology, 2012, 28(2) 0 20 40 60 80 100 0 20 40 60 80 100 % grade non-SBL questions % grade SBL questions Figure 2: Individual student performance on mid-semester exam questions on content supported and not supported by SBLi activities. 0 1 2 3 4 5 6 7 40 50 60 70 80 90 Course grade Number SBL attempted 0 2 4 7 9 12 14 Cumulative time on SBL Number of SBL Time on SBL Figure 3: Relationship of a student’s logged usage of SBLi problems with their final course mark (%) Seddon, McDonald and Schmidt 223 Measures of logged usage are the number of SBLi-supported problems undertaken by a student (squares, left axis, regression line solid, R2=0.058) and the cumulative time they were logged on to SBLi problems (Figure 3, triangles, right axis, regression line dashed, R2=0.009) 0 1 2 3 4 5 6 7 8 2 3 4 5 6 7 Prior year GPA Cumulative time on SBL (hours) Figure 4: Relationship of a student’s logged usage of SBLi problems relative to overall academic ability In Figure 4 the measure of logged usage is the cumulative time a student spent logged on to SBLi problems. Academic ability is indicated by a student’s GPA in the previous year of study. Note that two students were excluded because they did not have GPA scores calculated across identical courses as all other students. The maximum achievable GPA is 7. Regression line R2=0.016 Performance in the video assignment was significantly higher for students who undertook the two related SBLi problems (n=32, mean grade 12.1 ± 1.4 out of 15, versus n=37, mean grade 11.4 ± 1.2 out of 15; two-tailed t test p=0.02). There was also a significant increase in grades among the SBLi supported cohorts (2008) relative to the unsupported (2007) cohorts (grade out of 15, SBL supported range 6.8-14.3, mean 11.6 ± 1.4, n=83; SBL unsupported range 9-14, mean 10.6 ± 1.9, n=113, two-tailed t test, p<0.01). This is unlikely to be related to differences in ability among cohorts, as prior GPA was only weakly associated with assignment grade (2008 assignment grades versus GPA for semester 2 2007, r2=0.17). 224 Australasian Journal of Educational Technology, 2012, 28(2) The distribution of grades for the examination question on phylogenetics showed marked improvement from 2007 (mean grade out of five 1.8 ± 1.5) to 2008 (mean grade 3.3 ± 1.3, two-tailed t test p<0.01). Marker comments also indicated that the quality of answers improved, for example the use of the correct terminology such as ‘clade’. Discussion The teaching and learning benefits of rich, complex tasks embedded in authentic, real world contexts is widely appreciated by science educators and this study of the impact of ICT-supported SBL in a preclinical undergraduate program indicates a quantifiably positive effect on learning outcomes. However, it is intriguing to note that the relevance of ICT-supported SBLi in highlighting and reinforcing connectivity of content is not immediately evident to students. A key feature of this study was embedding content information into clinical scenarios as a means of modelling and facilitating the enhanced learning experiences required to develop students’ ability to review and selectively apply domain-specific knowledge in a wide range of situations. Conveying links between course content and professional practice is an essential step in encouraging deeper learning (Minasian- Batmanian et al., 2006). The receptivity of students to the introduction of ICT- supported SBL suggests an appreciation of attempts to integrate subject matter across the veterinary science program through use of authentic problems drawn from professional practice. However, the connection across disciplines was not obvious to many students. While students may be engaged with the activities, the nature of the SBLi and the need to integrate concepts across disciplinary boundaries may not have matched with their personal learning style (Könings et al., 2011). This explanation is supported by the comments from some students that they wanted to focus on only the genetics content. SBL or PBL-style teaching faces challenges in design or implementation strategies to engage students with differing learning styles (Ng et al., 2011). The assessment results show a measurable improvement in student performance as a result of SBL integration. Both examination and assignment results were linked to engagement with SBL, independent of individual ability. It is encouraging that student learning improved and supports the tenet that higher levels of student engagement are linked to increased academic outcomes. However, there are several potential biases to the interpretation of our data. There are obvious limitations to the use of login data for some scenarios where students were not in a controlled environment. A greater number of students returned questionnaires commenting on the SBLi problems than were recorded on the SBLi access logs, implying that it was not uncommon for some students to complete the tasks in small groups. The social nature of SBL is likely to encourage participation and learning and suggests that greater attention to group learning theories could further enrich the learning experiences of future cohorts. Furthermore, it is possible that the effect of SBL integration has been confounded with the impact of overall teaching time dedicated to coverage of specific topics and/or differences among student cohorts. Resolving these issues would require a longer-term study of multiple cohorts. Despite student engagement and indications of improved learning outcomes, fewer than half of the students believed that the integration of content across courses improved their learning or problem solving abilities. This perception by students that Seddon, McDonald and Schmidt 225 their learning was not enhanced has been noted previously. Students did not perceive any active learning strategies implemented in a lower division botany course to affect their learning, either positively or negatively, even though exam marks indicated otherwise (Goldberg & Ingram, 2011). A possible explanation given for such disparity is a lack of confidence by students in their ability, induced by the challenging nature of the activities. This is supported by students giving the lowest ranking to their own skills in a PBL-based course in food biotechnology (Ng et al., 2011). SBL-based activities may benefit from self-assessment tasks to enable students to monitor changes in their understanding of concepts (Appleton & Lawrenz, 2011). In addition to enhancing their confidence, this would assist teachers in ensuring that the SBL provide a sufficiently challenging environment, which is an essential component to engagement (Appleton & Lawrenz, 2011). Students expressed appreciation of the use of real life scenarios, yet usage of the SBL activities, particularly later in semester, was relatively low. Disparity between student attitudes and actions may reflect student concerns about workload demands and the growing trend for full time students to also carry substantial employment loads, with the consequent clash of commitments resulting in students spending less time than expected on study outside the classroom hours (Errey & Wood 2011). Truly effective use of SBL and/or PBL requires time to engage with the material, to develop and practise problem solving and knowledge recovery processes. Although student perceptions of workload do not necessarily correlate with class or study hours, and are strongly influenced by the teaching and learning environment (Kember & Leung, 2006), professional courses can involve substantial structured learning time and heavy assessment loads, and workload has been cited by veterinary science students as one of the most significant factors impeding their learning (Ruohoniemi & Lindblom-Ylanne, 2009). In this case, informal student comments suggest that the overall program workload impeded engagement with SBL tasks. Frustration with extensive background information may explain the fact that 45% of students restricted their activities within the ICT-supported SBL to questions relating to core genetic content. In a veterinary curriculum, planning and implementation of teaching strategies designed to facilitate the novice-expert transition may well be impeded if excessive workloads and/or time constraints pressure students into attitudes and practices more consistent with direct content acquisition models. Another explanation for the apparent disparity between student attitudes to the SBL and their use may be the student’s perceived need for the cross-disciplinary knowledge and integration skills. It has been shown that there can be an inconsistency in the scenarios that teachers and students think is real world, or of practical importance (Appleton & Lawrenz, 2011). Such inconsistency may result from students’ impressions of a career garnered from sources such as the media rather than direct experience and so scenarios based on the teacher’s real life experience may seem overly complex, or outside the student’s image of their future job. Gathering information on what the students see as relevant or increasing their knowledge of their future career may assist in the adoption of SBL learning activities. Course-specific assessment practices are, of course, a contributing factor. Further insights into the potential of ICT-supported SBL in a veterinary curriculum can be found in evidence that high and low performing students are equally likely to utilise online resources. Regardless of their overall course performance, students tended to invest greater time on scenarios implemented early in the course. This may 226 Australasian Journal of Educational Technology, 2012, 28(2) reflect a period of learning how to use the software, an effect of the novelty of the task, or a prohibitory increase in workload pressure as the semester progresses. The latter point might also be linked to students’ preference for SBL activities incorporated into traditional tutorial formats (either through discussion of results/concepts or complete embedding within tutorial timeslots). On a practical level, future manifestations of ICT-supported SBL should also take into consideration students’ desire for greater instruction on software operation in the first instance. Although some preliminary instruction was provided during an introductory lecture session, it might prove effective to allocate tutorial time to instructions relating to software operation as higher teacher-student ratios would provide an opportunity for students to have their individual ITC-related needs addressed. Revision of existing SBLi problems is also being undertaken to incorporate more detailed operational instructions and signposts (e.g. explicit direction to nested locations/links within scenarios). Overall, rationales for incorporation of PBL/SBL into science education programs tend to emphasise the significance of exposure to authentic, ‘real world’ problems in development of the rich and complex cognitive schemata characteristic of expertise (Chan, 2007), but implementation of ICT-supported SBL into a traditional lecture/practical format in a pre-clinical education program indicates that achieving the full potential of the methodology requires ongoing commitment to revision of teaching and learning practices. Ideally, this would continue to involve a collaborative network of teaching and research-oriented staff. Acknowledgments This project was funded by a Strategic Teaching and Learning Grant from The University of Queensland. The authors would like to thank the staff of the School of Veterinary Science, The University of Queensland, for their contributions to the scenarios. Thanks also to the students of the genetics course for their participation and interest in trying new teaching strategies. References Appleton, J. J. & Lawrenz, F. (2011). Student and teacher perspectives across mathematics and science classrooms: the importance of engaging contexts. School Science and Mathematics, 111, 143-155. http://dx.doi.org/10.1111/j.1949-8594.2011.00072.x Baker, C. M., McDaniel, A. M., Pesut, D. J. & Fisher, M. L. (2007). Learning skills profiles of Master's students in nursing administration: Assessing the impact of problem based learning. Nursing Education Perspectives, 28, 190-195. http://www.highbeam.com/doc/1G1- 167508284.html Barak, M. & Shachar, A. (2008). Projects in technology education and fostering learning: The potential and its realization. Journal of Science Education and Technology, 17, 285-296. http://dx.doi.org/10.1007/s10956-008-9098-2 Barker, P. (2008). Re-evaluating a model of learning design. Innovations in Education and Teaching International, 45(2), 127-141. http://dx.doi.org/10.1080/14703290801950294 Barrow, L. H. (2006). A brief history of inquiry: From Dewey to standards. Journal of Science Teacher Education, 17, 265-278. http://dx.doi.org/10.1007/s10972-006-9008-5 Seddon, McDonald and Schmidt 227 Basu, S. J. (2008). Powerful learners and critical agents: The goals of five urban Caribbean youth in a conceptual physics classroom. Science Education, 92, 252-277. http://dx.doi.org/10.1002/sce.20241 Black, A. A. (2005). Spatial ability and earth science conceptual understanding. Journal of Geoscience Education, 53(4), 402-414. http://nagt.org/files/nagt/jge/abstracts/Black_v53p402.pdf Bruning, R., Schraw, G., Norby, M. & Ronning, R. (2004). Cognitive psychology and instruction, Fourth Edition. New Jersey: Pearson Education Inc. Chan, L. J. & Chan, K. C. (2007). Integrating inter-disciplinary experts for supporting problem- based learning. Innovations in Education and Teaching International, 44(2), 211-224. http://dx.doi.org/10.1080/14703290701241026 Clyman, J. C., Nazir, F., Tarolli, S., Black, E., Lombardi, R. Q. & Higgins, J. J. (2007). The impact of a genetics education program on physicians' knowledge and genetic counseling referral patterns. Medical Teacher, 29, e143-e150. http://dx.doi.org/10.1080/01421590701477373 Downes, T. (2002). Pre-service teaching training and teacher professional development in the use of ICTs in the teaching of mathematics and science in participating SEAMEO countries. Canberra: Australian Government, AEI-International Education Network. http://www.seameo.org/vl/library/dlwelcome/projects/ictmath03/useofict.htm Endler, L. C. & Bond, T. G. (2008). Changing science outcomes: Cognitive acceleration in a US setting. Research in Science Education, 38, 149-166. http://dx.doi.org/10.1007/s11165-007- 9042-0 Errey, R. & Wood, G. (2011). Lessons from a student engagement pilot study: Benefits for students and academics. Australian Universities' Review, 53(1), 21-34. [verified 23 Feb 2012; 8 MB] http://www.nteu.org.au/library/view/id/1298 Foote, L. C. & Fitzpatrick, K. A. (2004). Introduction to biological investigations: A first-year experience in experimental design and scientific communication. Journal of College Science Teaching, 34(3), 35-40. http://learningcenter.nsta.org/product_detail.aspx?id=10.2505/4/jcst04_034_03_35 Goldberg, N. A. & Ingram, K. W. (2011). Improving student engagement in a lower-division botany course. Journal of the Scholarship of Teaching and Learning, 11(2), 76-90. https://www.iupui.edu/~josotl/archive/vol_11/no_2/v11n2goldberg.pdf Gordon, J. (2006). Seven revelations about e-learning. Training, 43, 28-32. http://www.trainingmag.com/article/seven-revelations-about-e-learning Hager, P., Gonczi, A. & Oliver, L. (1990). Competency-based approaches to professional education. Paper presented at the Australian Association for Research in Education, Sydney. http://www.aare.edu.au/90pap/hager90319.txt Haigh, M. (2007). Can investigative practical work in high school biology foster creativity? Research in Science Education, 37, 123-140. http://dx.doi.org/10.1007/s11165-006-9018-5 Harlow, K. C. & Sportsman, S. (2007). An economic analysis of patient simulators for clinical training in nursing education. Nursing Economics, 25, 24-31. http://www.highbeam.com/doc/1G1-160332318.html Hem-Stokroos, H. H., Daelmans, H. E. M., Vleuten, C. P. M., Haarman, H. J. T. M. & Scherpbier, A. J. J. A. (2003). A qualitative study of constructive clinical learning experiences. Medical Teacher, 25, 120-126. http://dx.doi.org/10.1080/0142159031000092481 228 Australasian Journal of Educational Technology, 2012, 28(2) Jacobs, P. A. & Newstead, S. E. (2000). The nature and development of student motivation. British Journal of Educational Psychology, 70(2), 243-254. http://dx.doi.org/10.1348/000709900158119 Kelly, D. J. & Bell, E. J. (2000). Issues and practices in deciding competency: A discussion paper. Brisbane: Queensland Board of Senior Secondary School Studies. http://nla.gov.au/nla.arc- 25005 Kember, D. & Leung, D. Y. P. (2006). Characterising a teaching and learning environment conducive to making demands on students while not making their workload excessive. Studies in Higher Education, 31(2), 185-198. http://dx.doi.org/10.1080/03075070600572074 Kerka, S. (1998). Competency-based education and training. ERIC Myths and Realities. [viewed 3 Feb 2002 at http://ericacve.org/org/docs/cbetmr.htm; verified 23 Feb 2012 at http://www.calpro-online.org/eric/docgen.asp?tbl=mr&ID=65] Klein, C. J. (2006). Linking competency-based assessment to successful clinical practice. Journal of Nursing Education, 45(9), 379-385. http://www.slackjournals.com/article.aspx?rid=34913 Könings, K. D., Brand-Gruwel, S. & van Merrienboer, J. J. G. (2011). The match between students' lesson perceptions and preferences: Relations with student characteristics and the importance of motivation. Educational Research, 53(4), 439-457. http://dx.doi.org/10.1080/00131881.2011.625155 McMeniman, M. (1989). Motivation to learn. In P. Langford (Ed.), Educational psychology: An Australian perspective (pp. 215-237). Melbourne: Longman Cheshire. ISBN 9780582712812 McWilliam, E., Poronnik, P. & Taylor, P. G. (2008). Re-designing science pedagogy: Reversing the flight from science. Journal of Science Education and Technology, 17(3), 226-235. http://dx.doi.org/10.1007/s10956-008-9092-8 Meier, D. K., Reinhard, K. J., Carter, D. O. & Brooks, D. W. (2008). Simulations with elaborated worked example modeling: beneficial effects on schema acquisition. Journal of Science Education and Technology, 17(3), 262-273. http://dx.doi.org/10.1007/s10956-008-9096-4 Minasian-Batmanian, L. C., Lingard, J. & Prosser, M. (2006). Variation in student reflections on their conceptions of and approaches to learning biochemistry in a first-year health sciences' service subject. International Journal of Science Education, 28(15), 1887-1904. http://dx.doi.org/10.1080/09500690600621274 Ng, B. L. L., Yap, K. C. & Hoh, Y. K. (2011). Students' perception of interdisciplinary, problem- based learning in a food biotechnology course. Journal of Food Science Education, 10(1), 4-8. http://dx.doi.org/10.1111/j.1541-4329.2010.00111.x Nunan, D. (1988). The learner-centred curriculum. Cambridge University Press. ISBN: 9780521358439 Ringsted, C., Skaarup, A. M., Henriksen, A. H. & Davis, D. (2006). Person-task-context: A model for designing curriculum and in-training assessment in postgraduate education. Medical Teaching, 28(1), 70-76. http://dx.doi.org/10.1080/01421590500237721 Ruohoniemi, M. & Lindblom-Ylanne, S. (2009). Students' experiences concerning course workload and factors enhancing and impeding their learning - a useful resource for quality enhancement in teaching and curriculum planning. International Journal for Academic Development, 14(1), 69-81. http://dx.doi.org/10.1080/13601440802659494 Seddon, McDonald and Schmidt 229 Seddon, J. M. (2008). Vets and videos: Student learning from context-based assessment in a pre- clinical science course. Assessment and Evaluation in Higher Education, 33(5), 559-566. http://dx.doi.org/10.1080/02602930701699023 Shen, J. & Confrey, J. (2007). From conceptual change to transformative modeling: A case study of an elementary teacher in learning astronomy. Science Education, 91(6), 948-966. http://dx.doi.org/10.1002/sce.20224 Sigelman, C. K. (1999). Life-span human development. Third edition. Boston: Brooks/Cole Publishing Company. ISBN: 9780534354428 Skrok, K. (2007). Formation of pupils' attitudes and behaviours in chemistry teaching/ Formación de valores y actitudes de los estudiantes en educación química. Journal of Science Education, 8(2), 107-110. [abstract only] http://www.accefyn.org.co/rec/source/rec2819ref.htm Wilson, A. S., Goodall, J. E., Ambrosini, G., Carruthers, D. M., Chan, H., Ong, S. G., et al. (2006). Development of an interactive learning tool for teaching rheumatology - a simulated clinical case studies program. Rheumatology, 45(9), 1158-1161. http://dx.doi.org/10.1093/rheumatology/kel077 Appendix: Assessment items compared across years Video assignment The assignment task asked the students to prepare a short vet-client interaction video that presented an interpretation of genetic test results and offered animal breeding advice. The format was similar in both years, although the disease investigated varied between years. Marking criteria for both years were 1. Describe genetic causes of [the disease] in a clear and accurate manner. 2. Use problem-solving skills to interpret genetic test results. 3. Provide sound advice on past and future breeding program. 4. Clear presentation; use of appropriate language. Video Assignment 2007 Clients of your veterinary clinic are breeders of Australian Shepherd dogs. They have several breeding dogs from different parts of Australia and have just produced their second litter. The first litter, from Cloud and Rain-Shadow, sold well with all 8 pups sold by ten weeks of age. They bring in two of their dogs – Misty, a two-year old breeding female, and Fog, a six-week old male. Misty shows a white/blue colouration in her eye and has become reluctant to chase a ball. Fog keeps rubbing his left eye, and there is a yellow-coloured discharge from the eye. After examining Misty and Fog, you are concerned about the possibility of hereditary cataracts that occur in this breed and have ordered DNA testing of the heat shock factor 4 (HSF4) gene. The results have just come in: Dog Sex Age MicrosatelliteHSF4_5_85.24 HSF4 exon 9 positions 85286573 to 85286584 Misty female 2 years 280 280 GCC CCC CCC CAC Rain-shadow female 2 years 290 290 GCC CCC CCC CCA Lightning female 4 years 286 286 GCC CCC CCC CCA Cloud male 3 years 280 286 GCC CCC CCC CMM Fog male 6 weeks 292 292 GCC CCC CCC CCA (Note that in the sequence, M means both A and C) 230 Australasian Journal of Educational Technology, 2012, 28(2) You invite the clients in for a consultation. In pairs, prepare a short video (approximately 5 minutes) that shows a conversation between a vet and client. During the conversation, the vet will explain the test results and advise which dogs should be used in a breeding program. It should include a discussion on whether they can expect any problems with the first litter that has already been sold. It is important to ensure that the conversation is based on lay language, easily understandable by the client. Video Assignment 2008 You are working as a veterinarian in an urban practice. Your client, a breeder of Samoyed dogs, brings their dog, Fluffy, in to see you. The owner is worried because Fluffy doesn’t seem herself and they have noticed that she is drinking a lot of water and is always asking to go outside to urinate, even though the weather is starting to turn colder and she is usually keen to stay inside. One of the conditions on the top of your differential diagnosis list is diabetes mellitus. You take a blood sample and a urine sample to test for glucose levels. The initial results are: Test Patient’s value Normal values Blood glucose level 21.2 mmol/L <5 mmol/L Urine glucose level +++ -ve You have also heard that researchers at the vet schools at the University of Queensland and the University of Sydney are investigating links between MHC (Major Histocompatibility Complex) molecules and type 1 diabetes mellitus. You send some blood to the researchers to find out which alleles are present at the class II MHC genes. Here are Fluffy’s results: Animal ID DLA-DRB1 DLA-DQA1 DLA-DQB1 Fluffy DRB1*009, DRB1*031 DQA1*001, DQA*008 DQB1*002, DQB*008 The client is booked to come in again tomorrow morning. What will you say? Prepare a video of the vet and client where the vet explains the results and the client asks questions to ensure that the explanation is clear. Don’t forget to tell your client the diagnosis. As this is a genetics assignment, make sure you explain what test has been done and interpret the MHC results. Your client is also concerned if the condition is inherited and whether Fluffy can be used for breeding. Examination question on phylogenetics The results for an examination question on phylogenetics were compared between 2007 (non- SBLi supported) and 2008 (SBLi supported). This question followed a lecture and tutorial on phylogenetic analysis. However, in 2007, a computer-based tutorial involved the production and interpretation of phylogenetic trees from DNA sequence alignments, with students working alone, or in small groups, and tutors circulating to answer questions. In 2008 the activity was restructured as a SBLi-supported problem consisting of four interrelated case studies centred on interpretation of phylogenetic trees and was deployed during a single tutorial session with groups of about 12-14 students. Students worked on each case study alone or in smaller groups of two or three, with intervening group discussions led by a tutor that included the interpretation of additional phylogenetic trees with minimal background information. Students also worked in groups of four or five to read a short scientific paper and present a verbal summary and an interpretation of the phylogenetic tree contained in the paper. Two of the case studies in the revised, SBLi-supported tutorial used identical data and trees to those used in 2007. In each year, the end of semester examination contained an exam question that required students to interpret relationships on a novel phylogenetic tree and describe facets Seddon, McDonald and Schmidt 231 of phylogenetic methodology, two tasks that had been practised in the tutorials. The style of questions and the level of difficulty of the questions were comparable across years. Question 2007 Look at the following phylogenetic tree based on DNA sequences from mammoths and elephants. [Tree from Capelli et al. (2006) Molecular Phylogenetics and Evolution, 40, 620-627] a. What are the numbers above the branches and what do they tell you about the branch? b. Are mammoths more closely related to African or Asian elephants? Explain your answer. c. Does this tree give any evidence that Savannah African elephants are a different species to Forest African elephants? Explain your answer. Question 2008 The following phylogenetic tree shows the relationships among Crocodylian species and genera based on DNA sequence data. [Tree from Gatesy & Amato (2008) Molecular Phylogenetics and Evolution, 48,1232-1237] a. Are alligators the closest relative of crocodiles on this tree? Explain your answer. b. What is the role of the outgroup in a phylogenetic tree? Authors: Dr Jennifer M. Seddon (contact person), School of Veterinary Science, The University of Queensland, Gatton QLD 4343, Australia. Email: j.seddon1@uq.edu.au Dr Brenda McDonald, School of Veterinary Science, The University of Queensland, Gatton QLD 4343, Australia. Email: b.mcdonald@uq.edu.au Adele L. Schmidt, School of Veterinary Science, The University of Queensland, Gatton QLD 4343, Australia; and Holland Park State High School, PO Box 197, Holland Park QLD 4121, Australia. Current address: Science Education, School of Education and Professional Studies, Griffith University, Mt Gravatt Campus QLD 4122, Australia. Email: adeles104@hotmail.com Please cite as: Seddon, J. M., McDonald, B. & Schmidt, A. L. (2012). ICT-supported, scenario-based learning in preclinical veterinary science education: Quantifying learning outcomes and facilitating the novice-expert transition. Australasian Journal of Educational Technology, 28(2), 214-231. http://www.ascilite.org.au/ajet/ajet28/seddon.html