2013: EFFECT OF INCREASING STUDENT ENGAGEMENT ON MULTIPLE-CHOICE TEST SCORES IN THE THERIOGENOLOGY DIAGNOSTICS COURSE Effect of increasing student engagement on multiple-choice test scores in the theriogenology diagnostics course Margaret V. Root Kustritz University of Minnesota College of Veterinary Medicine,, St. Paul, MN Abstract Third year veterinary students in two consecutive years completed a multiple-choice examination as a pre-test before exposure to course content. Students in Group I completed a take-home assignment and then “graded” an assignment provided by the instructor. Students in Group II completed only the take-home assignment. Students in both groups completed an on-line multiple-choice examination as a post-test two weeks after completion of assignments. Students in both groups showed significant improvement from the pre-test to the post-test. Mean pre-test scores did not differ significantly between the two groups (p = 0.57). Mean post-test scores did vary significantly (p = 0.02) with group I scores higher than those for Group II, demonstrating value in increasing student engagement in the material. Keywords: Theriogenology education, active learning, Introduction Active learning is use of educational methodologies that provide students with opportunities to work with information presented to them, by reflection, evaluation, analysis, synthesis, and communication.1,2 Active learning often is equated with group work, which serves to help students not only to master content but also to learn skills in the discipline of interest including teamwork and conflict resolution.3 Other types of active learning involve individual students working either inside or outside of class and may include asking students to process or translate course information into unusual expressive modes, for example poems or songs; requiring students to argue on behalf of unfamiliar positions; or asking the student to articulate the clearest or muddiest point in a presentation.4-6 Another example of an active learning instructional strategy is asking students to find or identify mistakes or to critique another student’s work.4.5.7 This forces the student to evaluate their grasp of the material, which often is strengthened as they struggle to defend why they think another student is in error, and promotes critical thinking.5 This is an authentic teaching tool as it mirrors what many people are asked to do in their daily work.4 Peer-instruction can vary from the “convince your neighbor” scenario espoused by Eric Mazur, which requires students to discuss concepts within class and to reclassify that material within their own thinking so as to be able to explain it to others, to formalized systems where students with higher levels of demonstrated competency provide instruction and feedback to students earlier in a given program of study.8,9 Peer instruction also can include assessment of other students’ work and providing of constructive feedback. This study evaluated differing levels of student engagement in a short, required course in theriogenology to determine if increased student engagement was associated with increased test scores on a multiple-choice assessment. Materials and methods Theriogenology Diagnostics is a required course for third year veterinary students. It is a multi- species survey course and is the only required clinical theriogenology course in the core curriculum; other material is provided in species-specific courses available elsewhere in the tracked curriculum. Students participated in this study in two consecutive years. In both years, students were not given advance notice of the study to prevent their studying for the pre-test portion. In the first year (Group I), students completed a six-question multiple-choice assessment as a pre-test and then were presented with course information about canine reproduction for two hours, supported by notes available electronically or as hard copy. Students then had two weeks to complete a take-home assignment; students could work on the assignment together and could use any resources but each student had to hand in a completed assignment 63 Clinical Theriogenology • Volume 5 Number 1 • March 2013Clinical Theriogenology • Volume 5 Number 1 • March 2013 individually. The assignment was graded by the instructor. The instructor then compiled an example of that completed take-home assignment with some correct and some incorrect responses, with the incorrect responses those most commonly seen on student assignments, and all students were required to “grade” that second assignment, with the instructor assessing their effort as pass or fail. Two weeks after completing that second assignment, students logged onto the course website and took the same six- question multiple-choice assessment, this time as a post-test. In the subsequent year (Group II), students took the pre-test, were presented with information, and completed the take-home assignment as with the previous group, but did not receive or “grade” the second assignment. They took the post-test on-line two weeks after handing in the first assignment. Grades on the pre-test and post-test were compared within classes using the paired t-test. Grades were compared between Groups I and II for the pre-test and post-test using the unpaired (Students) t-test. Significance was set at p < 0.05. Results There were 96 participants in group I and 82 participants in group II. Participation was not 100% in either group because some students did not arrive in class until after presentation had begun, precluding their ability to take the pre-test as a true measure of their prior knowledge. These students completed the pre- and post-tests for their own learning but were not included in the study. The two classes were similar in composition. Participants in group I were from a class of 97 students, 20 men and 77 women. Mean age at admission was 25.4 years and mean grade point average (GPA) on required courses at admission was 3.5. Participants in group II were from a class of 100 students, 25 men and 75 women. Mean age at admission was 24.4 years and mean GPA on required courses at admission was 3.6. Data for pre- and post-testing scores in both groups were normally distributed by visual assessment. Mean (± standard deviation) pre-test score for students in group I was 1.59 ± 0.95. Mean post-test score was 3.78 ± 1.05. This difference was significant, at p < 0.0001. Mean pre-test score for students in group II was 1.51 ± 0.95. Mean post-test score was 3.41 ± 1.04. This difference was significant, at p < 0.0001. Mean pre-test scores did not differ significantly between the two groups (p = 0.57). Mean post- test scores did vary significantly (p = 0.02) with group I scores higher than those for Group II. Discussion Both groups demonstrated significant gains in knowledge as assessed by multiple-choice testing. Between the groups, increasing student engagement was associated with a greater gain in knowledge as assessed by multiple-choice testing. This is supported by a body of literature showing increased performance on assessments from students participating in active learning compared to passive learners.6,7,10-14 Other courses completed in that same semester were the same between the two groups. The author was the single faculty member responsible for small animal theriogenology training in both years. The groups were similar in gender distribution, mean values for academic achievement, and mean age. Splitting of a class in one year and having half of that class complete the work of Group I and the other half complete the work of Group II may have decreased unexplained variation between the groups. The author chose not to do this because of the automatic decrease in sample size with splitting of a class of pre-defined total size of 100, because there was greater likelihood of student anxiety altering scores, especially in Group I, whose members were required to do more work than those in Group II, and because it is the author’s experience that when classes are split, students will “try out” the other alternative. While the author has no problem with the increased learning that occurs during this experimentation, it does negate clear-cut differences between treatments in experimental groups. This has clearly occurred to this author with splitting of a class at her home institution for another study.15 One could argue that because the study was implemented over two successive years, the students in Group II heard about it from those in Group I, somehow giving them an advantage. This was disproven by the students in Group I performing better on the post-test than those in Group II. 64Clinical Theriogenology • Volume 4 Number 1 • March 2012Clinical Theriogenology • Volume 5 Number 1 • March 2013 The study design did not permit identification of which aspects of the assignments were most beneficial in increasing student test scores. Many other tools for assessment could have been employed, including other types of assignments or a closed-book examination. This study does not attempt to prove that the chosen method is the best method for student instruction in theriogenology, or to demonstrate that retention was improved with one method compared to another. Conclusion Students do not always enjoy being forced to evaluate each other’s work; a study evaluating use of an interactive web-based training tool for ethics instruction showed that students were less satisfied with the training technique but also were better able to demonstrate achievement of learning objectives.10 Use of an asynchronous instructional strategy that required peers to evaluate each other’s work provided students with the benefits of active learning and increased their understanding of the material. References 1. Fink LD: Creating significant learning experiences: an integrated approach to designing college courses. San Francisco: Jossey-Bass; 2003. 2. Michael J: Where’s the evidence that active learning works? Advan Physiol Educ 2006;30:159-167. 3. Machemer P, Crawford P: Student perceptions of active learning in a large cross-disciplinary classroom. Active Learn Higher Educ 2007;8:9-30. 4. Lang JM: The benefits of making it harder to learn. Chron Higher Educ 2012;58:A33-A34. 5. Faust JL, Paulson DR: Active learning in the college classroom. J Exc Coll Tchg 1998;9:3-24. 6. Prince M: Does active learning work? A review of the research. J Engin Educ 2004;93:223-231. 7. Reese-Durham N: Peer evaluation as an active learning technique. J Coll Sci Teach 2011;40:338-343. 8. Mazur E: Peer instruction. Upper Saddle River (NJ): Prentice Hall; 1997. 9. Baillie S, Shore H, Gill D, et al: Introducing peer-assisted learning into a veterinary curriculum: A trial with a simulator. J Vet Med Educ 2009;36:174-179. 10. Root Kustritz MV, Copes L: Use of a civil discourse website for ethics training. J Vet Med Educ 2013 (in press). 11. Sharkey L, Overmann J, Flash P: Evolution of a course in veterinary clinical pathology: the application of case-based writing assignments to focus on skill development and facilitation of learning. J Vet Med Educ 2007;34:423-430. 12. Levy DP, Merenstein B: Working with stories: an active learning approach to theories of deviance. Teach Sociol 2005;33:66-73. 13. Yoder JD, Hochevar CM: Encouraging active learning can improve students’ performance on examinations. J Instr Physchol 2005;32:91-95. 14. Mullen ET: Teaching an engaged analysis class through active learning. Prob Res Issues Math Undergrad Studies 2012;22:186-200. 15. Root Kustritz MV: Use of stories as a way to increase retention of clinical small animal theriogenology information. Clin Therio 2009;1:347-348. 65 Clinical Theriogenology • Volume 5 Number 1 • March 2013Clinical Theriogenology • Volume 5 Number 1 • March 2013