2011: Determination of core content throughout the curriculum Determination of core content throughout the curriculum Jared Danielson College of Veterinary Medicine, Iowa State University, Ames, IA This contribution is written in response to a request to assist the academic theriogenology community in their efforts to: 1. define a theriogenology core for veterinary schools and 2. determine whether or not students are actually learning what has been identified as core. This is part of a process that is often referred to as a needs assessment or front-end analysis. I will describe the aspects of a needs assessment that are most relevant to these two questions, discuss what members of the theriogenology community have already done to answer these questions, and then suggest some additional steps that you might consider taking, both as a community and as individuals. Determining the core curriculum with discrepancy analysis Determining the core is another way of saying “deciding what should be taught to everybody” in a particular area of study. There a number of ways to approach this decision. One common approach is to use a discrepancy analysis. As the term implies, a discrepancy analysis involves determining the discrepancy, or gap, between what students should know, and what they currently know. Therefore, at the heart of this particular curricular problem is the ability to measure first what veterinary graduates of AVMA-accredited colleges need to know about theriogenology upon graduation, and second, what veterinary graduates of AVMA-accredited colleges actually do know about theriogenology upon graduation. The discussion in this paper assumes that the reader is primarily interested in preparing veterinary students to become general practitioners. A similar approach could be employed for defining a core for preparing specialists, researchers, and so forth. What students should know A common approach to determining what students need to know (or be able to do) is often referred to as task analysis. A task analysis involves examining the knowledge and skills of a competent professional in the area for which the students are preparing. Since this discussion targets a general veterinary practitioner, in our context this approach requires two assumptions. First, we assume that there is a definable set of important theriogenology knowledge and skills relevant to the general practitioner, and not specific to a specialty. Second, we assume that most experienced general practitioners possess these knowledge and skills. These may not always be completely safe assumptions. For instance, if theriogenology knowledge and skills vary greatly among common species, there may be some skills that are, by their nature, specialized in some species but core in others. Second, it is possible for experienced general practitioners to lack knowledge or skills in a specific area, such as an area of emerging emphasis or transitioning technology. In such cases, it may not be safe to rely entirely on a task analysis with experienced practitioners to establish the gold standard of what graduates should know. Nonetheless, generally speaking, answering those two questions should provide the information needed to determine the required theriogenology content in the veterinary curriculum. Identifying the problem A discrepancy analysis defines the gap between what a practitioner knows and what the recent graduate knows. However, it can also be important to determine the practical implications of that discrepancy. For example, are there some common skills that are learned quickly and easily in the field, but are not practical to teach in an academic setting because of cost or other factors? Do some knowledge and skills have much greater implications for patient outcome, cost, client satisfaction, and so forth, than others? Being clear about the real or potential problems associated with specific knowledge gaps can be very useful for determining what constitutes the core. Clinical Theriogenology • Volume 3 Number 4 • December 2011493 What you already know Two recent papers represent a very good start at needs assessment for the theriogenology education community. In one, published in JAVMA, Root Kustritz, Chenoweth, and Tibary surveyed practicing veterinarians to determine which of a number of theriogenology procedures had the most value in practice and were performed most competently by new veterinary graduates.1 In the other, published in JVME, Root Kustritz, Tibary and Chenoweth surveyed North American and Caribbean colleges of veterinary medicine to determine how much emphasis was given to theriogenology instruction given a variety of factors, such as species area and instruction type (lecture, lab, clinical).2 Since they share the same authors, I will refer to these studies as “the JAVMA study” and “the JVME study” respectively in this paper. Based on these two studies you now know some interesting things that support a number of conclusions with implications for action or further study: 1. As a community, you have a reasonable list of theriogenology procedures that matter to practicing veterinarians, and graduates generally perform these procedures well. The procedures included in the JAVMA study were those most commonly taught in veterinary school and requested for continuing education. Respondents were given the opportunity to suggest other procedures that might be important, but it does not appear that such a list emerged. Furthermore, graduates’ competence at any given skill was highly correlated with that particular skill’s perceived value in practice.1 2. As a community, you have a reasonably good idea of what matters to practitioners. Some skills, such as diagnosis and management of dystocia, seem to be important for all respondents and across all species. Other procedures, such as artificial insemination, are less important to practitioners.1 This sort of information provides the framework for a “recommended curriculum”, defined as the knowledge and skills needed to perform the procedures identified by practicing veterinarians as at least having some importance (perhaps a “2.5” or “3” on the scale used in the study.) 3. As a community, you have a reasonably good idea of where improvement is needed. The JAVMA study identified some clear species-specific needs, such as improvement of instruction/learning with vaginal cytologic interpretation, brucellosis testing, and performance of cesarean sections in dogs, dystocia management in cats, and more instruction generally with llamas, alpacas, and exotic species.1 These might easily be identified as global deficiencies, deserving of attention from veterinary schools in general. 4. Required instruction in theriogenology varies remarkably across veterinary schools. In the JVME study, total required lecture hours ranged from 4 to 60 and total required laboratory hours ranged from 3 to 42. Required clinical exposure to theriogenology likewise varied considerably. When required hours were broken down by species, the standard deviation of hours across schools was often nearly as large as the mean.2 This suggests that as veterinary educators, we have very little idea of how many hours of theriogenology instruction might be “optimal” or even minimally required in veterinary medical education. Clearly, some graduates are becoming licensed with minimal formal exposure to theriogenology, at least as identified by the respondents to the JVME survey. 5. While the global trend of actual competence matched value in practice well, there may be important missing curriculum-specific information. Respondents to the JAVMA study were evenly divided between those who said they had a good veterinary education in theriogenology, and those who said they did not. Standard deviations for competency at graduation are relatively large, given the scale (about 20% of the full scale value), though no larger than the standard deviations for the rated value in practice of a given procedure.1 These results, while beneficial, beg the question of whether or not students feeling unprepared can be grouped systematically in some way. This idea will be discussed further in the next section. Things you can do as a community The studies cited above are a strong start to defining a theriogenology core. In fact, one defensible approach for moving forward might be to make sure that all programs are covering the most common procedures identified in the JAVMA study, pat yourselves on the back, and focus your attention elsewhere. However, if the goal is to establish and support a coherent and defensible core across colleges, Clinical Theriogenology • Volume 3 Number 4 • December 2011 494 there is more information that would prove helpful. The following are four suggestions regarding directions your community might consider in conducting future research and evaluation: Identify how the general trends are represented at the local level, if at all. This discussion is an extension of the issues raised in points 4 and 5 of the prior section, in which it was observed that the amount of time dedicated to theriogenology instruction varies greatly across institutions, and that while about half of surveyed veterinarians in the JAVMA study felt their theriogenology training was adequate, half felt that it was not.1 An important question for those wanting to define a theriogenology core is: how are these halves divided? Do most graduates of the college that requires 60 lecture hours of theriogenology feel well-prepared (and perform well) while most graduates of the college that requires 4 hours feel unprepared? There are other potential/reasonable break-downs. Perhaps many graduates with a large or mixed animal focus feel well prepared while most graduates with a small animal focus feel less well prepared. If your community can determine how perceived/measured skills/knowledge relate to specific curricular practices, you will have a stronger case to make when recommending a theriogenology core to colleges and other stakeholders. The kind of data produced by the two studies discussed above, if combined, could produce very insightful implications for curricular practice. Consider, for instance, Figure 1, which represents hypothetical data charting the relationship between credit hours dedicated to theriogenology instruction (lab and lecture combined) and competence (1-5; 1=completely lacking competence, 3 = minimally competent, and 5 = extremely competent). If this were a real dataset, it would provide a compelling case that if schools want their graduates to feel competent in their theriogenology ability, they need to dedicate in the neighborhood of 40+ hours to formal teaching. Furthermore, 60-75 hours would seem to be the sweet spot, being associated with good competence overall (about 4 on the 5 point scale), with additional time (beyond about 75 hours) not being associated with significantly greater levels of competence. You might also encounter implications for exceptional curricular practice, such as the hypothetical institution that achieved the third highest competence ratings with its graduates, while investing slightly less than 40 hours of formal teaching. Such a school might merit a visit and potential emulation from schools that were trying to improve either effectiveness or efficiency. Of course, this hypothetical dataset generally envisions a strong linear relationship between credit hours and competence. This represents a reasonable hypothesis, but it is by no means the only possible relationship between credit hours and competence. It might be that other factors (such as clinical exposure, related case load, species-specific exposure in medicine courses, and so forth) have a more powerful impact on competence than simple hours in classroom and laboratory. A study such as the theriogenology JAVMA study1 combined with the theriogenology JVME study2 could provide the information needed to answer these questions. The sampling procedure would need to ensure that veterinary respondents could be associated with a specific college and curriculum. Because the complexity of establishing this sort of sampling procedure and ensuring that it were followed adequately might render such an approach impractical, there is another approach that could serve the 0 20 40 60 80 100 120 1 2 3 4 5 C re d it  H o u rs  D e d ic at e d  t o  T h e ri o Competence Figure 1 ‐ Hypothetical case ‐ credit hours vs. competence Clinical Theriogenology • Volume 3 Number 4 • December 2011495 same purpose. That second approach might be to share measures across veterinary schools, measuring students’ perceived and actual competence during their clinical training, where reasonable response rates can still be ensured. This approach, “shared benchmarks” is discussed next. Shared benchmarks In the business world companies benefit by benchmarking their performance against competitors’ performance. In very simple terms, companies compare results from measures that they share in common, and then attempt to emulate other companies’ successes. Entertainment companies compare ratings or dollars at the box office, car companies compare horsepower, gas mileage, percent of cars still on the road after ten years, and so forth. Universities use benchmarks such as research dollars and numbers of publications cited. Unfortunately, in veterinary medical education, we have few shared benchmarks of educational success that are of any value, and certainly none that are specific enough to inform meaningful changes in theriogenology curricula. However, that does not mean that the theriogenology community could not create such measures and share data. A test could be designed and administered to theriogenology students at multiple schools at similar points of the curriculum, or identical clinical assessments could be implemented in clinical rotations. Such tests or assignments could be designed to emphasize knowledge and skills that are considered fundamental by veterinary practitioners. With adequate involvement from colleges, curricular differences could be associated with learning outcomes. This approach would avoid the challenges of sampling error inherent in survey research with employers or alumni, and would eliminate uncontrolled factors such as employers’ potentially faulty memory of competence, and skills/abilities learned on the job after graduation. Explicitly identify employer expectations at a skill-specific level In addition to associating global trends with curricular attributes as discussed in the suggestions above, more might also be learned from practicing veterinarians about their expected level of competence by procedure. The JAVMA study showed that graduates tend to feel most competent with theriogenology procedures that are considered more valuable in practice, and less competent with skills that are considered less valuable. This is a desirable trend, and is valuable information. In addition to this information, it may be worth asking employers what level of competence they expect from new graduates for each skill. For instance Hubbell, Saville and Moore asked veterinarians to identify whether the proficiency expected for a number of clinical equine skills were “able to perform unsupervised,” “able to perform with little supervision”, “able to perform with much supervision,” and “no proficiency expected.”3 This approach could provide additional information regarding which skills are “day 1” and which are “year 1”, and might also provide more species-specific trends. Identify any recent change The two studies cited in the “what you know” section are now five years old. That is not long in the curricular life-cycle, so the information contained in them is likely still to be current. However, the last six years have seen two important changes that might have affected veterinary curricula for AVMA- accredited schools. First, many schools are continuing to implement new outcomes assessment efforts, and to receive accreditation visits for the first time under the new outcomes assessment requirements. Second, many colleges are experiencing unprecedented budget cuts. Either of the factors could substantially affect curricula, either for better or worse. Therefore, follow-up studies to both the JAVMA and JVME studies seem warranted. Of the four suggestions above, it is my opinion that the theriogenology community in veterinary medical education would derive the most benefit from pursuing the first two. These would provide information that would be both specific and generalizable (e.g., representative of many institutions rather than just a few.) However, solutions such as those require active participation from many partnering colleges and/or departments; sometimes such participation can be challenging to obtain. Therefore, I also offer some suggestions that could be implemented on a smaller scale, and could benefit the individual Clinical Theriogenology • Volume 3 Number 4 • December 2011 496 instructor seeking to better identify his or her own “core.” The “community” and “individual” recommendations are by no means mutually exclusive. Things that the individual instructor/small groups of instructors can do Determine what students already know The discrepancy analysis approach as it was presented in the first section of this paper focuses entirely on the actual outcome of participating in a curriculum when compared with a desired outcome. Another important and often overlooked aspect of needs assessment is determining a number of student characteristics, including what they already know when they enter a given course or curriculum. This is not likely to be a course of action that can be taken by the entire theriogenology community and prove to be generalizable, since results are likely to vary considerably among schools. Nonetheless, sometimes the results of a pre-test can be surprising and enlightening strictly at the local level. For example, one colleague decided to implement a pre-test in his core required medicine course. To his surprise, on average students scored 50% on the test, indicating significantly greater knowledge than he had assumed. Similarly, another colleague implemented an examination intended to measure clinical competence to a variety of students throughout the curriculum. To his surprise, a few students knew enough to pass the test during their first semester of veterinary school. This point reflects the fact that we do not know what our students already know unless we test them. At least one respondent replied to the Root Kustritz et al JAVMA study that “My undergrad courses were nearly as good as my veterinary school training in regard to [reproductive] physiology.”1 If this comment happened to be representative of students in your courses at your institution, it might suggest that some material currently taught by clinical faculty in veterinary school might be shifted to earlier courses in the professional curriculum or may have already been treated adequately in prerequisite courses, allowing more to be covered in clinical theriogenology courses. Learning task analysis There are many ways to conduct a task analysis, (e.g., analyze what experts know that learners need to learn).4 The Root Kustritz et al JAVMA study1 represents one approach to broadly identifying procedures that are important to practitioners. This can be considered a task analysis, as it explores the tasks that are considered of greatest importance to practitioners. Another and complementary approach is referred to as a learning task analysis.5 This approach also can be done in a number of ways, but in all cases involves breaking a task into its component parts and determining how those parts are related, usually in terms of prerequisite relationships. This is an exercise that most instructors engage in at least informally, when first designing a course. Instructors might often think of this as “deciding what to teach first.” Detailed learning task analyses are often time-consuming and tedious to construct, so they are generally used only when they are considered necessary – for instance when designing stand-alone instructional software tutorials. Nonetheless, learning task analyses can also be a useful tool when limited time and/or resources or inadequate performance create the need for a better than average understanding of what is being taught in a curriculum. Figure 2 represents a hierarchical learning task analysis for learning neurolocalization of spinal cord lesions. Note that the ultimate targeted task for this analysis is the top one in the figure: “Given a case with a spinal cord lesion, describe and justify the process that would be required to localize a lesion within one of the following segments (C1-C5, C6-T2, T3-L3, L4- S2).” Directly prerequisite to that task are four others: “Describe/explain the principle of Central Recognition of Pain”, “Describe/explain the principle and testing of proprioception”, and so forth. Each of those knowledge/skills also has prerequisite knowledge and skills, and so forth, all the way Clinical Theriogenology • Volume 3 Number 4 • December 2011497     Figure 2 ‐ Hierarchical learning task analysis for neurolocalization of spinal cord lesions. Courtesy of K. Bolser  Describe/explain the principle of local  spinal reflexes. Note: Withdrawal and  patellar tendon tap are examples.  (Principle)  Define and describe  synapse, neuron, signal  transduction  (Declarative Knowledge)  Given a case with a spinal cord lesion, describe and justify the  process that would be required to localize a lesion within one of  the following segments (C1‐C5, C6‐T2, T3‐L3, L4‐S2).  Correctly classify and describe the role  of each type of neuron (Concept) Distinguish between primary  sensory and projection  neurons (Concept)  Distinguish between upper  motor and lower motor  neurons (Concept)  Describe the  characteristics and  roles of motor  neurons (Concept)  Describe the  characteristics and roles  of sensory neurons  (Concept)  Diagram a typical  thoracic spinal  nerve (Declarative  Knowledge)  List the peripheral nerves  that supply the thoracic  limb, and those that supply  the pelvic limb.  Include the  spinal cord segments from  which they arise.  (Declarative Knowledge)  Describe/explain  the principle of  voluntary motor  activity. (Principle)  Describe/explain the  principle and testing  of proprioception  (Principle)  Describe/explain  the principle of  Central  Recognition of  Pain (Principle)  Clinical Theriogenology • Volume 3 Number 4 • December 2011 498 down to defining and describing “synapse, neuron. . .” etc. In the case of Figure 2, a line is drawn horizontally across the figure above the bottom three boxes, indicating that the knowledge represented by those boxes has already been learned, and is not intended to be included in the instruction for which this analysis was conducted. Analyses such as the one demonstrated in Figure 2 have several practical purposes for those who wish to define an instructional core. First, they help to ensure that essential knowledge/skills are not forgotten, or that, if forgotten, they are identified if their absence proves to be problematic. This is particularly important when the core is primarily being established by experts in a field. While experts are very good at knowing what experts need to know, they are often not very good at remembering the learning path that got them to their expert level, or how the knowledge pieces all fit together. They have often either done something so many times that they have automated it, forgetting, essentially, how all the pieces fit together, or they have learned the knowledge and skills without ever formally articulating them.6 It is in this way that we become proficient speakers of a native language without defining parts of speech or diagramming sentences. Learning task analyses, then, are a tool for experts to use in identifying pesky omissions in content areas that consistently give students trouble. The second benefit of a learning task analysis, is that it can help the instructor to identify key knowledge/skills upon which many other knowledge or skills are dependent. Consider, for example, Figure 2. Note that there is one skill, “correctly classify and describe the role of each type of neuron,” that serves as the pathway between all of the second-level skills, and nearly all of the fourth, fifth, and sixth level skills. Assuming that this learning task analysis is correct, this skill is certainly one that a wise instructor would wish to pay attention to when teaching, reviewing, or testing students’ readiness to learn the applied task of neurolocalization. Less is more An analogy can be drawn between well-meaning instructors who design curricula and well- meaning parents who pack for a child’s overnight hiking trip. Some parents might pack in an effort to ensure that their child is prepared for every eventuality, including things such as a portable cot and inflatable sleeping pad for comfort, several changes of clothing in case they get wet, an extra blanket for warmth, extra snacks to share with friends, etc. All of these items are helpful, and their inclusion is reasonable. However, this “just-in-case” mentality can produce a backpack so heavy that the child can barely get it from the porch to the car, much less through an all-day hike. As a result, things are removed from the backpack for the purpose of lightening it. However, the decision regarding what stays and what goes is made by the child several hundred yards into the hike, and not by the parent who took such care in packing. Similarly, some instructors see it as their duty to force into the curriculum every valuable detail that they have acquired over years of practice or study. They feel that the greatest disservice they can do to their students is to omit some someday-useful fact. However, research suggests that “just-in-case” curricular packing can lead to learning of unrelated facts without the ability to make sense of those facts.7 Note that this is not an argument to water-down the curriculum or to omit factual learning. It is simply a reminder that wise instructors know that students will not learn everything, and that it is the responsibility of the instructor to carefully pack the proverbial curricular backpack with a load that can be carried. If we, as teachers, don’t choose what not to include in the curriculum, our students certainly will. Conclusion Available evidence suggests that, in general, the theriogenology community in veterinary medical education is providing adequate knowledge and skills to graduating veterinarians. However, several important questions remain unanswered, specifically, how are general trends represented at the local level, how do employers specifically identify the importance of given procedures, and have trends changed substantially in the past 5-6 years since the available work was done? The theriogenology community could benefit greatly from benchmarking using shared measures with advanced students across multiple programs. Individual programs could also benefit by determining what students already Clinical Theriogenology • Volume 3 Number 4 • December 2011499 know when they enter courses that emphasize theriogenology, conducting learning task analyses, and remembering that “less is more” when adding content to the curriculum. References 1. Root Kustritz MV, Chenoweth PJ, Tibary A: Efficacy of training in theriogenology as determined by a survey of veterinarians. J Am Vet Med Assoc 2006;229:514-521. 2. Kustritz MV, Tibary A, Chenoweth PJ: Availability of theriogenology training at North American and Caribbean veterinary colleges. J Vet Med Educ 2006;33:140-144. 3. Hubbell JA, Saville WJ, Moore RM: Frequency of activities and procedures performed in private equine practice and proficiency expected of new veterinary school graduates. J Am Vet Med Assoc 2008;232:42-46. 4. Jonassen D, Hannum WH, Tessmer M: Handbook of task analysis procedures. New York: Praeger Publishers; 1989. 5. Smith PL, Ragan TJ: Instructional analysis: analyzing the learning task. In: Instructional design. 3rd ed. Hoboken: John Wiley & Sons; 2005. 6. Bereiter C, Scardamalia M: Surpassing ourselves: an inquiry into the nature and implications of expertise. Chicago: Open Court Publishing Co; 1993. 7. Bransford JD, Brown AL, Cocking RR: The design of learning environments.In: How people learn: brain, mind, experience, and school. Washington: National Academy Press; 2000. Clinical Theriogenology • Volume 3 Number 4 • December 2011 500