Microsoft Word - ALongitudinalMeasurearticle.docx Published by University of Minnesota Libraries Publishing A longitudinal measure of medical student empathy at a regional campus: Are we different? Could this be a valuable evaluation method for curriculum change? William J. Crump, MD; Craig H. Ziegler, PhD.; R. Steve Fricker. MPA DOI: https://doi.org/10.24926/jrmc.v4i1.3475 Journal of Regional Medical Campuses, Vol. 4, Issue 1 (2021) z.umn.edu/JRMC All work in JRMC is licensed under CC BY-NC William J. Crump, M.D., Associate Dean of the University of Louisville Trover Campus at Baptist Health Madisonville, Madisonville, KY Craig H. Ziegler, Ph.D., Biostatistician with the University of Louisville School of Medicine Office of Graduate/Undergraduate Medical Education, Louisville, KY R. Steve Fricker. M.P.A., Director of Rural Health/Student Affairs of the University of Louisville Trover Campus at Baptist Health Madisonville, Madisonville, KY Corresponding author: William J. Crump, M.D. Associate Dean, University of Louisville Trover Campus at Baptist Health Madisonville 200 Clinic Drive, Third North Madisonville, KY 42071; p. 270.824.3515 f. 270.824.3590 e. bill.crump@bhsi.com All work in JRMC is licensed under CC BY-NC Volume 4, Issue 1 (2021) Journal of Regional Medical Campuses Original Reports A longitudinal measure of medical student empathy at a regional campus: Are we different? Could this be a valuable evaluation method for curriculum change? William J. Crump, MD; Craig H. Ziegler, PhD.; R. Steve Fricker. MPA Abstract Introduction Empathy is an important characteristic of the ideal physician. Various quantitative measures of empathy have shown a steep decline during the third year of medical school. Methods We had 4 classes of medical students at our regional rural campus complete the Jefferson Scale of Empathy after each of the first 3 years. We report longitudinal results of 30 students, individually matched, including an analysis by gender. Separately, we report the cross-sectional results for 39 of our students as they began medical school. We compare our student scores to other allopathic and osteopathic student scores from large urban campuses. The Baptist Health Madisonville IRB approved the protocol as exempt. Results As they begin medical school, our students have similar scores to those at large urban campuses (difference of 1.1 points, p=.421). After the M-2 year, our students had significantly higher scores than those at urban campuses (5.7 points, p=.002) and after the M-3 year, they show an even larger positive difference (9.0 points, p<.001). As in previous publications, females had higher overall mean scores at each measure, but with our students this was only significant in post-M-2 measures (8.9 points, p=.01). Discussion We conclude that something about our students’ experience during their M-3 year is associated with a smaller decline in empathy measures than reported previously. We propose that some of this difference could be due to a formal professional identity curriculum we implemented recently during the M-3 year. However, without a concurrent or historical control group, we cannot be certain. We offer the concept of measuring empathy before and after curricular change as another useful evaluation tool for medical educators. Introduction Empathy is widely regarded as a key characteristic of a good physician. Defining this characteristic is difficult. Questions remain as to whether it is more an emotional or cognitive process and how best to measure it. The most widely used and well- validated measure was produced by a group that views empathy as largely cognitive.1 Higher values by students on this empathy measure have been associated with positive clerkship faculty ratings of student clinical competence2 as well as better clinical outcomes in patients with diabetes in physicians with higher scores.3-4 Most studies have also shown that measures of empathy decline across the M-3 year.5 Recent studies have addressed the role of empathy in the development of medical trainees’ professional identity.6 Although preliminary, some show that reflection exercises such as composing narratives, organized study of art, film, music, and literature, and opportunities to practice mindfulness have a positive effect on empathy measures.7-10 The general trend, regardless of the instrument used, was that measured scores increased after the intervention, and some have shown that the change was sustained for at least 10 weeks.10 At our regional rural campus, we began a professional identity curriculum that we intended to mitigate the decrease in empathy seen in previous studies of medical students as Journal of Regional Medical Campuses, Vol. 4, Issue 1 Original Reports their education progressed.11-12 For our purposes we defined sympathy as “I feel your pain” and empathy as “I understand your suffering”. Whereas someone cannot always perceive another’s internal empathy, we defined compassion as empathy in action where someone could infer that empathy is the motivation for a physician’s behavior. We designed and implemented a series of sessions with our M-3 students intended to reach our goal. For this effort, we needed a reliable measure of our students’ and residents’ empathy. After a review of the literature, it appeared that the Jefferson Scale of Empathy (JSE) was the best instrument for our use.3,13 Most reports using the JSE were cross-sectional in nature, sometimes measuring one class of students and sometimes measuring multiple classes but all at a single point in time.14-15 Others have been longitudinal in nature, following groups of students matched to their individual results, measuring how the same repeated empathy measures changed across time in medical school years.1,16 In this study, we set out to measure the JSE in 4 classes of students across the first 3 years of medical school, matched for their individual results. Method The regional campus was established in a town of 20 000 in 1998 and hosts 6-8 students each year for the clinical years after they complete the first 2 preclinical years in an urban university environment 160 miles away.17-18 Applicants indicate interest in our campus at the time of secondary application, and those with previous rural experience are interviewed at both campuses. A regional campus selection committee makes recommendations to the single school admissions committee, and the students are assigned to a campus at the time of admission. In recent years, we have received about 200 applications for our 8 positions each year. About 55% of graduates go on to practice in rural areas, with almost 50% choosing family medicine and almost 85% choosing generalist careers. Beginning in the fall of 2015, the empathy survey was completed either just before the new academic year began, or just after it ended, resulting in an annual survey for each student. Each survey had the student’s name included for later matching, and these were placed by the students into an envelope confidentially, and participants were assured that a research assistant unknown to them would place an ID number and subsequently no one would be able to connect their responses to their name. About one student per year had to miss the required session, and a staff member then had each student who missed this conference complete the survey within 3 days, again with confidentiality preserved. For this sample of 30, only those students who had all 3 annual DOI: https://doi.org/10.24926/jrmc.v4i1.3475 surveys completed were included. The results are from the graduating classes of 2018-2021. Beginning also in the fall of 2015, a formal professional identity (PI) curriculum was implemented during the M-3 year after the students relocated to the regional campus, with one session per month of the recurring “Dean’s Hour” being dedicated to PI. We used the other twice monthly Dean’s Hour sessions for clinical reasoning case presentations and chart review of the student-directed free clinic patients. The dean himself facilitated the first year while developing the PI session content, with subsequent years done by the same campus MD faculty across the M-3 year. That first year is not included in this matched data set. The PI curriculum was very similar to the residency PI curriculum previously reported from this campus.12 This included an overview session on foundational concepts of professional identity vs professionalism, burnout, cynicism, and sympathy vs empathy examples. Subsequent monthly sessions focused on prevention and management of burnout, mindfulness techniques, and reflective writing and drawing. The latter included the career eulogy exercise previously reported from this campus,12,19, 20 as well as discussion facilitated by drawing a “comic” with stick figures and text balloons representing a “best” day and “worst” day of the student’s recent experience.21 Students also completed this “best and worst” reflection in the second half of the year using blank art paper and watercolors, a particular student favorite. We summarized demographic information using frequencies and percentages. We compared JSE scores between the regional campus and the Jefferson Medical College (JMC) in Philadelphia1 at post M1, post M2, and post M3 years using independent-sample t tests at each time. Because some classes had already begun when we started the project and to report a fully matched set of results, baseline measures were not included. However, we had baseline measures from subsequent classes that have not yet completed their M-3 year. We report that group of 39 students separately as a cross section baseline measure to address the issue of whether our students (who were largely from small towns and had chosen our rural campus) might have different JSE scores at entry into medical school from those reported from other medical schools. For comparison, we used the only 2 similar studies that reported baseline measures. These were Jefferson Medical College in Philadelphia1 and Boston University.14 To assess if differences existed among the 3 schools on the baseline JSE scores we performed a one-way analysis of variance. Because all previous reports had shown significant gender differences in JSE scores, we also assessed gender Journal of Regional Medical Campuses, Vol. 4, Issue 1 Original Reports differences for the regional campus at post M1, post M2, and post M3 using independent-sample t tests. We used IBM SPSS Statistics for Windows (version 26.0, 2019, IBM Corporation, Armonk, NY, 877-426-6006) to analyze the data. We created figures with the R package ggplot2.22 Statistical significance was set by convention at p <0.05. The Baptist Health Madisonville IRB approved the protocol as exempt. Results As shown in table 1, the majority of students in the matched set of the post M1, post M2, and post M3 scores were female (21/30 [70%]) and predominately white (28/30 [93%]). Eighty- three percent (25/30) of the students were from rural areas and 18/30 (60%) from what the authors considered very rural areas. Table 1: Demographics of students completing post M1 through post M3 Jefferson Scale of Empathy Freq (%) Gender Male 9 (30%) Female 21 (70%) Race White 28 (93%) Asian 2 (7%) Rurala Yes 25 (83%) No 5 (17%) Very Ruralb Yes 18 (60%) No 12 (40%) aRural was defined as a hometown population of <30,000 and a non-metro Rural Urban Continuum Code (RUCC).23 bVery rural was defined as a hometown population of <15,000 and a non-metro Rural Urban Continuum Code (RUCC).23 As shown in figure 1, the regional campus and JMC JSE scores do not differ at the post M1 measure, but significantly diverge at the post M2 and post M3 years. At post M2, the regional campus JSE average is 5.7 points higher than the JMC, t=3.15, df=149, p=0.002. By post M3, this mean difference increased to 9.0, t=3.95, df=149, p<0.001. DOI: https://doi.org/10.24926/jrmc.v4i1.3475 Figure 1: JSE means across medical school year comparing the regional campus to JMC.a aThere were significant differences between institutions at the post M2 (p=0.002) and the post M3 (p<0.001) measures. Error bars reflect 95% confidence intervals. For the regional campus, Table 2 shows significant gender differences at the post M2 year on the JSE, as females had higher scores than males (mean difference=8.9, t=2.75, df=28, p =0.010). At post M1 and post M3, the differences between genders are not significant, although females still have higher scores. DOI: https://doi.org/10.24926/jrmc.v4i1.3475 Journal of Regional Medical Campuses, Vol. 4, Issue 1 Original Reports Table 2: Gender comparisons of the Jefferson Empathy Scale for the post M1, post M2, and post M3 scores for the regional campus. post M1 post M2 post M3 N Mean (SD) Mean (SD) Mean (SD) Jefferson Scale of Empathy Males 9 113.6 (6.8) 115.2 (8.3) 112.9 (8.3) Females 21 119.7 (8.5) 124.1 (8.0) 119.4 (14.0) P-Value 0.066 0.010 0.204 Figure 2: JSE means across medical school year by gender at the regional campus.a aA significant difference at post M2 was found between genders, p=0.010. Post M1 and post M-2 differences were not significant. Error bars reflect 95% confidence intervals. Table 3 shows the Jefferson Scale of Empathy baseline mean score for the urban schools and the regional campus. There are no significant differences among the 3 schools. Table 3: Comparison of baseline Jefferson Scale of Empathy scores among the 3 schools Baseline Count Mean (SD) P – Value Jefferson Scale of Empathy JMCa 456 115.1 (10.0) 0.421 BUb 658 115.5 (1.8) regional campus 39 116.3 (8.05) aJefferson Medical College bBoston University Discussion Our results show that our students at this rural regional campus start medical school with remarkably similar JSE scores to those from 2 private northeastern metropolitan schools. In comparison to the only comparable published longitudinal study, our students beginning their M-3 year have significantly higher scores. The difference at the end of their M-3 year is even larger, while still showing the decrease that all previous studies, including cross sectional, have shown. Our students also showed the same gender differences reported previously with the JSE. We found the JSE to be an effective tool, requiring about 5 minutes to complete. Our standard deviations were also similar, but naturally larger because the sample size is smaller. We now have 5 years of experience with using it along with other surveys in our annual longitudinal database. Our results suggest that something is different about our students’ early M-3 through the late M-3 measured empathy from previous reports. It is tempting to attribute this difference to our PI curriculum, which we focused during the M-3 year. The actual increase in scores post M-2 was almost entirely from those who identified as female. Previous comparable studies actually showed a very small decrement from post M-1 to post M-2 of 0.9 points1 and 0.8 points.15 We do have a pathways component during the M-2 year that returns our students to their hometowns over the December holiday, and it is possible that this had a larger effect on female students.24 Alternatively, there could have been something during the M-2 year while our students were at the urban campus that affected women more than men. This sample was almost 70% female, and all studies using the JSE have shown higher scores in women. On average, our classes are about 50% female, so using a larger data set in future studies may provide more insight. Overall, since our students began medical school with JSE scores very similar to students from urban, more selective private schools, we conclude that something about our M-3 year does buffer the decline in the JSE score in both genders. Limitations and Strengths As with almost all educational interventions, a systemic selection bias is a concern. Our students really want to be at our campus, and so when they return, the buffering of the decline in JSE scores could be from generally being more satisfied rather than truly a measure of empathy. The scale itself has well established internal consistency reliability as measured by Cronbach’s alphas in the .75 range, and test-re- test reliability of .60.13 Socially desirable answers bias is also a possibility, and some studies have addressed this with the JSE as well.15 To separate any effects of our PI curriculum from the general effects of our campus experience, it would have been ideal if DOI: https://doi.org/10.24926/jrmc.v4i1.3475 Journal of Regional Medical Campuses, Vol. 4, Issue 1 Original Reports we randomized half of our students to a control group not receiving the PI curriculum exposure, something that is not feasible. In hindsight, even a historical control providing a comparison before the PI curriculum was begun would have been useful. Findings from our use of this PI curriculum with our family medicine residents where we did have a baseline measure, however, provide some support that the PI curriculum is effective. Those sessions were required and were on a different day of the week and week of the month each month. Therefore, the only residents not attending the sessions were those on rotations requiring them to be out of town that day or those on the inpatient service which rotates monthly, so no systemic scheduling issue could be involved. In this situation where resident choice was removed, there was a clear trend toward smaller decrements in JSE score associated with the number of sessions attended. Residents who only attended 3 or fewer sessions decreased by a mean of 6.83 points while residents who attended 5 or more decreased only by a mean 0.38 points. Residents who attended 4 or fewer sessions decreased an intermediate degree, by a mean of 4.50 points.12 The complete matching of individual results over time was a strength of that study, as it is with this one. There is also concern that our small, selected sample may not be generalizable to other campuses. However, a national study of almost 11,000 DO students recently found that end of year M-3 DO students showed a mean JSE score within 1 point of ours, again almost 7 points higher than previously reported allopathic school JSE results.15 Future studies We continue to collect our longitudinal data including the JSE and offer our process for consideration for use at other, larger and more diverse regional campuses. When we at regional campuses choose or are compelled to make curriculum changes, we are accustomed to completing careful program evaluations including traditional academic pre- and post-intervention quantitative measures and student satisfaction surveys. If entering classes share similar JSE scores, it may be possible to use the JSE as another evaluation measure of the effects of such curricular changes. Comparison measures prior to the changes are of course preferred. As we accumulate larger data sets, we will look more closely at gender differences. We are also studying other measures of compassion longitudinally and will share those results when available. References 1. Hojat M, Vergare MJ, Maxwell K, Brainard G, Herrine SK, Isenberg GA, Veloski J, Gonella JS. The devil is in the third year: A longitudinal study of erosion of empathy in medical school. Acad Med. 2009;84(9):1182-1191. doi: 10.1097/ACM.0b013e3181b17e55. Erratum in: Acad Med. 2009 Nov;84(11):1616. 2. Hojat M, Gonnela JS, Mangione, et al. Empathy in medical students as related to academic performance, clinical competence, and gender. Med Educ. 2002;36:522-527. doi: 10.1046/j.1365- 2923.2002.01234.x. 3. Hojat M, Louis DZ, Markham FW, Wender R, Rabinowitz C, Gonnela JS. Physicians’ empathy and clinical outcomes for diabetic patients. Acad Med. 2011;86(3):359-364. doi: 10.1097/ACM.0b013e3182086fe1. 4. Del Canale S, Louis DZ, Maio V, et al. The relationship between physician empathy and disease complications: An empirical study of primary care physicians and their diabetic patients in Parma, Italy. Acad Med. 2012;87(9):1243-1249. doi: 10.1097/ACM.0b013e3182628fbf. 5. Newton BW, Barber L, Clardy J, Cleveland E, O’Sullivan P. Is there hardening of the heart during medical school? Acad Med. 2008;83(3):244-9. doi: 10.1097/ACM.0b013e3181637837. 6. Cruess RL, Cruess SR, Boudreau JD, Snell L, Steinert Y. A schematic representation of the professional identity formation and socialization of medical students and residents: A guide for medical educators. Acad Med. 2015;90(6):718-25. doi: 10.1097/ACM.0000000000000700. 7. Misra-Hebert AD, Isaacson JH, Kohn M, Hojat M, Papp, KK, Calabrese L. Improving empathy of physicians through guided reflective writing. Int J Med Ed. 2012;3:71-77. doi: 10.5116/ijme.4f7e.e332. 8. Hojat M. Ten approaches for enhancing empathy in health and human services cultures. J Health Hum Serv Adm. 2009;31(4):412-50. https://www.jstor.org/stable/25790741. 9. Charon R. The patient-physician relationship. Narrative medicine: a model for empathy, reflection, profession, and trust. JAMA. 2001;286(15):1897- 1902. doi: 10.1001/jama.286.15.1897. 10. Hojat M, Axelrod D, Spandorfer J, Mangione S. Enhancing and sustaining empathy in medical students. Med Teach. 2013;35(12):996-1001. doi: 10.3109/0142159X.2013.802300. Epub 2013 Jun 11. 11. Crump, WJ. Professional identity curriculum at the University of Louisville Trover campus: reflection and meaning in medical education. J KY Acad Fam Physicians. 2017;Winter:88:18. 12. Crump WJ, Ziegler CH, Fricker RS. A residency professional identity curriculum and a longitudinal measure of empathy in a community-based program. J Reg Med Campuses. 2018;1(4). doi: 10.24926/jrmc.v1i4.1353 . DOI: https://doi.org/10.24926/jrmc.v4i1.3475 Journal of Regional Medical Campuses, Vol. 4, Issue 1 Original Reports 13. Hojat M, Gonnella JS. Eleven years of data on the Jefferson Scale of empathy-Medical student version (JSE-S): Proxy norm data and tentative cutoff scores. Med Princ Pract. 2015;24(4):344-350. doi: 10.1159/000381954. Epub 2015 Apr 28. 14. Chen D, Lew R, Hershman W, Orlander J. A cross- sectional measurement of medical student empathy. J Gen Intern Med. 2007;22(10):1434-1438. doi: 10.1007/s11606-007-0298-x. 15. Hojat M, Shannon SC, DeSantis J, Speicher MR, Bragan L, Calabrese LH. Does empathy decline in the clinical phase of medical education? A nationwide, multi-institutional, cross-sectional study of students at DO-granting medical schools. Acad Med. 2020;95(6):911-918. doi: 10.1097/ACM.0000000000003175. 16. Hojat M, Mangione S, Nasca TJ, et al. An empirical study of decline in empathy in medical school. Med Educ. 2004;38(9):934-941. doi: 10.1111/j.1365- 2929.2004.01911.x. 17. Crump WJ, Fricker RS, Ziegler C, Wiegman DL, Rowland ML. Rural Track Training Based at a Small Regional Campus: Equivalency of Training, Residency Choice, and Practice Location of Graduates. Acad Med. 2013;88(8): 112-1128. doi: 10.1097/ACM.0b013e31829a3df0. 18. Crump WJ, Fricker RS, Ziegler CH, Wiegman DL. Increasing the Rural Physician Workforce: A Potential Role for Small Rural Medical School Campuses. J Rural Health. 2016;32(3):254-259. doi: 10.1111/jrh.12156. Epub 2015 Oct 30. 19. Yu E, Wright SM. “Beginning with the end in mind”: Imagining personal retirement speeches to promote professionalism. Acad Med. 2015;90(6):790-793. doi: 10.1097/ACM.0000000000000690. 20. Crump WJ, Fricker RS, Crump-Rogers A. A Career Eulogy Reflective Exercise: A View into Early Professional Identity Formation. Marshall J Med. 2020;6(2). doi: 10.33470/2379-9536.1266. Available at: https://mds.marshall.edu/mjm/vol6/iss2/12 21. Green MJ. Comics and medicine: peering into the process of professional identity formation. Acad Med. 2015;90(6):774-779. doi: 10.1097/ACM.0000000000000703. 22. Wickham H. (2016). ggplot2: Elegant Graphics for Data Analysis. New York, N.Y.: Springer-Verlag; https://ggplot2.tidyverse.org. Accessed September 17, 2020. 23. United States Department of Agriculture, Economic Research Service. 2003 Rural Urban Continuum Codes. http://www.ers.usda.gov/data- products/rural-urban-continuum-codes.aspx last accessed September 17, 2020. 24. Crump WJ, Fricker RS. Keeping Rural Medical Students Connected to their Roots: A “Home for the Holidays” Immersion Experience. Marshall J Med. 2016;2(1):8. Doi: 10.18590/mjm.2016.vol2.iss1.8.