Microsoft Word - TheWriteStuffArticle.docx Published by University of Minnesota Libraries Publishing The WRITE Stuff: A Rural Longitudinal Integrated Clerkship Addresses Workforce Needs. Misbah Keen, MD, MBI, MPH; Danielle Gunder, Med; Toby Keys, MA, MPH; Douglas Schaad, PhD; David Evans, MD DOI: https://doi.org/10.24926/jrmc.v2i5.2103 Journal of Regional Medical Campuses, Vol. 2, Issue 5 (2019) z.umn.edu/JRMC All work in JRMC is licensed under CC BY-NC Misbah Keen MD, MBI, MPH; Professor and Executive Vice Chair, Department of Family Medicine, University of Washington, 1959 NE Pacific St, Box 356390, Seattle, Washington 98195 USA; mkeen@uw.edu Danielle Gunder MEd; Education Specialist, Office of Rural Programs, University of Washington School of Medicine, 1959 NE Pacific St, Box 356340, Seattle, Washington 98195 USA; dbienz1@uw.edu Toby Keys MA, MPH; Lecturer, Department of Family Medicine, University of Washington, 1959 NE Pacific St, Box 356390, Seattle, Washington 98195 USA; keyst@uw.edu Douglas Schaad PhD; Division Head of Medical Education and Evaluation, Biomedical Informatics and Medical Education, University of Washington, Box 358047, Seattle, Washington 98195 USA; schaad@uw.edu David Evans MD; Rosenblatt Family Endowed Professor in Rural Health, Program Director University of Washington Family Medicine Residency, 331 NE Thorton Place, Box 358732 Seattle, Washington 98125 USA; evansd9@uw.edu Corresponding author: Misbah Keen MD, MBI, MPH; Professor and Executive Vice Chair, Department of Family Medicine, University of Washington, 1959 NE Pacific St, Box 356390, Seattle, Washington 98195 USA; mkeen@uw.edu All work in JRMC is licensed under CC BY-NC Volume 2, Issue 5 (2019) Journal of Regional Medical Campuses Original Reports The WRITE Stuff: A Rural Longitudinal Integrated Clerkship Addresses Workforce Needs. Misbah Keen, MD, MBI, MPH; Danielle Gunder, Med; Toby Keys, MA, MPH; Douglas Schaad, PhD; David Evans, MD Abstract Introduction: The University of Washington School of Medicine has 6 campuses in the 5 state WWAMI (Washington, Wyoming, Alaska, Montana, and Idaho) region. The WRITE (WWAMI Rural Integrated Training Experience) program is a 22 to 24 week long rural longitudinal integrated clerkship experience offered to medical students in their clinical phase (third year) of training. This program seeks to meet the rural workforce needs of the WWAMI region by increasing the number of medical students going into primary care. Critics of LICs have expressed concern about overall quality control of the more remote educational experience and the lack of specialty specific teaching. The aim of this study was to compare medical school and PGY-1 performance of WRITE and non-WRITE students while determining how well each cohort is meeting the primary care regional workforce needs. Methods: The study group was all UWSOM students who matriculated from 2009 to 2013, advanced to graduation, and subsequently matched to a residency through the National Residency Match Program. WRITE and non-WRITE cohorts were compared for USMLE step 1 and 2 scores, MSPE (Medical Student Performance Evaluation) key word, and self and program director assessments in the first year of residency. The match results of the 2 cohorts were also compared to determine the proportions entering primary care residencies. Finally, for both cohorts the specialty choice at matriculation was compared with the match results. Descriptive statistics were used to test the comparisons. Results: The medical school performance of the WRITE and non-WRITE cohorts was equivalent in all metrics (USMLE Step 1 and 2, MSPE key word, self and program director assessment of performance in the first year of residency). WRITE students were significantly more likely to match into primary care (67.6% vs. 48.3%, p<0.001) overall and, in particular, Family Medicine as their specialty (40% vs. 14.3%, p<0.001). WRITE students were also more likely to match into the same specialty that they indicated on the UWSOM matriculation survey. For Family Medicine the loss of fidelity between matriculation and match among WRITE students was 3% (43.4 - 40.4) and among non-WRITE students, it was 6.3% (20.6 - 14.3). Conclusions: Performance outcomes of the WRITE program are equivalent to a traditional block curriculum. However, the WRITE cohort is significantly more likely to go into primary care fields, especially family medicine, and is more likely to stay with the declared specialty at matriculation. Medical schools that seek to increase the number of students going into primary care may benefit from adopting a similar model. INTRODUCTION The United States healthcare system faces a shortage of primary care physicians. This shortage is especially critical in rural communities. There are approximately 80 primary care physicians per 100,000 people in the United States; however, they are unequally distributed with 68 per 100,000 in rural areas and 84 per 100,000 in urban areas. 1 Washington, Wyoming, Alaska, Montana, and Idaho (WWAMI) comprise 27% of the US landmass and 3.4% of the population, Journal of Regional Medical Campuses, Vol. 2, Issue 5 Original Reports accounting for some of the most rural areas in the United States. These states are all part of the WWAMI program, the University of Washington’s regional medical education partnership. Development of a strong rural physician workforce in this region is of paramount importance to the University of Washington School of Medicine (UWSOM) and its WWAMI partners in this 5 decades old regional medical education program. 2 The clinical phase of medical education should optimally allow learners to participate in a social learning process through continuous and collective activities within a community. 3 Training in an environment that provides students an opportunity to observe and experience relationships among physicians, patients, and members of the health care team encourages learners to become a member of the healthcare community. 4 Longitudinal integrated clerkships (LICs) were created with continuity as a main pillar of structure. This style of clerkship allows students to stay in one place with a set of continuity patients and integration of multiple clinical specialties for extended periods of time, thus enhancing the educational experience in primary care. In contrast to the traditional block rotation curriculum, LICs seek to allow the student to master clinical competencies across multiple specialties concurrently, participate in continuity of care with a community of patients, and develop a continuous relationship with preceptors on site. Critics of medical education in a community-based setting have expressed concern about overall quality control of the more remote educational experience, including maintaining faculty development, faculty expertise, exposure to adequate patient numbers, and patient acuity. 5-8 There is also a perception that the lack of specialty specific teaching during a longitudinal clerkship detracts from knowledge acquisition. 9 Developed in 1996, the WWAMI Rural Integrated Training Experience (WRITE) is a rural LIC offered to medical students during their third year of medical school at UWSOM. The aim is to help meet the need for rural primary care physicians in the WWAMI region. Administered by the Office of Rural Programs with the support of the Regional Clinical Deans across 6 regional campuses, WRITE students participate in an integrated experience spanning 22 to 24 weeks within one assigned rural community. This rotation consists of an 18- week core curriculum, including 6 weeks of family medicine, 6 weeks of outpatient internal medicine, 3 weeks of outpatient pediatrics, 3 weeks of outpatient psychiatry, and either 6 weeks of obstetrics/gynecology or a 4-week advanced family medicine clerkship. Students spend at least one day per week in clinic with their primary care preceptor and consistent community-based patient panel. The remainder of the week is spent with preceptors within the community to achieve the required components of the other clinical disciplines. The remainder of clerkships, including inpatient internal medicine, pediatrics, and psychiatry, are done during the other half of the clinical year in tertiary care centers connected to the academic healthcare system. The surgery clerkship is not part of the WRITE curriculum and is completed anywhere in the WWAMI region during the 6- month “inpatient” clerkship block. WRITE sites are selected based on clinical services offered in the community and physician commitment to teaching. Quality of the clinical experience is maintained through a collaborative effort of involved departments, Regional Clinical Deans and the WRITE Director, and staff through regular meetings and biennial site visits. This study reports on the academic performance of WRITE students compared to non-WRITE students who entered the UWSOM over the 5-year period from 2009 to 2013. It further examines match rates into primary care specialties between WRITE versus non-WRITE students. The results of this study contribute to an ongoing vigorous discussion of the role of LICs in medical education, their academic integrity, and their ability to meet stated workforce objectives. METHODS Subjects This retrospective cohort study analyzed data from all UWSOM students who matriculated from 2009 to 2013, advanced to graduation, and subsequently matched to a residency through the National Residency Match Program (NRMP). The total number of students in the study group who completed the WRITE program was 99, referred to as WRITE students. There were 1016 UWSOM students meeting the selection criteria who did not complete WRITE, referred to as non-WRITE students. Setting At the time of the study, UWSOM was the only allopathic medical school in the 5 state WWAMI (Washington, Wyoming, Alaska, Montana, and Idaho) region. WWAMI has 6 regional campuses (Seattle, Spokane, Anchorage, Bozeman, Moscow, Laramie) in 5 states. UWSOM has 100+ decentralized clinical training sites in the region. UWSOM’s mission, in part, is to provide for the physician workforce needs in this rural region. Students complete the third year required clerkships either in traditional block format (majority of students) or in the longitudinal integrated clerkship model in the WRITE program. WRITE clinical sites are all in rural areas of the WWAMI region and administered, in part, by regional campus assistant deans. Students interested in WRITE participation apply to the program during their MS2 year and are interviewed by WRITE leaders for admission to the program. WRITE candidates are interviewed and selection is based on interest in a rural experience and good academic standing. All UWSOM students take USMLE Step 1 before their clerkship experience, then take Step 2 after their core clinical clerkships, including WRITE. Data sources Journal of Regional Medical Campuses, Vol. 2, Issue 5 Original Reports De-identified student data on academic parameters, NRMP match specialty, and specialty interests at matriculation were obtained from the University of Washington Division of Medical Education and Evaluation for the entering year (EY) classes of 2009 to EY 2013 (5-year period). The Division of Medical Education contacts graduates and their residency program directors 6 months into the PGY1 year to assess graduate performance as a resident compared to peers. Parallel questions are asked of the graduate and the residency program director. Written consent by the graduate is required to contact residency program directors. Surveys are sent by mail to the graduate in care of the residency program into which they matched. Response rates during the years studies very highly variable for the student survey as well as for the Directors survey. The variability is likely because our school requires student consent before these surveys are administered. Analysis Differences between WRITE and non-WRITE students with respect to academic performance and specialty selection were examined. Measures were selected to be considered as important milestones and evidence of overall performance in medical school. Parameters utilized included comparison of USMLE step 1 scores, USMLE step 2 scores, and Medical Student Performance Evaluation (MSPE) key words (good, very good, excellent, outstanding) as metrics of overall performance in medical school. The MSPE keyword is a summative of the medical student performance and shows where each student ranks in comparison to their peers. To assess preparedness for residency, PGY-1 self and program director assessments were compared. The specialty match results of WRITE and non-WRITE students were compared, and specialties were divided into primary care and non-primary care according to the AAMC’s definition of Family Medicine, Internal Medicine, Pediatrics, and Med/Peds. The match rates of individual primary care specialties were then compared. WRITE and non-WRITE student indicated specialty choice at matriculation and this was compared with actual specialty choice at match. The WRITE and non-WRITE comparisons were made as a cohort for all categories except for the USMLE Step 1 and 2 scores. This was necessary because of the large variation in year to year USMLE mean scores. Descriptive statistics were calculated for all comparisons. We calculated the mean, standard deviation, and P scores using the paired t- test. The University of Washington Human Subjects Division determined this study not to be Human Subject Research. RESULTS WRITE students performed at an equivalent level to students in the traditional curriculum (non-WRITE students). No statistically significant differences between WRITE and non- WRITE students were found in USMLE Step 1, USMLE Step 2, MSPE key word, self-assessed performance in PGY-1 year, or residency Program Director assessed performance during PGY-1 year (Tables 1-2, Figures 1-2). Table 1: Comparison of Medical Student Performance Evolution (MSPE) Keywords between WRITE and non-WRITE Students Table 2: Comparison of USMLE Step 1 and 2 performances between WRITE and non-WRITE Students by entering class cohorts Journal of Regional Medical Campuses, Vol. 2, Issue 5 Original Reports Figure 1: Comparison of Residency Program Director PGY-1 Assessment of Performance between WRITE and non-WRITE Students Figure 2: Comparison of PGY-1 Self-Assessment of Performance between WRITE and non-WRITE Students WRITE students were significantly more likely to choose primary care (67.6% vs 48.3%, p<0.001) overall and, in particular, Family Medicine as their specialty (40% vs 14.3%, p<0.001) (Figure 3). WRITE students were also more likely to match into the same specialty that they indicated on the survey done at matriculation. For Family Medicine the loss of fidelity between matriculation and match among WRITE students was 3% (43.4 - 40.4) and among non-WRITE students, it was 6.3% (20.6 - 14.3). For total primary care the loss of fidelity between matriculation and match among WRITE students was 3.7% (71.3 - 67.6) and among non-WRITE students, it was a gain of 2.9% (45.4 - 48.3). Figure 3: Comparison of Specialty Match Data between E09 - E13 WRITE and non-WRITE Students DISCUSSION In this study, it was found that students who participated in the WRITE program performed in medical school and in PGY- 1 at an equivalent level to that of the students following the traditional curriculum (non-WRITE students). MSPE key word distribution (key summary metric of medical school performance), USMLE Step 1 and Step 2 performance and the PGY-1 self and residency program director assessment were considered. While overall performance was the same, the WRITE students were much more likely to choose primary care overall and, in particular, Family Medicine as their specialty. Smaller positive trends were also present in Internal Medicine and Pediatrics specialty choice among the WRITE students but were not statistically significant. Furthermore, WRITE students who chose family medicine as a specialty choice at matriculation were more likely to match in family medicine than non-WRITE students who chose family medicine at matriculation. WWAMI is a 5-state area encompassing over one million square miles. Except for in Washington, about 50% of the WWAMI population live in a non-metro area. It has been estimated that by 2035 the U.S. needs 33,000 additional Primary Care Physicians to meet the needs of the growing and aging population. 10 This gap can only be closed by increasing the number of medical students matching into primary care specialties. 11 In 2010, Washington ranked number 17 for number of active physicians per 100,000 population with Wyoming at 47 th Alaska at 26 th , Montana at 32 nd and Idaho at 49 th . 12 As additional states within WWAMI expand Medicaid access under the Affordable Care Act, it is expected that workforce needs will be further aggravated. 13 In general, educational and admissions interventions that have been shown to be effective in attracting medical students to primary care and Family Medicine include: targeted admissions, supportive longitudinal curricula, and exposure to positive role models. The WRITE program is a Journal of Regional Medical Campuses, Vol. 2, Issue 5 Original Reports longitudinal program that exposes students to rural primary care clinicians as role models and has been previously described. 14 Previous studies of LICs have largely compared program structure. Other studies have compared academic performance outcomes, such as Step scores and NBME exams. This research demonstrates similar or better performance among longitudinal students compared to traditional block students. 8, 15-21 Studies of LICs that have examined workforce outcomes have previously shown graduates are more likely to practice primary care. 20, 22-24 Our findings are consistent with these previous studies of primary care-focused longitudinal clerkships and highlight the importance of such programs to address workforce goals. There are multiple strengths to this study. Data was examined over a 5-year period and directly compared our study group to an appropriate control group. Multiple commonly accepted variables used to evaluate student performance were also examined. This study has limitations as well. It was performed at one institution with a unique structure and a stated interest in meeting a rural workforce needs. Students self-select to participate in the WRITE program and may already be inclined to match into primary care and family medicine. Our workforce results may be influenced by disproportional exposure among WRITE students to primary care preceptors during clinical clerkships. The results show an association between WRITE participation and Family Medicine match, but the study is not designed to show causality. The Post Graduate Survey (PGY) response rate is variable and frequently low. However, this is the best available data to measure PGY performance. Conclusion This study confirms that the educational outcomes of the WRITE program are similar to a traditional block curriculum. While performance measures between WRITE and non- WRITE students were similar, WRITE students selected primary care specialties, especially Family Medicine, at higher rates than non-WRITE students, thus addressing the workforce needs of the rural WWAMI region. These results are important to medical educators and workforce policy makers and contribute to the growing body of literature supporting LICs as both a quality educational experience and a potential solution to specific workforce needs. References 1. Petterson SM, Phillips Jr RL, Bazemore AW, Koinis GT. Unequal distribution of the U.S. primary care workforce. American Family Physician [Internet]. 2013; 87(11). Available from: https://www.aafp.org/afp/2013/0601/od1.html. 2. Ramsey PG, Coombs JB, Hunt DD, Marshall SG, Wenrich MD. From concept to culture: the WWAMI program at the University of Washington School of Medicine. Acad Med. 2001;76(8):765-75. 3. Hirsh DA, Ogur B, Thibault GE, Cox M. "Continuity" as an organizing principle for clinical education reform. N Engl J Med. 2007;356(8):858-66. DOI: 10.1056/NEJMsb061660 4. Teherani A, Irby DM, Loeser H. Outcomes of different clerkship models: longitudinal integrated, hybrid, and block. Acad Med. 2013;88(1):35-43. DOI: 10.1097/ACM.0b013e318276ca9b 5. Morrison T, Brown J, Bryant M, Nestel D. Benefits and challenges of multi-level learner rural general practices--an interview study with learners, staff and patients. BMC Med Educ. 2014;14:234. DOI: 10.1186/1472-6920-14-234 6. AlHaqwi AI, Taha WS. Promoting excellence in teaching and learning in clinical education. Journal of Taibah University Medical Sciences. 2015;10(1):97- 101. 7. Murray E, Jolly B, Modell M. A comparison of the educational opportunities on junior medical attachments in general practice and in a teaching hospital: a questionnaire survey. Med Educ. 1999;33(3):170-6. 8. Couper I, Worley PS, Strasser R. Rural longitudinal integrated clerkships: lessons from two programs on different continents. Rural Remote Health. 2011;11(2):1665. 9. Hemmer P. Longitudinal, integrated clerkship education: is different better? Counterpoint. Acad Med. 2009;84(7):822. DOI: 10.1097/ACM.0b013e3181a843b1 10. Petterson SM, Liaw WR, Tran C, Bazemore AW. Estimating the residency expansion required to avoid projected primary care physician shortages by 2035. Ann Fam Med. 2015;13(2):107-14. DOI: 10.1370/afm.1760 11. Rosenblatt RA. Commentary: do medical schools have a responsibility to train physicians to meet the needs of the public? The case of persistent rural physician shortages. Academic medicine: journal of the Association of American Medical Colleges. 2010;85(4):572. DOI: 10.1097/ACM.0b013e3181d306b8 12. The Kaiser Family Foundation State Health Facts. Data Source: The Centers for Disease Control and Prevention (CDC), National Vital Statistics Reports (NVSR), Vol. 66, No. 1: Births: Final Data for 2015, January 5, 2017. 13. Allen SM, Ballweg RA, Cosgrove EM, Engle KA, Robinson LR, Rosenblatt RA, et al. Challenges and Journal of Regional Medical Campuses, Vol. 2, Issue 5 Original Reports opportunities in building a sustainable rural primary care workforce in alignment with the Affordable Care Act: the WWAMI program as a case study. Acad Med. 2013;88(12):1862-9. DOI: 10.1097/ACM.0000000000000008 14. Erickson JS, Norris TE, Schaad DC. Administering a Rural Longitudinal Integrated Clerkship Across 5 US States. The Open Medical Education Journal [Internet]. 2013; 6:1-11. 15. Zink T, Halaas GW, Finstad D, Brooks KD. The rural physician associate program: the value of immersion learning for third-year medical students. J Rural Health. 2008;24(4):353-9. DOI: 10.1111/j.1748- 0361.2008.00181.x 16. Myhre DL, Woloschuk W, Jackson W, McLaughlin K. Academic performance of longitudinal integrated clerkship versus rotation-based clerkship students: a matched-cohort study. Acad Med. 2014;89(2):292-5. DOI: 10.1097/ACM.0000000000000110 17. Woloschuk W, Myhre D, Jackson W, McLaughlin K, Wright B. Comparing the performance in family medicine residencies of graduates from longitudinal integrated clerkships and rotation-based clerkships. Acad Med. 2014;89(2):296-300. DOI: 10.1097/ACM.0000000000000113 18. Norris TE, Schaad DC, DeWitt D, Ogur B, Hunt DD, Consortium of LIC. Longitudinal integrated clerkships for medical students: an innovation adopted by medical schools in Australia, Canada, South Africa, and the United States. Acad Med. 2009;84(7):902-7. DOI: 10.1097/ACM.0b013e3181a85776 19. Hirsh D, Gaufberg E, Ogur B, Cohen P, Krupat E, Cox M, et al. Educational outcomes of the Harvard Medical School-Cambridge integrated clerkship: a way forward for medical education. Acad Med. 2012;87(5):643-50. DOI: 10.1097/ACM.0b013e31824d9821 20. Walters L, Greenhill J, Richards J, Ward H, Campbell N, Ash J, et al. Outcomes of longitudinal integrated clinical placements for students, clinicians and society. Med Educ. 2012;46(11):1028-41. DOI: 10.1111/j.1365-2923.2012.04331.x 21. Latessa R, Beaty N, Royal K, Colvin G, Pathman DE, Heck J. Academic outcomes of a community-based longitudinal integrated clerkships program. Med Teach. 2015;37(9):862-7. DOI: 10.3109/0142159X.2015.1009020 22. Halaas GW, Zink T, Finstad D, Bolin K, Center B. Recruitment and retention of rural physicians: outcomes from the rural physician associate program of Minnesota. J Rural Health. 2008;24(4):345-52. DOI: 10.1111/j.1748- 0361.2008.00180.x 23. Zink T, Center B, Finstad D, Boulger JG, Repesh LA, Westra R, et al. Efforts to graduate more primary care physicians and physicians who will practice in rural areas: examining outcomes from the university of Minnesota-Duluth and the rural physician associate program. Acad Med. 2010;85(4):599-604. DOI: 10.1097/ACM.0b013e3181d2b537 24. Hirsh D, Walters L, Poncelet AN. Better learning, better doctors, better delivery system: possibilities from a case study of longitudinal integrated clerkships. Med Teach. 2012;34(7):548-554. DOI: 10.3109/0142159X.2012.696745