Microsoft Word - ApplicantSelectiontoaRegionalarticle.docx Published by University of Minnesota Libraries Publishing Applicant Selection to a Regional Medical Training Program: Interviewer Predictions of Mission-Specific Outcomes Terry D. Stratton, Paula K. Arnett, Anthony D. Weaver, J. Bodie Stevens, Carol L. Elam DOI: https://doi.org/10.24926/jrmc.v3i1.2240 Journal of Regional Medical Campuses, Vol. 3, Issue 1 (2020) z.umn.edu/JRMC All work in JRMC is licensed under CC BY-NC Terry D. Stratton Paula K. Arnett Anthony D. Weaver J. Bodie Stevens Carol L. Elam All work in JRMC is licensed under CC BY-NC Volume 3, Issue 1 (2020) Journal of Regional Medical Campuses Original Reports Applicant Selection to a Regional Medical Training Program: Interviewer Predictions of Mission-Specific Outcomes Terry D. Stratton, Paula K. Arnett, Anthony D. Weaver, J. Bodie Stevens, Carol L. Elam Introduction Long recognized as a public health concern,1 the detrimental health effects of being rural2 remain exacerbated by the maldistribution of physicians and other health personnel away from these areas. In the United States, where expansion of the Affordable Care Act (ACA) has recently heightened the demand for primary care providers,3 rising student debt and comparatively lower physician salaries have further dissuaded interest in rural practice.4,5 These and other factors have motivated many medical schools to develop or expand specialized tracks and/or regional medical campuses (RMCs). During the past decade, the rise in RMCs – many with uniquely-dedicated rural service missions6 – has been a characteristic offshoot of many established training programs. Such examples include the University of Alabama’s Rural Medical Scholars Program (RMSP),7 the University of Illinois-Rockford’s Rural Medical Education (RMED) program,8 Michigan State’s Rural Physician Program (RPP),9 and the Rural Physician Associate Program (RPAP) at the University of Minnesota,10 to name but a few. Modeled in part after these existing programs, the Rural Physician Leadership Program (RPLP) was created in 2008 at the University of Kentucky College of Medicine (UK COM) to attract and train applicants interested in practicing medicine in rural areas. Students’ pre-clerkship years (M1 and M2) occur at the main UK COM campus in Lexington (KY), while their third and fourth years are completed at a regional campus located approximately one hour away in Morehead (KY) – where they receive clinical instruction and leadership training. Ten students are admitted annually, with preference given to applicants with the backgrounds, interests, and experiences that might encourage eventual medical practice in rural areas, in general, and Kentucky, in particular. RPLP Admission Process For more focused medical training programs, the task of admission is to assess: 1) professional suitability and preparedness and 2) the likelihood of achieving mission- specific outcomes. For this reason, we conduct semi- structured, face-to-face interviews with academically qualified applicants to compliment written responses to items contained on our secondary application form. Over 2 consecutive days, RPLP applicants complete interviews at both main and regional medical campuses. At each site, 2 trained interviewers (4 per applicant) with access to standardized applicant data (e.g., demographic characteristics, residency status, undergraduate college and grade point average (GPA), Medical College Admission Test (MCAT) scores, and relevant experiences) independently offer subjective, narrative assessments of applicants’ backgrounds and qualifications – as well as an overall rating of acceptability on a 7-point scale ranging from “unacceptable” (1) to “outstanding, clearly superior” (7). Using a scale from 0 (“no chance”) to 100 (“absolute certainty”), interviewers at both sites are also asked to estimate the likelihood that the RPLP applicant will ultimately practice rural medicine. Composite ratings for both measures consisted of averages across all 4 interviewers. All decisions to admit or reject RPLP applicants are made by the College’s standing admissions committee with input from a voting RPLP faculty member who summarizes the opinions of the RPLP interview board. Toward this end, we posed the following research questions: 1) What academic and socio-demographic characteristics are associated with interviewers’ ratings of applicants’ likelihood of rural practice in Kentucky? and 2) Do these characteristics coincide with where RPLP graduates, thus far, have chosen to practice medicine? Methods For the first research question, the study population consisted of 210 first-time RPLP applicants granted admission interviews from 2009-2016. For the 10 repeat applicants who interviewed in multiple years, initial interview data were used. U.S. counties followed U.S. Census Bureau designations as being “mostly urban,” “mostly rural,” or “completely rural” per the last (2010) decennial census.11 Appalachian counties were designated according to Appalachian Regional Commission (ARC) definitions.12 RPLP applicants from outside the U.S. (n = 8) were excluded, since comparable demographic origins could not be established. DOI: https://doi.org/10.24926/jrmc.ADDHERE Journal of Regional Medical Campuses, Vol. 3, Issue 1 Original Reports Along with interviewers’ ratings of overall acceptability and likelihood of rural practice, the following demographic and academic variables were examined: 1) gender, 2) race/ethnicity (white, non-white), 3) age [traditional (< 26), non-traditional (> 26)], 4) undergraduate major (biological science, other), 5) undergraduate institution (doctoral, Master’s, baccalaureate granting), 6) residency status (in- state, out-of-state), 7) county of origin (rural, non-rural), 8) county of origin (Appalachian, other), 9) total MCAT score and, 10) cumulative undergraduate GPA. The study population for the second research question consisted of 35 RPLP graduates completing the program from 2013-2016 (i.e., entering in 2009-2012). This time frame affords maximal opportunity to include some graduates who, having completed residency training, have gone on to establish medical practices. A critical value of p = < .05 was specified for inferential analyses. All analyses were conducted using SPSS13 (Version 25). This study protocol was approved as exempt by our local institutional review board. Results Sample Demographics As shown in Table 1, of the 163 first-time U.S. applicants having complete interviewer rating data, 127 (77.9%) were in- state and 36 (22.1%) were out-of-state residents. Among in- state residents, 47 (37.0%) originated from primarily urban counties and 80 (63.0%) from rural counties. Of the in-state applicants, 76 (59.8%) hailed from Appalachian counties. In total, 109 (66.9%) and 80 (49.1%) RPLP applicants originated from rural and Appalachian counties, respectively. Ninety- four (57.7%) applicants were male and 69 (42.3%) were female. Regarding undergraduate education, 74 (45.4%), 70 (42.9%), and 19 (11.7%) of applicants held degrees from doctoral, Master’s, and baccalaureate granting institutions, respectively. Applicants’ average total MCAT score was 27.9 (median = 27.0, SD = 3.1), and their cumulative undergraduate GPA was 3.68 (median = 3.76, SD = 0.30). Interviewers’ combined ratings of applicant acceptability averaged 5.3 (median = 5.4, SD = 1.01) and ranged from 2.5 to 7.0. For the likelihood of practicing in rural Kentucky, mean ratings were 76.6% (median = 81.3%, SD = 25.1%) and ranged from 10.0% to 99.8%. An assessment of composite and campus-specific interviewer reliabilities has been presented elsewhere.14 Table 1. Key Descriptive Variables of Applicants to a Rural Physician Leadership Program (RPLP), 2009-2016 ‡ Includes African American, Asian, Hispanic, and other races/ethnicities. ¥ Counties designated as “mostly rural” or “completely rural” by the U.S. Census Bureau. † Counties designated by the Appalachian Regional Commission (ARC). & Designations are based on the Carnegie Classification of Institutions of Higher Learning. € Includes majors in biology, biomedical science, biochemistry, chemistry, nutrition, physiology, psychobiology, and pre-med. Likelihood of Rural Practice A multivariate regression analysis was conducted to generate estimates reflecting the size and direction of various predictors on applicants’ predicted likelihood of eventual rural in-state practice. The dependent variable (likelihood of eventual rural in-state practice) was transformed to approximate a more normal distribution by squaring the original values. As shown in Table 2, the results of this analysis predicting interviewers’ assessments of RPLP applicants’ likelihood of practicing in rural Kentucky was statistically significant (F (9.50) = 10.42, p < .001) and comprised largely of socio- demographic factors: 1) residence (β = .345), 2) rural (β = .215), 3) Appalachian (β = .164); (4) race/ethnicity (β = .187), and 5) GPA (β = .149) – collectively explaining roughly 43% of the variance in the dependent variable. DOI: https://doi.org/10.24926/jrmc.ADDHERE Journal of Regional Medical Campuses, Vol. 3, Issue 1 Original Reports Table 2. Predictors of Estimated Likelihood of Rural In-State Practice of Rural Physician Leadership Program (RPLP) Applicants To more closely approximate a standard normal distribution, the dependent variable (“Likelihood of rural practice”) has been transformed by squaring the original values. Practice Location Of the 35 graduates admitted to the RPLP from 2009-2012, 2 were fulfilling military obligations, 5 remained in residency training, and 3 were completing advanced fellowship training at the time of this analysis. Subsequently, 25 had fully completed graduate training and had established medical practices. Of these 25, all but one (96.0%) were in-state residents. Eighteen (72.0%) completed primary care residencies (family medicine, general internal medicine, pediatrics, or obstetrics/gynecology) – 12 (48.0%) in the state of Kentucky. Equal proportions (44.0%) originated from “mostly urban” and “mostly rural” Kentucky counties; 2 (8.0%) hailed from “completely rural” counties. Fourteen (58.3%) were raised in an Appalachian area of Kentucky. The vast majority (92.0%) were white, and most (64.0%) were female. A majority (12/18, or 66.7%) of RPLP graduates practicing in Kentucky completed their residency training in-state; all 7 (100.0%) graduates practicing out-of-state completed residencies outside the state of Kentucky. (See Table 3). Among all 25 RPLP graduates, 8 (32.0%) were practicing in “mostly rural” and 17 (68.0%) in “mostly urban” counties. No RPLP graduate was practicing in a “completely rural” county. Of the 18 (72.0%) graduates practicing in Kentucky, the percentages were slightly more evenly distributed: 8 (44.4%) were in “mostly rural” and 10 (55.6%) were in “mostly urban” counties. Table 3. Socio-Demographic Characteristics of RPLP Graduates By State and County of Current Medical Practice (n = 25) ‡ Includes African American, Asian, Hispanic, and Other races/ethnicities. ¥ Counties designated as “mostly rural” or “completely rural” by the U.S. Census Bureau. † Counties designated by the Appalachian Regional Commission (ARC). Although the decision to practice medicine in Kentucky did not vary between males and females, gender was implicated in where graduates chose to establish their in-state practices: All 9 males (100.0%) were located in “mostly urban” counties, while females were evenly split between “mostly urban” (50.0%) and “mostly rural” (50.0%) counties. Neither specialty choice (primary vs. non-primary care) nor rural/Appalachian origin moderated this relationship. For the 20 practicing RPLP graduates for whom complete “likelihood” data were available, there was no significant difference in the median ratings between those currently practicing in Kentucky (n = 7) and those currently practicing outside the state (n = 13). Mean ratings varied by about 10 points (79.9 vs. 88.8, respectively); however, the small sample size tempers the rigor of these estimates. The magnitude of association was modest (rs = 0.34, p = 0.15). Discussion The likelihood of RPLP graduates’ practice in rural areas was determined largely by applicants’ socio-demographic background characteristics – namely, originating from a rural Kentucky county. These findings corroborate earlier research suggesting RPLP applicants’ rural values, as expressed in admission essays, are unrelated to interviewers’ assessments of overall acceptability.15 DOI: https://doi.org/10.24926/jrmc.ADDHERE Journal of Regional Medical Campuses, Vol. 3, Issue 1 Original Reports Since interviewers’ assessments of applicants’ overall acceptability did not differ by race/ethnicity,14 that non- Caucasian RPLP applicants were deemed less likely to establish medical practice in rural Kentucky may reflect the relative homogeneity of the state – especially in rural areas. However, the limited number of RPLP graduates in medical practice precludes any meaningful analysis. Successfully recruiting practicing physicians to rural settings is a task involving both tangible16 and intangible17,18 considerations. Pipeline programs, some starting as early as middle school, encourage and kindle early interest in medical careers; dedicated rural training programs must then select applicants “culturally-attuned” to these areas of medical need. A part of this “experiential integration with place”17 involves some level of community engagement or immersion19-21 that allows learners to nurture their “rural identities”.21 From prior research on rural medical practice, considerable attention has been paid to applicants’ related backgrounds, interests, and experiences.21-24 Indeed, of the 107 (38.8%) RPLP applicants not invited for interviews during our study time frame, most lacked meaningful rural experience and/or sufficient academic performance. Other programs which train providers specifically for practice in rural Appalachia also target recruitment efforts on those from the region or, interestingly, on those having military experience as medics.25 Similarly, elective “externships” like East Tennessee State University’s month-long Appalachian Preceptorship provide a combined clinical/classroom experience in rural culture and medical practice.26 Hence, our findings reflect the consideration of applicant backgrounds most amenable to addressing an area’s dominant healthcare needs. Some programs have experimented with giving added consideration to rural applicants,27 although not all have noted differences in the competitiveness or academic qualifications of rural versus non-rural applicants.28 Wright and Woloschuk, for example, found that despite the lower number of rural applicants relative to the population, there were no differences in ratings issued by admissions reviewers.29 In contrast, Australian researchers found that applicants from rural or remote regions had significantly lower entrance and interview scores and, when admitted, lower academic performance in medical school.30,31 Obviously, any admission considerations will depend, among other things, on the pool from which rural applicants are drawn. Predicting future events - be it academic performance, specialty choice, or eventual practice locale – is at best an inexact science. Still, since the overriding goal of the RPLP is to recruit and train physicians who will practice medicine in rural Kentucky, it makes sense that this prospect be explicitly addressed during the admission process. In addition, to the extent that RPLP graduates are expected to practice not just in rural areas, but in rural underserved areas, assessing applicants’ initial interest in primary care may also be warranted. Limitations These findings are limited by several factors. First, this study is based on a singular rural track training program at one institution. As a result, how widely these findings may generalize beyond this context is unknown. Second, our training protocol did not emphasize a standardized definition of “rural”. As a result, interviewers’ assessments of likelihood of rural practice may have been confounded by varying conceptions of what this entails.32 Finally, given the brief history of the RPLP, efforts to assess RPLP graduates’ practice location are necessarily preliminary – and these associated analyses are relatively underpowered based on the small cell sizes. Conclusions Dedicated rural medical tracks or programs have been shown to be effective strategies in producing primary care physicians for practice in rural, often underserved areas33 – especially when provided in settings (like RMCs, for example) that offer meaningful learning experiences outside the larger, urban environment.34 Key to the success of these efforts is the selection of candidates most qualified to meet programmatic goals. In the case of the RPLP, interviewers positively weighted in- state residence, rural, and Appalachian origins as indicators of eventual rural in-state practice. While outcome data are perhaps too limited to draw firm conclusions, these factors appear to exert a general influence on in-state practice – but not necessarily in rural areas. Indeed, virtually all 7 RPLP graduates who chose to practice medicine outside of Kentucky are doing so in non-rural areas. With 12 of 18 of those practicing in Kentucky having also completed in-state residencies (compared with 0/7 practicing out-of-state), the best predictor of in-state practice, thus far, is residency location. Unfortunately, this provides little predictive utility at the admission stage. Perhaps more useful in this capacity is the greater prevalence of female RPLP graduates practicing in rural areas (50% vs. 0%). Since all current rural medical practices are also in Kentucky – and no males report rural practices – further disentangling these effects is problematic. It is worth noting, however, that RPLP DOI: https://doi.org/10.24926/jrmc.ADDHERE Journal of Regional Medical Campuses, Vol. 3, Issue 1 Original Reports admissions interviewers did not differentially weight applicant sex in their assessments of eventual practice locale. In Kentucky, the RPLP was designed to meet this need by admitting applicants who rural practice preferences and training them in settings with supportive physician and community role models. These findings suggest that recruiting academically capable applicants with substantive rural backgrounds plays a guiding role in their choice of future practice locales – even if beyond the borders of Kentucky. As the RPLP matures, the continued follow-up of graduates will help determine the accuracy with which longer-term outcomes can be optimized during the very early stages of admission into the profession. While the initial results are preliminary, the number of RPLP program graduates practicing in rural areas is encouraging. As the number of RPLP graduates continues to grow, follow-ups are planned with those who have entered practice to identify potential factors influencing their decisions regarding rural medicine and practicing in Kentucky. Seeking such targeted input from our graduates will help the RPLP in refining not only its selection processes, but also the associated curriculum and socio-cultural learning experiences. Future research should expand our understanding of factors that contribute to the choice of rural practice and examine the role of rural experiences and unique curricula developed for RPLP students. References 1. Pusey WA. Medical education and medical service, I: The situation. Journal of the American Medical Association 1925; 84:281-285. DOI: 10.1001/jama.1925.26620300002013. 2. Matthews KA, Croft JB, Liu Y, et al. Health-Related Behaviors by Urban-Rural County Classification — United States, 2013. MMWR Surveillance Summary. 2017; 66 (No. SS-5):1–8. DOI: 10.15585/mmwr.ss6605a1. 3. Huang ES, Finegold K. Seven million Americans live in areas where demand for primary care may exceed supply by more than 10 percent. Health Affairs 2013; 32(3):614-621. DOI: 10.1377/hlthaff.2012.0913. 4. Palmeri M, Pipas C, Wadsworth E, Zubkoff M. Economic impact of a primary care career: A harsh reality for medical students and the nation. Academic Medicine 2010; 85(11):1692-1697. DOI: 10.1097/ACM.0b013e3181f5b754. 5. Wendling AL, Shipman SA, Jones K, Kovar-Gough I, Phillips J. Defining rural: The predictive value of medical school applicants’ rural characteristics on intent to practice in a rural community. Academic Medicine 2019. DOI: 10.1097/ACM.0000000000002924. [Epub ahead of print] 6. Norris TE, Coombs JB, House P, et al. Regional solutions to the physician workforce shortage: The WWAMI experience. Academic Medicine 2006; 81(10):857-862. DOI: 10.1097/01.ACM.0000238105.96684.2f. 7. Avery DM, Wheat JR, Leeper JD, et al. Admission factors predicting family medicine specialty choice: A literature review and exploratory study among students in a Rural Medical Scholars Program. Journal of Rural Health 2012; 28:128-136. DOI: 10.1111/j.1748-0361.2011.00382.x. 8. MacDowell M, Glasser M, Hunsaker J. A decade of rural physician workforce outcomes for the Rockford Rural Medical Education (RMED) Program, University of Illinois. Academic Medicine 2013; 88(12):1941- 1947. DOI: 10.1097/ACM.0000000000000031. 9. Wendling AL, Phillips J, Short W. Fahey C, Mavis B. Thirty years training rural physicians: Outcomes from the Michigan State University College of Human Medicine Rural Physician Program. Academic Medicine 2016; 91(1):113-119. DOI: 10.1097/ACM.0000000000000885. 10. Zink T, Center B, Finstad D, et al. Efforts to graduate more primary care physicians and physicians who will practice in rural areas: Outcomes from the University of Minnesota-Duluth and the Rural Physician Associate Program. Academic Medicine 2010; 85(4):599-604. DOI: 10.1097/ACM.0b013e3181d2b537. 11. County Classification Lookup Table. United States Census Bureau. Accessed 09/09/19 at: https://www.census.gov/geo/reference/urban- rural.html. 12. Counties in Appalachia. Appalachian Regional Commission (ARC). Accessed 09/09/19 at: https://www.arc.gov/counties. 13. IBM Corp. IBM SPSS for Windows, Version 25.0. Armonk, MY: IBM Corp. 14. Stratton TD, Kreiter CD, Elam CL. Main and regional campus assessments of applicants to a Rural Physician Leadership Program: A generalizability analysis. Journal of Regional Medical Campuses 2019; 2(2). DOI: 0.24926/jrmc.v2i1.1981. 15. Elam CL, Weaver AD, Whittler ET, et al. Discerning applicants’ interests in rural medicine: A textual analysis of admission essays. Medical Education Online 2015; March 19;27081. DOI: 10.3402/meo.v20.27081. 16. Daniels ZM, Vanleit BJ, Skipper BJ, Sanders ML, Rhyne RL. Factors in recruiting and retaining professionals for rural practice. Journal of Rural Health 2007; 23(1):62-71. DOI: 10.1111/j.1748- 0361.2006.00069.x. DOI: https://doi.org/10.24926/jrmc.ADDHERE Journal of Regional Medical Campuses, Vol. 3, Issue 1 Original Reports 17. Cutchin MP. Community and self: Concepts for rural physician integration and retention. Social Science and Medicine 1997; 44:1661-1674. DOI: 10.1016/s0277-9536(96)00275-4. 18. Hancock S, Steinbach A, Nesbitt TS, Adler SR, Auerswald CL. Why doctors choose small towns: A developmental model of rural physician recruitment and retention. Social Science and Medicine 2009; 69:1368-1376. 19. Eidson-Ton WS, Rainwater J, Hilty D, et al. Training medical students for rural, underserved areas: A rural medical education program in California. Journal of Health Care for the Poor and Underserved 2016; 27:1674-1688. DOI: 10.1353/hpu.2016.0155. 20. Greer T, Kost A, Evans DV, et al. The WWAMI targeted rural underserved track (TRUST) program: An innovative response to rural physician workforce shortages. Academic Medicine 2016; 91(1):65-69. DOI: 10.1097/ACM.0000000000000807. 21. Florence JA, Goodrow B, Wachs J, Grover S, Olive KE. Rural health professions education at East Tennessee State University: Survey of graduates from the first decade of the community partnership program. Journal of Rural Health 2007; 23(1):77-83. DOI: 10.1111/j.1748-0361.2006.00071.x. 22. Brokaw JJ, Mandzuk CA, Wade ME, et al. The influence of regional basic science campuses on medical students’ choice of specialty and practice location: A historical cohort study. BMC Medical Education (Online) 2009; Jun 6;9:29. DOI: 10.1186/1472-6920-9-29. 23. Sureshkumar P, Roberts C, Clark T, et al. Factors related to doctors’ choice of rural pathway in general practice specialty training. Australian Journal of Rural Health 2016;Jul 5. DOI: 10.1111/ajr.12311. 24. Stagg P, Greenhill J, Worley PS. A new model to understand the career choice and practice location decisions of medical graduates. Rural and Remote Health 2009; 9(4):1245. 25. Bushardt RL, Whitt FK, Gregory T. Training physician assistants for rural Appalachia: An academic partnership for interprofessional collaboration. North Carolina Medical Journal 2014; 75(1):53-55. 26. Lang F, Ferguson KP, Bennard B, Zahorik P, Sliger C. The Appalachian preceptorship: Over two decades of an integrated clinical-classroom experience of rural medicine and Appalachian culture. Academic Medicine 2005; 80(8):717-723. DOI: 10.1097/00001888-200508000-00002. 27. Basco WT, Gilbert GE, Blue AV. Determining the consequences for rural applicants when additional consideration is discontinued in a medical school admissions process. Academic Medicine 2002; 77(10 Suppl):S20-S22. DOI: 10.1097/00001888-200210001- 00007. 28. Longo DR, Gorman RJ, Ge B. Rural medical school applicants: Do their academic credentials and admission decisions differ from those of non-rural applicants? Journal of Rural Health 2005; 21(4):346- 350. DOI: 10.1111/j.1748-0361.2005.tb00105.x. 29. Wright B, Woloschuk W. Have rural background students been disadvantaged by the medical school admission process? Medical Education 2008; 42:476- 479. DOI: 10.1111/j.1365-2923.2007.02938.x. 30. Ray RA, Woolley T, Sen Gupta T. James Cook University’s rurally oriented medical school selection process: Quality graduates and positive workforce outcomes. Rural and Remote Health 2015; 15(4):3424. 31. Raghavan M, Martin BD, Burnett M, et al. Multiple mini-interview scores of medical school applicants with and without rural attributes. Rural and Remote Health 2013; 13(2):2362. 32. Owen JA, Conaway MR, Bailey BA, Hayden GF. Predicting rural practice using different definitions to classify medical school applicants as having a rural upbringing. Journal of Rural Health 2007; 23(2):133- 140. DOI: 10.1111/j.1748-0361.2007.00080.x. 33. Rabinowitz HK, Petterson S, Boulger JG, et al. Medical school rural programs: A comparison with international medical graduates in addressing state- level rural family physician and primary care supply. Academic Medicine 2012; 87(4):488-492. DOI: 10.1097/ACM.0b013e3182488b19. 34. Roseamelia C, Greenwald JL, Bush T, et al. A qualitative study of medical students in a rural track: Views on eventual rural practice. Family Medicine 2014; 46(4):259-266.