Microsoft Word - ImpactofVirtualInterviewing Article.docx Published by University of Minnesota Libraries Publishing Impact of Virtual Interviewing on Time and Financial Costs for NRMP© Applicants: Did Regional Campus Students Save More? Tiffany Schwasinger-Schmidt MD, PhD; Tessa Rohrberg MD; Anne Walling MB ChB; Kari Nilsen PhD DOI: https://doi.org/10.24926/jrmc.v4i4X.3925 Journal of Regional Medical Campuses, Vol. 4, Issue 4 (2021) z.umn.edu/JRMC All work in JRMC is licensed under CC BY-NC Tiffany Schwasinger-Schmidt MD, PhD; University of Kansas School of Medicine, Wichita, Department of Internal Medicine, 1010 North Kansas, Wichita, Kansas 67214 Tessa Rohrberg MD; University of Kansas School of Medicine, Wichita, Department of Family and Community Medicine, 1010 North Kansas, Wichita, Kansas 67214 Anne Walling MB ChB; University of Kansas School of Medicine, Wichita, Department of Family and Community Medicine, 1010 North Kansas, Wichita, Kansas 67214 Kari Nilsen PhD; University of Kansas School of Medicine, Wichita, Department of Family and Community Medicine, 1010 North Kansas, Wichita, Kansas 67214 Corresponding author: Kari Nilsen, PhD. University of Kansas School of Medicine-Wichita, Department of Family and Community Medicine, 1010 North Kansas, Wichita, KS 67214. Email. knilsen@kumc.edu All work in JRMC is licensed under CC BY-NC Volume 4, Issue 4 (2021) Journal of Regional Medical Campuses Original Reports Impact of Virtual Interviewing on Time and Financial Costs for NRMP© Applicants: Did Regional Campus Students Save More? Tiffany Schwasinger-Schmidt MD, PhD; Tessa Rohrberg MD; Anne Walling MB ChB; Kari Nilsen PhD Abstract Background and Objectives: The sudden change from in-person to remote interviews by the National Residency Matching Program© (NRMP©) in 2020 was expected to result in significant financial and time savings for applicants. This project aimed to compare savings before and after the 2020-2021 interviewing season reported by students graduating from our institution’s regional and main campuses. Methods: Data were collected over a six-year period at a Midwestern medical school. Each year, approximately 120 main campus and 75 regional campus students are surveyed regarding specialty choice, number of applications and interviews, time, and expenses to complete the NRMP. Chi-square and t-tests were used to determine statistical differences by campus and by specialty in savings during the 2020-21 interviewing season compared to the previous five years. Results: Data were provided by 957 students. The response rates were 81.5% (regional) and 82% (main campus). Compared to the previous five years, in 2021 main campus students saved $3,990 (79.9%) and regional campus students saved $2,789 (77.1%). The previous highly significant differences in expenses between campuses ($1,386 ± $243) dropped to $185 (p = 0.3). On both campuses, applicants to non-primary care specialties saved more than their classmates applying to primary care. The largest average saving was reported by non-primary care applicants on the main campus ($4,207) and the smallest by regional applicants to primary care ($2,328). Main campus applicants reported saving 13.1 and regional campus 15.4 days interviewing in 2021. The smallest average time saving was reported by main campus applicants to non-primary care (12.7 days) and the largest (16.2 days) by regional campus applicants to primary care. No significant changes occurred in number of applications, interviews, or Match outcomes in 2021 compared to previous years. Conclusion: Prior to 2021, students from our institution’s regional campus reported lower costs and similar interviewing time than their peers on the main campus. Cost and time were significantly decreased in 2021 for both regional and main campus students applying to any specialty and differences between campuses reduced to non- significant levels. The number of applications, completed interviews, and Match outcomes remained similar to previous years. Changes to the NRMP© incorporating remote interviewing may reduce applicant costs by 80% and provide up to two weeks of available curricular time in the senior year of medical school. INTRODUCTION The dramatic change in 2021 to remote interviewing for residency applications by the National Residency Matching Program© (NRMP©) in response to the COVID-19 pandemic was expected to significantly reduce applicant costs and time requirements.1-3 Such savings could be greater for regional campus students, especially those applying to non-primary Journal of Regional Medical Campuses, Vol. 4, Issue 4 Original Reports care specialties, who previously often travelled nationally to interview at competitive programs.4 Prior to the pandemic, seniors at US allopathic medical schools reported spending an average of $3,000 to $4,000 interviewing for residency positions.5-7 Expenses reported by individual applicants ranged from less than $100 to over $25,000 with lower expenses generally reported in primary care specialties (usually defined as internal medicine, family medicine, pediatrics, and medicine/pediatrics).5-7 Travel accounted for 60% to 70% of all expenses with other costs predominantly for food and accommodation.6,8 None of the over 20 studies of NRMP© applicant expenses have reported data for regional campus students,5-23 but studies have commented on higher costs for “out of town” applicants and increased travel burden for those from non-metropolitan areas.21,22 One large study attributed lower costs for students from the Northeastern region to the concentration of medical schools and residency programs in a single 500 mile area.8 To assess the impact of the NRMP© changes on costs and interviewing time for applicants based on both the regional and the main campus of our institution, we used data from an on-going annual survey of all fourth-year students participating in NRMP©. The main campus, with approximately 600 students, is part of an academic medical center located in a metropolitan area with a population of 2.2 million. The community-based regional campus has approximately 200 students and is situated in a metropolitan area with a population of approximately 650,000, located 200 miles southwest of the main campus. METHODS We examined data from a six-year study (2016-2021) of interviewing time and expenses for students at our institution that has been reported in previous publications.6,15,23 The participants were all fourth- year medical students who participated in the NRMP© to secure first-year residency positions. Each year, approximately 120 students from the main campus and 75 from the regional campus apply nationwide to residencies in the full spectrum of medical specialties. The research team consists of faculty members involved in both medical student and graduate medical education plus students from the third-and fourth-year classes during each survey year. The initial survey questionnaire was based on literature reviews and piloted on the regional campus in 2015.15 Modifications to the survey have been made each year based on feedback from students and faculty, developments in the literature, and changes in the NRMP© process. The current questionnaire contains 46-items addressing topics such as time and cost, number of applications and interviews, specialty choice, and sources of information utilized for residency program selection. The survey instrument is available for review in the Appendix. The questionnaire is distributed annually by e-mail once a week for four weeks in late February to early March, after residency rankings have been submitted and before announcement of NRMP© match results. Class leaders use social media reminders two to three times weekly and personal contacts to encourage classmates to complete the questionnaire. As an incentive, a donation proportional to the response rate is offered to the student graduation celebration fund. The School of Medicine Institutional Review Board has approved this study as “non-human subjects research” since inception. Descriptive analyses provided demographic information about participants in all six years of the study and survey responses regarding specialty choice (primary care versus non-primary care); number of residency program applications, interview offers, interviews completed, and programs ranked; expenses incurred; and time spent interviewing. T- tests and chi-square analyses were used to determine any statistical differences between applicants based on the regional campus and those on the main campus for all years, with special attention to changes between 2021 and previous years. Comparisons were made for all applicants and by primary care vs. non- primary care. Internal validity measures included review of findings by student leaders, educational committees, and discussion of results with students and faculty. RESULTS Participants (Table 1) Survey data were available from 957 students for an overall response rate of 81.8% (957/1170). The response rate was 81.5% (375/460) for regional campus students and 82.0% (582/710) for those from the main campus. Approximately half of respondents Journal of Regional Medical Campuses, Vol. 4, Issue 4 Original Reports were male on both campuses (49.3%). A significantly greater proportion of regional campus respondents applied to primary care programs [52.8% vs. 39.0%; χ2(1) = 8.1, p = 0.004, 95% CI 4.3% to 23.0%]. Changes in Numbers of NRMP© Applications, Interviews and Programs Ranked (Table 2) During 2016-20, students from the main campus consistently submitted significantly more applications [47.3 vs. 35.7; t(809) = 6.9, p < 0.001, 95% CI 8.9 to 16.0], received more interview offers [17.6 vs. 14.9; t(807) = 4.8, p < 0.001, 95% CI 1.7 to 4.1], and completed more interviews than those on the regional campus [12.2 vs. 10.3; t(807) = 5.9, p < 0.001, 95% CI 1.2 to 2.4]. Main campus students also ranked more programs (11.6 vs. 10.5; p = 0.1). In 2021, while students from the main campus again submitted more applications (47.2 vs. 38.1; p = 0.08), received more interview offers (13.9 vs. 12.3; p = 0.2), completed more interviews, (12.2 vs. 10.3; p = 0.08), and ranked more programs (11.4 vs. 9.9; p = 0.7) than those on the regional campus, these differences were not statistically significant. Changes in Interviewing Costs (Tables 2 and 3, Figure 1) For the years 2016-2020, regional campus students reported average costs $1,386 lower than colleagues on the main campus [$5,005 vs. $3,619; t(769) = 5.7, p < 0.0001, 95% CI $909 to $1,862]. They also reported approximately $70 less in cost per completed interview [$351 vs. $421; t(736) = 3.0, p = 0.003, 95% CI $25 to $116]. In 2021, costs dropped dramatically for all applicants. In 2021, regional students saved an average $2,789 [77.1% of pre-pandemic costs; t(350) = 6.1, p < 0.0001, 95% CI $1,887 to $3,692] and main campus students $3,990 [79.7%; t(542) = 10.2, p < 0.0001, 95% CI $3,225 to $4,755]. The difference between main and regional students’ total costs dropped from $1,386 to $185 [t(894) = 5.1, p < 0.0001, 95% CI $709 to $1,602] and the difference in cost per completed interview to approximately $5 ($83 vs. $78, p = 0.9). Highly significant savings were reported by students on both campuses applying to both primary care and non-primary care specialties. The greatest savings were reported by main campus applicants to non- primary care specialties [$4,207; 76.9%; t(323) = 7.6, p < 0.0001, 95% CI $3,119 to $5,296] but the greatest percentage saving was in regional campus applicants to primary care [$2,328; 85.1%; t(186) = 4.4, p < 0.0001, 95% CI $1,284 to $3,372]. The differences between campuses dropped by $687 [87.5%; t(487) = 2.1, p = 0.03, 95% CI $56 to $1,425] for non-primary care and $1,272 [81.1%; t(405) = 4.5, p < 0.0001, 95% CI $694 to $1,761] for primary care applicants. Changes in Interviewing Time (Tables 2 and 4, Figure 2) During the 2016-2020 period, the one-day difference in total average time between campuses was not statistically significant (29.2 vs. 28.1 days; p = 0.3) but the 0.4-day average time per completed interview was significantly greater for regional campus students [2.8 vs 2.4 days; t(752) = -3.5, p = 0.001, 95% CI -0.6 to -0.2]. In 2021, students reported significant time savings of 13.1 days on the main campus [t(553) = 7.9, p < 0.0001, 95% CI 9.9 to 16.4] and 15.4 days on the regional campus [t(358) = 6.0, p < 0.0001, 95% CI 10.3 to 20.5]. The greatest time savings were reported by regional applicants to primary care [16.2 days; 60.9%; t(189) = 4.5, p < 0.0001, 95% CI 9.1 to 23.3] and non- primary care specialties [15.3 days; 51.2%; t(167) = 4.1, p < 0.0001, 95% CI 7.9 to 22.7]. The time per completed interview was identical for main and regional campus applicants (1.2 days). DISCUSSION This study demonstrates that average estimated interviewing costs dropped by over 70% for all applicants in 2021. Those based on the main campus reported larger savings in both total amounts (nearly $4,000) and as a percentage of previous expenditures (nearly 80%). The cost differences between campuses dropped dramatically from over $1,300 to $185 and cost differences between campuses per completed interview almost disappeared. Despite these dramatic overall savings, applicants to non-primary care on the main campus spent almost double the amount of their classmates applying to primary care. This difference was even more pronounced on the regional campus where non-primary applicants reported expenses nearly triple those of applicants to primary care. Students on the main campus continued to report higher numbers of applications and interviews in 2021 but neither campus showed a surge in applications in response to the uncertainties of the pandemic year and the removal of travel requirements. Journal of Regional Medical Campuses, Vol. 4, Issue 4 Original Reports While modest compared to the total debt of graduating medical students, estimated at a median of $200,000,24 these savings are especially welcome for those with limited resources considering non- primary care specialties, and may contribute to improving diversity in some specialties.25-27 The time consumed by the residency interviewing process has received relatively little attention in the literature but is a major source of stress and impediment to leaning in the senior year of medical school.28-31 Our results indicate savings of up to 16 days by replacing in-person interviewing with virtual interviews. In addition to the actual time spent on interviews, student narratives commented on reductions in time and stress throughout the fourth year (and end of third year) in arranging interviews and scheduling visits to multiple locations. Students were also often able to obtain permission to video- interview during rotations and avoid the disruption and administrative implications of days away from coursework. The potential time savings in the senior year offer valuable curricular opportunities to better prepare students for the transition to residency and should be a major consideration in any redesign of the NRMP© system. The study has several limitations, principally that our findings are unique to one institution and one time period when many aspects of applicant interview process had to be improvised by each program and medical school. Each institution and regional campus have distinctive features, and this study may not be generalizable to other institutions or to future years. In addition, the wide range of costs reported indicates that averages may have limited applicability to individual students or small groups of students such as those applying to highly competitive specialties, interviewing in more than one specialty, or students with academic disadvantages. The data depend on student report after interviewing and could be subject to inaccuracy or recall bias. The data also do not take into consideration the potential impact of the Supplemental Offer and Acceptance Program (SOAP©) process. Additional studies of regional campus students at institutions across the country would provide more insight on the generalizability of our findings. Nevertheless, the study adds to our understanding of how students navigated the changes in the NRMP© interviewing process and may contribute to discussions about the future of the process. As medical education considers strategies to alleviate student debt, the expenses incurred during interviews for residency positions must be considered. Even with the dramatic reduction in costs documented after the change to remote interviewing, the pressure to over- apply and to undertake excessive numbers of interviews must be addressed, not just to benefit applicants but also to alleviate residency programs that are currently overwhelmed by excessive numbers of applications. CONCLUSION Students interviewing in 2021 reported costs almost 80% lower than applicants in the previous years. The greatest average savings (over $4,000) were for main campus students applying to non-primary care specialties, but the highest percentage savings (85%) were reported by regional campus applicants to primary care. Cost differences between the campuses diminished for both primary care and non-primary care applicants. Students reported saving 13 to 16 days through the change to remote interviewing, with regional campus students reporting the largest time savings. Conflict Disclosure: Tiffany Schwasinger-Schmidt, MD, PhD has conducted clinical trials research as principal investigator for the following pharmaceutical companies over the last twelve months: 1. Allergan 2. Eisai 3. Lundbeck 4. Janssen 5. SAGE pharmaceuticals 6. Sarepta 7. Corcept 8. Boehringer Ingelheim 9. Astra Zeneca 10. Axsome All clinical trial and study contracts were with and payments were made to the University of Kansas Medical Center Research Institute, which is a research institute affiliated with Kansas University School of Medicine-Wichita (KUSM-W). Journal of Regional Medical Campuses, Vol. 4, Issue 4 Original Reports The additional authors declare no conflicts of interest or competing interests. Acknowledgements: The authors would like to thank the team of medical students, residents, and faculty who created the survey used to collect the data for this study, as well as Drs. Mark Meyer and K. James Kallail for their help collecting additional data for this manuscript References 1. Haas MRC, He S, Sternberg K, Jordan J, Deiorio NM, Chan TM, Yarris LM. Reimagining residency selection: Part 1-A practical guide to recruitment in the post-COVID-19 era. J Grad Med Educ. 2020 Oct;12(5):539-544. doi:10.4300/JGME-D-20-00907.1. PMID: 33149819; PMCID: PMC7594771. 2. Sternberg K, Jordan J, Haas MRC, He S, Deiorio NM, Yarris LM, Chan TM. Reimagining residency selection: Part 2-A practical guide to interviewing in the post-COVID-19 era. J Grad Med Educ. 2020 Oct;12(5):545-549. doi: 10.4300/JGME-D-20-00911.1. PMID: 33149820; PMCID: PMC7594790. 3. Gabrielson AT, Kohn JR, Sparks HT, Clifton MM, Kohn TP. Proposed changes to the 2021 residency application process in the wake of COVID-19. Acad Med. 2020 Sep;95(9):1346- 1349. doi: 10.1097/ACM.0000000000003520. PMID: 32459675; PMCID: PMC7268839. 4. Wong JG, Flanagan MP, Horn KM. National Residency Matching Program (NRMP) results for students at the Main campus versus the Regional Medical Campus: A 6-year comparison at a single medical school. J Reg Med Campuses. 2020 May;3(1). doi: 10.24926/jrmc.v3i1.2971. 5. Fried JG. Cost of applying to residency questionnaire report 2015. Association of American Medical Colleges. Washington, DC. https://www.aamc.org/system/files/c/2/43090 2-costofapplyingtoresidency.pdf. Accessed December 2020. 6. Walling A, Nilsen K, Callaway P, Grothusen J, Gillenwater C, King S, Unruh G. Student expenses in residency interviewing. Kans J Med. 2017 Aug 30;10(3):1-15. PMID: 29472969; PMCID: PMC5733449. 7. Benson NM, Stickle TR, Raszka WV Jr. Going ‘‘fourth’’ from medical school: Fourth-year medical students’ perspectives on the fourth year of medical school. Acad Med. 2015 Oct; 90(10):1386-1393. doi: 10.1097/ACM.0000000000000802. PMID: 27002891. 8. Kerfoot BP, Asher KP, McCullough DL. Financial and educational costs of the residency interview process for Urology applicants. Urology. 2008 June;71(6):990-5. doi: 10.1016/j.urology.2007.11.102. Epub 2008 Mar 4. PMID: 18295310. 9. Teichman JM, Anderson KD, Dorough MM, Stein CR, Optenberg SA, Thompson IM. The Urology residency matching program in practice. J Urol. 2000 Jun;163(6):1878-87. PMID: 10799214. 10. Little DC, Yoder SM, Grikscheit TC, Jackson CC, Fuchs JR, McCrudden KW, Holcomb GW 3rd. Cost considerations and applicant characteristics for the Pediatric Surgery match. J Pediatr Surg. 2005 Jan;40(1):69-73; discussion 73-4. doi: 10.1016/j.jpedsurg.2004.09.013. PMID: 15868561. 11. Tichy AL, Peng DH, Lane AT. Applying for Dermatology residency is difficult and expensive. J Am Acad Dermatol. 2012 Apr;66(4):696-7. doi: 10.1016/j.jaad.2011.10.005. PMID: 22421121. 12. Guidry J, Greenberg S, Michael L. Costs of the residency match for fourth year medical students. Tex Med. 2014 June;110:e1. PMID: 24945234. 13. Agarwal N, Choi PA, Okonkwo DO, Barrow DL, Friedlander RM. Financial burden associated with the residency match in Neurological Surgery. J Neurosurg. 2017 Jan;126(1):184-190. doi: 10.3171/2015.12.JNS15488. Epub 2016 Apr 8. PMID: 27058197. 14. Camp CL, Sousa PL, Hanssen AD, Karam MD, Haidukewych GJ, Oakes DA, Turner NS. The cost of getting into Orthopedic residency: Analysis of applicant demographics, expenditures, and the value of away rotations. J Surg Educ. 2016 Sep-Oct;73(5):886-91. doi: Journal of Regional Medical Campuses, Vol. 4, Issue 4 Original Reports 10.1016/j.jsurg.2016.04.003. Epub 2016 May 12. PMID: 27184179. 15. Callaway P, Melhado T, Walling A, Groskurth J. Financial and time burdens for medical students interviewing for residency. Fam Med. 2017 Feb;49(2):137-140. PMID: 28218940. 16. Blackshaw AM, Watson SC, Bush JS. The cost and burden of the residency match in Emergency Medicine. West J Emerg Med. 2017 Jan;18(1):169-173. doi: 10.5811/westjem.2016.10.31277. Epub 2016 Dec 19. PMID: 28116032; PMCID: PMC5226755. 17. Chang PS, Rezkalla J, Beard M. An analysis of the financial burden associated with the residency match at the University of South Dakota Sanford School of Medicine. S D Med. 2018 Feb;71(2):66-69. PMID: 29990414. 18. Susarla SM, Swanson EW, Slezak S, Lifchez SD, Redett RJ. The perception and costs of the interview process for Plastic Surgery residency programs: Can the process be streamlined? Plast Reconstr Surg. 2017 Jan;139(1):302e-309e. doi: 10.1097/PRS.0000000000002912. PMID: 27632394. 19. Polacco MA, Lally J, Walls A, Harrold LR, Malekzadeh S, Chen EY. Digging into debt: The financial burden associated with the Otolaryngology match. Otolaryngol Head Neck Surg. 2017 Jun;156(6):1091-1096. doi: 10.1177/0194599816686538. Epub 2017 Jan 24. PMID: 28116996. 20. Cabrera-Muffly C, Chang CWD, Puscas L. Current interview trail metrics in the Otolaryngology match. Otolaryngol Head Neck Surg. 2017 Jun;156(6):1097-1103. doi: 10.1177/0194599817690723. Epub 2017 Feb 7. PMID: 28168889. 21. Fogel HA, Liskutin TE, Wu K, Nystrom L, Martin B, Schiff A. The economic burden of residency interviews on applicants. Iowa Orthop J. 2018;38:9-15. PMID: 30104919; PMCID: PMC6047386. 22. Nikonow TN, Lyon TD, Jackman SV, Averch TD. Survey of applicant experience and cost in the Urology match: Opportunities for reform. J Urol. 2015 Oct;194(4):1063-7. doi: 10.1016/j.juro.2015.04.074. Epub 2015 Apr 23. PMID: 25912495. 23. Nilsen KM, Walling A, Grothusen J, Irwin G, Meyer M, Unruh G. Time and financial costs for students participating in the National Residency Matching Program (the Match©): 2015 to 2020. Kans J Med. 2021 Mar 19;14:53- 63. doi: 10.17161/kjm.vol1414568. PMID: 33763180; PMCID: PMC7984744. 24. Youngclaus J, Fresne JA. Physician Education Debt and the Cost to Attend Medical School: 2020 Update. Washington, DC: AAMC; 2020. https://store.aamc.org/downloadable/downlo ad/sample/sample_id/368/. Accessed August 25, 2021. 25. Vasquez R, Jeong H, Florez-Pollack S, et al. What are the barriers faced by under- represented minorities applying to dermatology? A qualitative cross-sectional study of applicants applying to a large dermatology residency program. J Am Acad Dermatol. 2020 Dec;83(6):1770-1773. 26. Wilson LT, Milliken L, Cagande C, Stewart C. Responding to recommended changes to the 2020-2021 residency recruitment process from a diversity, equity, and inclusion perspective. Acad Med. 2021 Aug 10. doi: 10.1097/ACM.0000000000004361. Epub ahead of print. PMID: 34380938. 27. Coalition for Physician Accountability’s Work Group on Medical Students in the Class of 2021 Moving Across Institutions for Post Graduate Training. Final Report and Recommendations for Medical Education Institutions of LCME-Accredited, U.S. Osteopathic, and Non-U.S. Medical School Applicants. Washington, DC: AAMC; 2021. https://www.aamc.org/media/44736/downloa d. Accessed August 14, 2021. 28. Nilsen K., Walling, A., Callaway, P. et al. “The End Game”- Students’ perspectives of the National Residency Matching Program: A focus group study. Med Sci Educ. 28, 729–737. 29. Wolf SJ, Lockspeiser TM, Gong J, Guiton G. Students' perspectives on the fourth year of medical school: A mixed-methods analysis. Acad Med. 2014 Apr;89(4):602-7. 30. Walling A, Merando A. The fourth year of medical education: A literature review. Acad Med. 2010 Nov;85(11):1698-704. doi: Journal of Regional Medical Campuses, Vol. 4, Issue 4 Original Reports 10.1097/ACM.0b013e3181f52dc6. PMID: 20881826. 31. Luftig D. The Residency Interview Scheduling Process: Unintended consequences and a proposal for change. J Grad Med Educ. 2015 Mar;7(1):134. doi: 10.4300/JGME-D-14- 00603.1. PMID: 26217445; PMCID: PMC4507910. Figure Legend Figure 1. Average Cost Per Year for Primary Care and Non-Primary Care Applicants by Location Figure 2. Average Time Spent Per Year for Primary Care and Non-Primary Care Applicants by Location Journal of Regional Medical Campuses, Vol. 4, Issue 4 Original Reports APPENDIX MS4 NRMP Match Survey 2021 1. Did you participate in the 2020-2021 NRMP© match process? • No i. If no, in which process did you participate to find a residency position? • Yes **End of survey for participants who did not participate in the NRMP© process** 2. What was your primary specialty choice? 3. To how many specialties did you apply? • One • Two Three or more 4. Why did you apply to more than one specialty? • Interest in more than one specialty • Need for a “safety net” option • Other (Please specify) 5. Did you attend meet-and-greets in your specialty(ies) of choice prior to submission of your application to those programs? • No • Yes 6. Was the list of programs to which you applied affected by your participation in pre- submission meet-and-greets? • No • If yes, did your participation result in your applying to: i. More programs? ii. Fewer programs? 7. Did you apply to any transitional/preliminary programs? • No • If yes… i. To how many transitional/preliminary programs did you apply? ii. How many transitional/preliminary programs offered you an interview? iii. How many transitional/preliminary program interviews did you complete? iv. How many transitional/preliminary programs did you include on your rank list? 8. To how many categorical residency programs did you apply? 9. How many categorical residency programs offered you an interview? 10. How many categorical residency program interviews did you complete? 11. How many categorical residency programs did you include on your rank list? 12. In retrospect, do you think the number of programs you applied to was: • Too many • About right • Too few 13. In retrospect, was the number of interviews you completed: Journal of Regional Medical Campuses, Vol. 4, Issue 4 Original Reports • Too many • About right • Too few 14. Did you participate in couples match? • No • Yes 15. Did you apply to more programs due to the virtual format? • Yes, absolutely • Maybe • No • I am not sure 16. Did you accept more interviews due to the virtual format? • Yes, absolutely • Maybe • No • I am not sure 17. Did you complete more interviews due to the virtual format? • Yes, absolutely • Maybe • No • I am not sure 18. To what extent did any of the following limit your decision to accept interview invitations? (Response options: very limiting, somewhat limiting, not limiting) • Cost associated with interviewing • Time spent interviewing • Issues with scheduling • Participation in the program's pre- submission meet-and-greet • Virtual away experiences with programs • Other (Please specify) 19. Other than cost and time, how important were the following factors in deciding which interviews to accept? (Response options: not important, somewhat important, very important) • Recommendation from Student Affairs • Recommendation from medical school department or faculty • Advice from residents • Recommendation from classmates and other students • Literature/websites and published information from programs • Reputation of program including clinical/ medical expertise • Program offered a good fit with my interests • Experience working in the program (e.g., away rotation or sub-internship) • Desire to stay in a specific city/ location • Perception of program’s commitment to resident wellness and well-being • Scholarship/ academics of the program • Program commitment to diversity and/or underserved populations • Program reputation • Other (Please specify) 20. What was your total estimated expense related to your interviews (in whole dollars)? 21. How did you pay for your interview expenses? Please check all that apply: • Personal savings • Credit card • Medical student loans • Private loans • Gift from family • Other (Please specify) 22. What platform(s) did you use to coordinate your interview schedule (i.e., ERAS, Thalamus, etc.)? 23. If you used more than one program, which one did you prefer? 24. How many days in total did you participate in interviews? 25. How many interviews did you attend during: • October? ____ • November? ____ • December? ____ • January? ____ • February? ____ Journal of Regional Medical Campuses, Vol. 4, Issue 4 Original Reports 26. On average, how many days’ notice did you receive prior to your interview date? 27. What was the least amount of notice that you received prior to an interview date (in days)? 28. On average, how quickly did you need to respond to interview requests to secure a spot? • 1-5 minutes • 6-10 minutes • Within an hour • Within 24 hours • More than 24 hours 29. On average, how soon after receiving an interview invitation did you decline it? • 1-5 minutes • 6-10 minutes • Within an hour • Within 24 hours • More than 24 hours 30. What were the reasons you declined interviews upon receipt? Please check all that apply: • I had personal or family reasons • I had logistical issues (travel, MS schedule, timing, etc.) • I had financial issues • I found issues with the residency program after applying • I changed my mind about the residency program • I was less likely to rank a program after receiving invitations from other programs (low priority of program) • Other (Please specify) 31. How many interviews did you accept but then cancel later? 32. What were the reasons you canceled scheduled interviews? Please check all that apply: • I had personal or family reasons • I had logistical issues (travel, MS schedule, timing, etc.) • I had financial issues • I found issues with the residency program after applying • I changed my mind about the residency program • I was less likely to rank a program after completing interviews with other programs (low priority of program) • Preference of spouse/significant other on where we should live • Other (Please specify) 33. On average, how close to the scheduled interview date did you cancel interviews? • Same day • Within a few days • Within a week • Within a month • Never 34. Did the programs you accepted invitations from offer pre- or post-interview resident meet-and-greets? • No • If yes, did you participate in: i. All meet-and-greets ii. Some meet-and-greets iii. No meet-and-greets 35. Did any programs request additional information before offering you an interview? • No • If yes, what additional information did they request? 36. Did any programs require that you travel in- person to their location? • Yes • No 37. Did any of the programs you interviewed with offer post-interview activities? Please check all that apply: • Mandatory activities i. No ii. If yes, what kind of activities did expect you to attend? • Optional activities i. No ii. If yes, what kind of activities did they offer? • No activities offered Journal of Regional Medical Campuses, Vol. 4, Issue 4 Original Reports 38. Have you received any communication from programs following your interview with them (such as emails or letters)? • No • If yes, what additional communication have you received? Demographics 39. What is your gender? • Male • Female • Chose not to disclose • Other (Please specify) 40. What is your age? 41. Which campus do you attend? 42. Are you originally from [our state]? • Yes • No 43. How would you characterize your hometown? • Urban • Suburban • Midsize Rural • Small Rural Open-ended Questions 44. How did you personally feel about the virtual format of residency interviewing? 45. If you could do anything to improve the NRMP© process, what would you do? 46. Please provide any comments or feedback that you think we should know about your experience with the NRMP© process. THANK YOU and BEST WISHES FOR SUCCESS IN YOUR RESIDENCY CAREER If you have any questions, please contact [the research lead].