Microsoft Word - EvidenceBasedFacultyArticle.docx Published by University of Minnesota Libraries Publishing Evidence-based Faculty Development Programming for Regional Medical Campuses and the BASK Framework Ralitsa Akins, MD, PhD DOI: https://doi.org/10.24926/jrmc.v2i4.1304 Journal of Regional Medical Campuses, Vol. 2, Issue 4 (2019) z.umn.edu/JRMC All work in JRMC is licensed under CC BY-NC Ralitsa Akins, MD, PhD; has served in academic leadership roles for over 15 years, including Associate Dean of Academic Affairs, Senior Associate Dean, Associate Dean of Faculty Affairs, and is currently the Provost at Des Moines University. Corresponding author: Ralitsa Akins, MD, PhD Email: ralitsa_akins@yahoo.com All work in JRMC is licensed under CC BY-NC Volume 2, Issue 4 (2019) Journal of Regional Medical Campuses Original Reports Evidence-based Faculty Development Programming for Regional Medical Campuses and the BASK Framework Ralitsa Akins, MD, PhD Abstract Medical schools’ regional campuses respond to vital needs in medical education. About one third of US medical schools have regional campuses. It is important to create and maintain, in all geographically separate locations, climate and culture conducive to effective teaching and learning. An evidence-based approach to designing and implementing faculty development programs for regional medical campuses is described, and the BASK assessment framework is introduced, recognizing the interconnectedness between desired changes in behaviors, and learners’ attitudes, skills, and knowledge. IRB approval was obtained for the completion of this study. Introduction There is a paucity of publications on the topic of effective approaches in faculty development. Traditionally, it has been difficult to reach to all faculty in geographically separate locations, and online tools have varied reports of success. A literature review was completed to identify best practices and evidence-supported successful approaches in implementing faculty development programs for regional campuses. An experienced outreach librarian supported a database search for articles on faculty development, with focus on approaches in distributed healthcare education. The articles from the initial search were reviewed by the author for relevance and an annotated bibliography was developed. Based on the outcomes and topics within the articles, including LCME publications about most often cited deficiencies in medical schools, a faculty development program was developed and implemented in a new medical school with 4 regional campuses. A new framework for evidence-based faculty program assessment was developed: BASK, based on Behaviors, Attitudes, Skills and Knowledge. Findings from the literature Faculty development is not a luxury, rather an imperative for every medical school. Sustainable faculty development requires a medical education unit or department, staffed with respected faculty developers who are academic role models. The literature suggests the following: faculty development should be tailored to suit the needs of individuals, disciplines, and the institution, activities used in faculty development programs should encourage experiential learning and reflection (e.g. peer evaluation, portfolios), and that faculty development should strive for collaboration across medical disciplines, and where possible, across professions.10 Leslie et al (2013) published a review exploring the nature and scope of faculty development in medical education, the quality of publications on faculty development, and identified meaningful areas for future research. The study found that most commonly used faculty development format was a longitudinal series of presentations for physicians only, and that the faculty development activities usually aimed to improve teaching, leadership, and scholarship. In studying faculty development programs, non-validated surveys were the most common data collection method, participants were the usual data source, and the commonly reported outcome was self-reported behavior changes. A gap was found in the current literature in exploring the impact of contextual/institutional factors in faculty development success and the need to implement more rigorous evaluation methods in faculty development assessment. Focus groups have been often used to study the reasons for attendance and the deterrent factors for attending faculty development sessions, and how to make faculty development programs more pertinent to faculty’s needs. Steinert et al (2010) suggested that faculty participated when they perceived that faculty development enabled personal and professional growth, they valued learning and self- improvement, the workshop topics were viewed as relevant to teachers’ needs, the opportunity to network with colleagues was appreciated, and initial positive experiences promoted ongoing involvement. Barriers against participation cited by non-attendees included volume of work, lack of time, and logistical factors. Suggestions for increasing participation included introducing a ‘buddy system’ for junior faculty members, an orientation workshop for new staff, and increased role-modelling and mentorship.18 At McGill University, an all-faculty development program was implemented, and the motivators and deterrents to participation in faculty development programs were studied, along with perceived barriers for involvement.11 It was determined that participants who regularly attended faculty development sessions perceived the topics as relevant to DOI: https://doi.org/10.24926/jrmc.ADDHERE Journal of Regional Medical Campuses, Vol. 2, Issue 4 Original Reports their daily work and professional growth and valued the opportunities to network. Non-attendees were deterred by inconvenient locations (e.g. program offered on central location only), inconvenient timing of sessions, lack of protected time, and lack of recognition and financial rewards for teaching. Approaches in faculty development that were considered as effective included medical education presentations at grand rounds, site-specific workshops, and follow-up “booster sessions”. Those involved in providing faculty development may be among only a few individuals for whom faculty development is an interest and priority within their work setting, and oftentimes funding to support faculty development is limited.15 Graziano et al (2018) noted that concerns about clinical productivity, along with challenges in identifying physicians willing to teach were among the barriers to engaging community-based preceptors, and considered that faculty development could be a positive solution for both, assisting with improved teaching efficiency and sustained clinical productivity. It can be a challenge to recruit, train, and retain community preceptors. It is time-consuming to provide training, answer questions, maintain a connection, and ensure that preceptors are recognized for the role they play in educating the next generation of physicians. The Society of Teachers of Family Medicine has developed a resource for practitioners-preceptors and residency programs, available online. TeachingPhysician.org streamlines training, and includes topics on preparing a practice team for a student or resident, integrating a student into office routines, setting expectations teaching strategies, giving feedback, evaluating learners, billing issues; etc. Community preceptors can log in to access tools and resources to help them teach the next generation of physicians to provide high-quality, patient- centered care.16 Communicating faculty development news and information requires reaching faculty members in the multiple communities of practice. Traditional strategies such as face- to-face programs and printed newsletters no longer have the audience or impact they once had. Few schools use social media as tools to engage faculty. A faculty development blog, linked to the school’s medical education webpage, could offer customized, on-demand information about professional development topics, brief literature syntheses, and announcements of events and resources. Blogging could be used for time-efficient information sharing,5 yet it has been empirically known that not all faculty embrace e-learning. Reilly, Vandenhouten and Gallagher-Lepak (2012) completed a literature review on faculty development and e-learning and described a multi-campus faculty development program using distance technology within a community of practice model in nursing. They concluded that evaluation of faculty development programs should be planned early in the design process, faculty development programs should build on previous activities, there should be opportunities to discuss classroom experiences with colleagues, ongoing professional communication of instructors with similar concerns should be encouraged, and faculty development for instructional technology must be focused on pedagogy, and not simply on technology skill acquisition. When preceptors in a required preceptorship program were offered traditional continuing medical education (CME), a preceptor listserv, an electronic clinical teaching discussion group, an orientation videotape, a CD-ROM on teaching skills, and technology support, the preceptors agreed that the listserv and the electronic clinical teaching case discussion were most useful and that compared to the rest of electronic tools used, held the best promise for preceptor faculty development.1 On the other end of the spectrum from electronic communications, are the in-person sessions, where the instructor and the faculty-learners are in the same classroom – in a group or one-on-one setting. Moser, Dorsch and Kellerman (2004) suggested using “academic detailing” as a method to deliver real-time faculty development to nearly all community preceptors, both rural and urban, using the RAFT technique (Rapport-building, Assessment, Focused preceptor development, Thanks/Trinkets). The method could be expensive in terms of faculty time and might be difficult to schedule, yet could be a valuable, learner-centered method that reaches preceptors missed by traditional faculty development workshops. Studies suggest that current faculty competencies may be lagging behind the expectations related to teaching and evaluation. Holmboe et al. (2011) suggested that faculty development efforts should be focused on improving evaluation skills and developing life-long learning skills. The authors offered a 5-step model to improve assessment in medical education including 1) utilizing Frame-of-Reference training approach, 2) giving feedback to faculty about their performance as evaluators, 3) providing working knowledge of basic core psychometric concepts, 4) ensuring availability of web-based training modules for faculty development, and 5) learner active involvement, including completing self- assessment. Nichols, Kulaga and Ross (2013) studied emergency medicine faculty’s skills in providing feedback and found that while preceptors provided good verbal feedback, skills were lacking in documentation/written feedback. One-on-one training was provided to preceptors, their skills in providing feedback were assessed using a 5-point scoring system, and part of the faculty also viewed a short video of a learner in the clinical setting, used for deliberate practice in writing feedback. A pre-post skills assessment revealed that if the deliberate practice (assessing a video) portion was not implemented, there was only a minimal improvement of preceptors’ skills, and therefore deliberate practice was considered a key step in changing behaviors in both, learners and teachers.13 Steinert (2005) suggested that faculty development initiatives could bring about change at the individual and the organizational level and recommended that such sessions target diverse stakeholders, take place in a variety of settings, DOI: https://doi.org/10.24926/jrmc.ADDHERE Journal of Regional Medical Campuses, Vol. 2, Issue 4 Original Reports use diverse formats and educational strategies and have deliberate educational design. While faculty development self-reported outcomes data, such as participant satisfaction or confidence, are easily obtained, it is more challenging to measure higher-level outcomes of a successful faculty development program. Guglielmo et al. (2011) suggested that validated, reliable evaluation tools should be used in the assessment of faculty development, such as the Kirkpatrick’s model. The Kirkpatrick’s model measures the reaction of participants (i.e. participant satisfaction), learning (knowledge, skills, and attitudes), behavioral changes, and the impact on learners. Furthermore, the optimal evaluation of any faculty development program should utilize both quantitative and qualitative measures, and should include a pre- and post-evaluation, and/or a delayed post-evaluation.3 Faculty development has been effective in improving faculty perceptions about the value of teaching, increasing motivation and enthusiasm for teaching, increasing knowledge and behaviors, and in disseminating teaching skills. Lancaster et al (2014) described 10 steps for building a successful faculty development program, to include: 1. Build stakeholders by listening to all perspectives 2. Ensure effective program leadership and management 3. Emphasize faculty ownership 4. Cultivate administrative commitment 5. Develop guiding principles, clear goals, and assessment procedures 6. Strategically place faculty development within the organizational structure 7. Offer a range of opportunities, but lead with strengths 8. Encourage collegiality and community 9. Create collaborative systems of support 10. Provide measures of recognition and awards Langlois and Thach (2003) described the lessons learned in developing, using, and disseminating a collection of preceptor development materials designed to be relevant to community-based faculty and easy to use. Topics were oriented to meet community preceptors’ needs. There were 9 topics and 5 faculty development formats used. Topics included setting expectations, evaluation, feedback, teaching and learning styles, one-minute preceptor, integrating learners into clinical offices, dealing with difficult situations, teaching at the bedside, and advanced training for experienced preceptors. The 5 formats included seminars, monographs, web modules, thumbnails (one-page handouts), and videos that could be downloaded and customized. Lancaster et al (2014) also proposed specific topics for faculty development in teaching, learning, and assessment, to include syllabus/course design, writing objectives, constructing assessments, rubric design, grading strategies, student motivation, learning disabilities, classroom management, active learning, presentation and communication skills, self-reflection, and searching and evaluating evidence. Hunt et al (2016) described the most common issues leading to severe accreditation actions against medical schools, such as probation, when judging schools’ compliance with the accreditation standards. A number of the issues they identified could be successfully addressed through faculty development activities, including curriculum management, comparability across instructional sites, systematic review and revision of the curriculum, career counseling, midcourse feedback, educational program objectives, student mistreatment, health care providers’ involvement in student assessment, and formative and summative assessment. In summary, faculty development needs to be systematic, involving planning, implementation and evaluation, and its outcomes should be realistic, task-oriented and measurable.10 In addition, faculty development should help the faculty and the institution to meet the medical program’s goals at all instructional sites. BASK: An evidence-based faculty development assessment framework As demonstrated above, while various elements of what works in faculty development have been studied, faculty development literature does not offer a cohesive, overarching approach to faculty development. Findings from the literature review were used to design a year-long faculty development program for a new medical school that started with the simultaneous development of 4 regional medical campuses and develop a new evidence-based framework for assessment of faculty development programming, BASK, based on Behaviors, Attitudes, Skills, and Knowledge. Program areas/topics, methods of delivery and instructional methodology that were identified as most effective in the literature were implemented. Data from the attendee assessment forms from the faculty development sessions were analyzed. Analyses were performed using IBM SPSS Statistics v. 24 software. In addition, the faculty development program was designed following the principles of Continued Quality Improvement, and the BASK assessment framework was introduced. Introducing the BASK assessment framework Traditionally, acquisition and attainment of knowledge and skills, and assessment of the attitudes of the learners have been considered as major outcomes in medical education, including student and faculty assessment. This triad of knowledge, skills, and attitudes has become a staple in DOI: https://doi.org/10.24926/jrmc.ADDHERE Journal of Regional Medical Campuses, Vol. 2, Issue 4 Original Reports assessment and accreditation reviews. Since the major goal of education and continued education is a positive impact on behaviors, and change in behaviors is brought about by participants’ attitudes, which are in turn supported by their knowledge and skills, the author suggests a new framework to reflect this interdependence in achieving incomes: the BASK framework. BASK stands for Behaviors, Attitudes, Skills, and Knowledge, and describes the outcomes dependence of all desired behaviors on learners’ attitudes in implementing and sustaining such improved behaviors, and in turn, learners’ attitudes’ dependence on their skills and knowledge acquisition, sustainment, and continued improvement. The acronym has a positive connotation and is easy to remember. Quality improvement design of the faculty development program utilizing the BASK assessment framework Prior to the start of the faculty development program, a continued quality improvement model was embedded in its planned delivery, to include: • Assessment using the BASK framework: Behaviors, Attitudes, Skills and Knowledge • Assessment of the satisfaction, learning and behavior elements of the Kirkpatrick’s program assessment model, as reported by participants at each session • A pre- and post-test assessment format, to support the BASK framework and the Kirkpatrick’s elements assessment • Yes/No questions to assess if participants gained new knowledge/learning • Yes/No questions to assess if participants plan to change behaviors after session • “You Said – We Did” report to participants • Session outcomes summary feedback to presenters Institutional Review Board approval was sought to analyze the quality improvement data from the faculty development program upon completion of the 2016-2017 academic year, when the program was delivered, and exempt status of the study was granted under project IRB #16228/2017. Results from the implementation of the BASK framework In academic year 2016-17, seventy-three faculty development sessions were delivered on the main campus and the 4 regional campuses of one new medical school. A majority of sessions offered video- and teleconferencing options for participants, all sessions amenable for video-recording were recorded, archived, and accessible (excluding sessions involving exclusively small-group discussions), and 12 sessions were delivered on-site at the campus locations. Three hundred and three individual faculty attended one or more sessions, for a total number of session attendees of 1 125. On average, each session was attended by 15 individuals, ranging from 6 to 41. Sessions designed for course directors had more limited attendance related to the target audience, and sessions on curriculum, assessment and hot topics in education attracted larger audiences. Attendee satisfaction was measured at each session, and a pre-post-test design of assessment based on a 5-point Likert scale was implemented for questions exploring: 1) participants’ self-reported intention to change behaviors, 2) change in attitudes, 3) gaining of new skills, and 4) gaining of new knowledge, as related to the faculty offered development sessions. In addition, at the end of the sessions, Yes/No questions to assess if participants gained new knowledge/learning, and Yes/No questions to assess if participants plan to change behaviors were asked. Table 1 below presents the faculty development program topic distribution. Table 1. Faculty development topics distribution in percentages Participant satisfaction (an element of the Kirkpatrick’s program assessment model) was measured for each session, and the outcomes are presented in Table 2. For the faculty development program, there we 11 male presenters who presented a total of 25 sessions, 10 female presenters who presented a total of 39 sessions, and 9 sessions were presented by teams of both genders. We asked the participants to measure their satisfaction with the sessions, with the perceived knowledge of the presenters, and the perceived communication skills of the presenters. As seen on table 2, team presentations yielded the least satisfaction. Table 2. Session satisfaction 7 30 12 12 10 29 0 10 20 30 40 Assessment & Evaluation Curriculum Design Instructional Methodology Leadership Research & Scholarship Multiple/Mixed Topics DOI: https://doi.org/10.24926/jrmc.ADDHERE Journal of Regional Medical Campuses, Vol. 2, Issue 4 Original Reports Paired two-tailed T-test was used to assess statistically- significant changes, and significance was reported at the 0.05 level. After the conclusion of 71% of the sessions, participants reported that they were planning to change behaviors based on what they learned during the faculty development sessions. Thirty-two percent of the sessions impacted participants’ attitudes in the desired direction. Participants perceived that their skills improved after 79% of the sessions, and that their knowledge improved after 82% of the sessions. In 5% of the sessions, there was no statistically significant change in attendees’ behaviors, attitudes, skills, or knowledge. Ninety-five percent of the sessions yielded participant self-reports of positive change in at least one of the BASK elements (behaviors, attitudes, skills and knowledge), and 26% of the sessions impacted all BASK elements, i.e. there was a statistically-significant difference after the session in all BASK elements. The results of the analyses are presented in Table 3. Table 3. Pre-post assessment of the BASK elements After the conclusion of the faculty development sessions, a summary report about the session outcomes and feedback from participants was sent to each presenter to aid future quality improvement of faculty development sessions. Answers to questions raised during the sessions, and requested materials not available during the presentations were sent to the session participants in the fashion of “You Said, We Did” reports. Feedback from attendees and presenters indicated such follow-up communication and reporting was helpful and much appreciated. Study limitations The implementation of the evidence-based faculty development program included the 4 campuses of one medical school, and, therefore, considerations regarding local circumstances need be factored in when applying the principles utilized by this study to other medical schools and their regional campuses and geographically separate instructional sites. Conclusion The evidence-based nature of this study, including a literature review about faculty development at regional campuses, utilizing data from the accreditation body for medical schools (LCME) regarding factors leading to severe accreditation Session Satisfaction (average, scale 1-5) Assessment of Presenter Knowledge (average, scale 1-5) Assessment of Presenter Communic ation Skills (average, scale 1-5) Male Presenters (n=11, presented 25 sessions) 4.4 4.71 4.59 Female Presenters (n=10, presented 39 sessions) 4.48 4.71 4.67 Team Presentation (F, M) (presented 9 sessions) 3.91 4.28 4.07 4.4 4.71 4.59 4.48 4.71 4.67 3.91 4.28 4.07 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 Male Presenters (n=11, presented 25 sessions) Female Presenters (n=10, presented 39 sessions) Team Presentation (F, M) (presented 9 sessions) BASK Framework Elements Sessions with Statistically Significant Difference Pre-Post B - Behaviors 71% A - Attitudes 32% S - Skills 79% K - Knowledge 82% No difference pre-post in any BASK element – 5% of sessions At least one BASK element affected – 95% of sessions All BASK elements affected – 26% of sessions DOI: https://doi.org/10.24926/jrmc.ADDHERE Journal of Regional Medical Campuses, Vol. 2, Issue 4 Original Reports decisions, as well as the quality improvement design of data collection and the pre-post testing model, suggest that the results of the study could inform the efforts of medical schools in building evidence-based faculty-development programs. Additional studies on the topic of faculty development in medical schools, and regional medical campuses in particular, could lead to the development of a “core” faculty development series for those behaviors, attitudes, skills, and knowledge considered uniform across medical school settings, and in that way advance towards a national curriculum in “core” competencies for medical school faculty. Such national curriculum could be informed by the BASK framework, recognizing the pivotal importance of behaviors, attitudes, skills, and knowledge in designing evidence-based faculty development programs and implementing meaningful faculty development outcomes assessment. Acknowledgements The author would like to acknowledge Kathryn Vela, MLIS, AHIP, Health Sciences Outreach Librarian with the Spokane Academic Library at Washington State University for her support in the literature review articles search. References 1. Bramson R, VanLandingham A, Heads A, Paulman P, Mygdal Wi. (2007). Reaching and Teaching Preceptors: Limited Success from a Multifaceted Faculty Development Program. Fam Med 39(6):386- 388. 2. Graziano SC, McKenzie ML, Abbott JF, Buery-Joyner SD, Craig LB, Dalrymple JL, Forstein DA, Hampton BS, Page-Ramsey SM, Pradhan A, Wolf A & Hopkins L (2018): Barriers and Strategies to Engaging Our Community-Based Preceptors, Teaching and Learning in Medicine, DOI: 10.1080/10401334.2018.1444994 (accessed 4/7/18). 3. Guglielmo BJ, Edwards DJ, Franks AS, Naughton CA, Schonder KS, Stamm P, Thornton P, & Popovich NG. A Critical Appraisal of and Recommendations for Faculty Development. American Journal of Pharmaceutical Education 2011; 75 (6) Article 122. 4. Holmboe ES, Ward DS, Reznick RK, Katsufrakis PJ, Leslie KM, Patel VL, Ray DD, & Nelson EA. (April, 2011). Faculty Development in Assessment: The Missing Link in Competency-Based Medical Education. Academic Medicine 86(4): 460-467. 5. Huggett K. (2015). A faculty development blog: curated and convenient. Medical Education, 49:1160-1161. 6. Hunt D, Migdal M, Waechter DM, Barzansky B & Sabalis RF. (2016). The Variables That Lead to Severe Action Decisions by the Liaison Committee on Medical Education. Acad Med. 91:87–93. 7. Lancaster J, Stein SM, MacLean LG, Van Amburgh J, & Persky AM. (2014). Faculty Development Program Models to Advance Teaching and Learning Within Health Science Programs. American Journal of Pharmaceutical Education 2014; 78 (5) Article 99. 8. Langlois JP. & Thach SB. (2003). Bringing Faculty Development to Community-based Preceptors. Academic Medicine 78(2): 150-155. 9. Leslie K, Baker L, Egan-Lee E, Esdaile M, & Reeves S, PhD. (July, 2013). Advancing Faculty Development in Medical Education: A Systematic Review. Academic Medicine, 88(7): 1038-1045. 10. McLean M, Cilliers F & Van Wyk JM. (2008). Faculty development: Yesterday, today and tomorrow. Medical Teacher, 30:555-584. 11. McLeod PJ, Steinert Y, & Boillat M. (2011). How Faculty Development Research Can Inform Practice. Medical Education, 45: 1150-1151. 12. Moser SE, Dorsch JN, Kellerman R. (2004). The RAFT Approach to Academic Detailing with Preceptors. Fam Med 36(5):316-8. 13. Nichols D, Kulaga A, & Ross S. (2013). Coaching the Coaches: Targeted Faculty Development for Teaching. Medical Education, 47: 534-535. 14. Reilly JR, Vandenhouten C, Gallagher-Lepak S. (2012). Faculty Development for E-learning: A Multi- campus Community of Practice (COP) approach. Journal of Asynchronous Learning Networks, Volume 16: Issue 2, 99-110. 15. Sicat BL, Kreutzer K, Gary J, Ivey CK, Marlowe EP, Pellegrini JM, Shuford VP & Simons DF. (2014). A Collaboration among Health Sciences Schools to Enhance Faculty Development in Teaching. American Journal of Pharmaceutical Education; 78 (5) Article 102. 16. Society of Teachers of Family Medicine (2012). New Website helps departments and programs effectively train and support preceptors. Ann Fam Med 10(1):82-83. 17. Steinert Y. (2005). Learning together to teach together: Interprofessional education and faculty development. Journal of Interprofessional Care, (May 2005) Supplement 1: 60-75. 18. Steinert Y, Macdonald ME, Boillat M, Elizov M, Meterissian S, Razack S, Ouellet MN, & McLeod PJ. (2010). Faculty development: if you build it, they will come. Medical Education, 44: 900-907.