SUMMARY Journal of Islamabad Medical & Dental College (JIMDC); 2016:5(3):134-138 134 Original Article Evaluation of Faculty Development Program: A Collaborative Project of NIH&SS Pakistan and PHMI, USA Kauser Abid1, Farzana Majeed2 and Abdul Majeed3 1Assistant Professor, Physiology Department, PBL Coordinator, Medical Education Department, Faculty of Medicine, University of Tabuk, KSA 2Assistant Professor Department Physiology, Islamabad Medical & Dental College 3Professor, HOD, Department of Physiology, Islamabad Medical & Dental College (2&3 Shaheed Zulfiqar Ali Bhutto Medical University, Islamabad) Abstract Objective: To develop logic model for the planning, evaluation and assessment of the process of collaborative faculty development program of Partners Harvard Medical International (PHMI) and National Institute of Health and Social Sciences (NIH&SS). Material & Methods: This study was done in the Capacity development building of NIH&SS in March 2010. The faculty development program was carried out by NIH&SS in collaboration with PHMI, USA. At the initial stage a logic model was developed to highlight the planning and evaluation of the program for all the stake holders. In second stage, quasi- experimental study was designed and data was collected from mediation. Comparison of the groups was obtained by carrying out pretest and a post-test questionnaire. Purposive sampling was done from target population being mentors serving as faculty in healthcare profession. Descriptive statistics was done using Univariate and Bivariate analysis. Results: Univariate analysis showed that 95% of program participants did meet their personal professional needs; more than 95% reported change in attitude towards team work in diverse group setting, more than 98% reported gain in knowledge and skill for innovative teaching strategies, 96% of participants considered experimental learning helpful in developing assessment items. The bivariate analysis showed that the educational activities and exercises of the program succeeded to accomplish the outcomes. Conclusion: Capacity building of mentors in health profession at the workplace can address the needs of all the stakeholders in terms of time and resource management Key Words: Capacity building, Educators, Health care logic model, Work place learning. Introduction To maintain academic vigor, faculty development is an important milestone which in turn helps to meet the changing needs in the field of medical education. 1 Continuous Professional Development (CPD) for mentors is the process through which health profession educators can maintain, improve and expand current standards of best professional practices. Faculty development programs (FDPs) are crucial for the existing and new incoming faculty members for professional development in order to fulfill their varying roles in planning and implementing curricular changes. These programs are meant to empower the medical educator’s community in a way to create a positive institutional environment and can range from basic orientation programs for new faculty members to postgraduate medical education programs for health professionals. Overall, the aim of all these training programs is to support educators in the health profession to bring reforms in the teaching strategies and to enhance the efficiency and performance of their teaching skills. This would not only improve their work satisfaction but also teaching confidence. Moreover, institutes should design faculty development models according to their own needs and setting. 2-5 In clinical setting we observe that clinicians have time constrains for arranging FDPs that can fit in their tight schedule. Anupma Wadhwa, et al designed a limited time committed FDP for the clinicians which showed improvement in their scholarly efficacy. 6 Faculty Development Program Context FDPs for educators in the healthcare profession was launched for the first time in the region of Islamabad, Pakistan. National Institute of Health & Social sciences (NIH&SS), Pakistan in collaboration with Partners, Harvard Medical International (PHMI), USA offered a platform that could provide support and teaching/learning mentoring services to the educators of health care profession. For this program logic model was developed which provided a chance to communicate with all the stakeholders about the Corresponding Author: Dr Kauser Abid Email address: drkauserabid@gmail.com Received: Sept 2015; Accepted: Oct 9, 2016 http://www.hindawi.com/93834740/ mailto:drkauserabid@gmail.com Journal of Islamabad Medical & Dental College (JIMDC); 2016:5(3):134-138 135 program's theory of collaboration. The objectives and outcomes that could be achieved in the first three years of the project will help in elaborating program activities and strategies, and identifying methods and procedures for monitoring and evaluation. Literature survey strongly recommend its utility as developing a logic model and great way forward to show the feasibility of any program before implementing it. 7,8 In a nutshell it provides an insight into the program resources, planned activities, expected outputs and initial outcomes.  Program Objective: To inculcate philosophical foundation of education, appraise current trends in curriculum development with emphasis on teaching and learning strategies, and update educators on essentials of leadership and management skills.  Rationale: At institutional level professional development does not accompany educational reforms and it is the need of time for teaching the faculty in health profession. The availability and effectiveness of faculty development has been identified as a predictor of the success or failure of reforms initiatives.  Project Settings: These interventional activities took place in the Capacity Building Center; Phase I Islamabad, Pakistan, the first functional unit of NIH&SS Mega purpose built project for the professional development of faculty. Over twenty-nine intensive weeks, participants immersed themselves in the learning process acquiring the skills they need to effectively lead their educational strategies. In a diverse group setting learning environment, they were exposed to experience a range of different teaching and learning techniques from six unique perspectives.  Intervention Activities: Collaborative faculty partners from PHMI and local universities in the education discipline facilitated sessions in the light of best evidence and feedback from literature survey. Guest speakers participated to provide enhanced knowledge in different disciplines. Integrated activities were offered by diverse group of educators from health professional institutes. Logistic support was provided throughout the program for face to face & virtual learning environment. The stake holders provided in time consultancy and support to the program evaluator.  Intervention Targets: The faculty and staff who facilitate the sessions in the FDP, the participating educators of Basic Sciences, Clinical Sciences, Nursing, Social Sciences and Information Technology, Students taught by the participants of the program, Patients whom they care, faculty and committees that deal with curriculum, and educational leadership. Materials & Methods Logic model: A logic model was developed (Fig 1) to share with the organizing committee and the facilitators of the FDP and the outcomes of current program for future improvement strategies.  Study Design: A Case Study approach with nonequivalent control group Quasi-experimental design was used to evaluate the program retrospectively from different contexts likewise how the program operated, what were the inputs and activities that ultimately led to the outcomes. A Pre- Test/Post-Test approach with non-randomization to intervention/comparison group was used to gather data.  Sample Selection Procedures: The target population comprised of participants who were educators serving as faculty in Medicine, Nursing, and Basic Sciences, Social Sciences and Information technology. The comparison group selected was similar to the intervention group on all relevant characteristics and other factors associated with program exposure. Purposive sampling was done as the unit selected had the potential to reveal the reasons to produce the desired outcomes. Number of participants who attended the study was 27 and those who did not participate was 30. A change in educator’s knowledge, skills and attitude was studied as an outcome of the intervention.  Data Collection: The data was collected before and after the FDP as pre and post intervention difference in different domains of professional development from educators of intervention group. From comparison group same information was collected before and after the FDP as pre and post but without exposure to intervention. In data collection mixed method employing both qualitative and quantitative approaches were utilized. The information was collected about the program participants, their perceptions for the program, activities they had, how and where activities were done, and any change in knowledge, attitude and skills occurred as a result of these activities, how much their professional needs met at personal as well as institutional level. Tools for data collection were direct observation, review of documents, and rated questionnaire. A 10 item rated questionnaire on 5 point Likert scale with 5= strongly agree to 1= strongly disagree was developed to get feedback of program participants. Pre and post program questionnaire was used for both intervention and comparison groups on 5 point Likert scale about the impact of program. For comparison group not exposed to intervention disclosing any change was attributed to factors other than the program for their knowledge and skills as educators.  Data Analysis and Interpretation: Data collection and data analysis was done concurrently. For Descriptive statistics SPSS 19 version was used. Univariate and bivariate analysis was done to interpret the rated questionnaire Journal of Islamabad Medical & Dental College (JIMDC); 2016:5(3):134-138 136 Input/Resources Activities Outputs Short term Outcomes Outcomes Outcomes Intermediate Outcomes Long term Outcomes Administration of Capacity development building, Education Deans and educational university consultants. Faculty Partners: from NIH&S, Pakistan and PHMI, USA. Resources: Capacity development building, literature, e- learning supportive class rooms, Educational expertise, experienced facilitators, logistic support, library, supportive leadership, program evaluation office. Funding: Defence Phase I Authority Islamabad and revenue generated from the Program fee Time: participants twice a week sessions to be attended in person, for facilitators to prepare and facilitate sessions and faculty and staff to develop, and implement program. Design instructional approaches to meet the needs of the participants, collaboration with faculty partners from PHMI and local universities in the education discipline, designing and developing sessions and methods in light of best evidence and feedback from literature survey, arranging guest speakers, integrating faculty development for diverse group of educators from health professional institutes, providing logistic support in the face to face& virtual learning environment, providing in time consultancy and support for program evaluators. Scope and sequence of the program, schedule of the three terms with expectations from the participants for each term module, well designed sessions by trained facilitators, NIHS & PHMI faculty, provide consultancy to participants, evaluation results, timely feedback by facilitators and from other teaching faculty. Video recording of micro teaching with feedback from facilitator and peers, Case studies, working in similar and diverse group setting., designing course curriculums with value addition. Participants’ satisfaction with the teaching approaches. Participants practicing educational theories and learned teaching strategies for adult learning in the sessions, participants change in behavior by working in teams, participants identifying educational leader ship concept in different situations, participants level of satisfaction by the gain in knowledge, skill and attitudes as educators Applying educational theories and philosophy in developing different course curriculums in a diverse group setting with multiple input leading to value addition. Developing assessment plans in the light of new educational technology using diverse test items and identifying their strengths and weaknesses, Improvement in teaching methodology constructivist approach and facilitation of small group PBL sessions. Practice competency in different teaching methodologies, designing teaching strategies to address the learning needs of different learning styles students, staying in touch with literature to have best evidence, change in attitude for working in diverse group setting with a spirit of team work, using assessment tools which are reliable and valid, practicing educational leadership and be a source of value addition when designing curriculums. PROGRAM STRUCTURE PROGRAM OUTCOMES CONTEXTUAL FACTORS Diverse participants of FDP: basic medical sciences, Physicians, Nurses, Social Sciences, Information technology. Similar activities started by other agencies, electricity crises, petrol prices going high capacity development building too far away from main city, security issues needing rescheduling of session timings. THE LOGIC MODEL FOR FACULTY DEVELOPMENT PROGRAM Journal of Islamabad Medical & Dental College (JIMDC); 2016:5(3):134-138 137 Results Univariate and bivariate analysis was done. Univariate analysis (Figure 1) showed that 95% of the program participants met their personal professional needs, 95% reported change in attitude in terms of team work in diverse group setting. About 98 % reported a gain in knowledge and skills for designing new teaching strategies, 96 % of participants considered while experiential learning to develop assessment items useful, 100 % of the participants agreed that the hand on activity of curriculum exercise is a proper method to learn curriculum development. The bivariate analysis showed that the interventional educational activities and exercises succeeded to accomplish the program outcomes. The pre- post test questionnaire for intervention and comparison groups showed achievement of program outcomes in the intervention group. For comparison group no change was documented showing that the group not exposed to intervention with no change in the baseline from any other source during the 29 weeks’ period. 92% 93% 94% 95% 96% 97% 98% 99% 100% 101% personal professional needs met team work attitude developed knowledge for new teaching strategies experiential learning of assessment items hands on activity for curriculum Figure 1: Frequency distribution of different variables of FDP Discussion Overall satisfaction with faculty development programs was high. Participants consistently found program, useful and relevant to their objectives. Participants reported positive changes in attitude toward faculty development and teaching. Regarding the faculty development program all the participants have felt its need and considered it as a mandatory part of training for the new entering faculty in teaching profession. In the study done by Ozvaris SB et al, 83 faculty participants expressed a high degree of satisfaction with the content and organization, as well as the course of trainers' teaching. 9 But their study had utilized a test of knowledge related to the course content and participants also performed a microteaching session. In the present faculty development program experiential learning was utilized while Bransford et al reported that the faculty in their study learned what was taught in the workshop, but they were left alone to successfully develop and implement active-learning teaching strategies. 10 The importance of on-site network of expert support in the form of performance feedback is an essential part of any professional development activity. In the NIH&SS program the replays of teaching, with expert’s feedback, were the most powerful tools that were utilized. The study done by Rogan et al suggested that faculty development programs with communities of educators with similar goals can support one another through expert feedback while practicing implementation strategies. 11 McQuiggan CA and later Desselle BC addressed the time constrains at the work place by designing one day workshop for faculty teaching in pediatric residency program. 12,13 The pre and post workshop questionnaire showed increase in self-efficacy and confidence. At personal level several participants believe that such activities to be carried out at institutional level as a comprehensive program instead of patchy efforts of faculty training. With proper training as an educator effectiveness of teaching can benefit the learners as identified by most of the participants. Majority of participants suggested that such activities should be welcomed at working place as value addition in the educational programs. As far as participants views regarding the satisfaction level all have taken it as a knowledgeable practical experience that has given them a chance to work together as a team in diverse group setting. Conclusion Professional development at the workplace for the teaching faculty to have in job training will not only help in time management but also resource management. At institutional level this is the best solution to bring change in teaching attitudes and behaviors among today’s educator of health care profession. Use and Dissemination of Results NIH&SS experiential learning FDP is a bench mark in this region that has provided evidence that professional development at work place can help in resource management and gives a sense of owner ship to the faculty. Such programs should be planned and implemented by the institutions under supervision of Accrediting bodies like Pakistan Medical and Dental Council and Higher Education Commission for enhancing effectiveness of educators for their potential new roles. Conflict of Interest This study has no conflict of interest to declare by any author. References 1. Amin Z, Khoo HE, Chong YS, Tan CH, Goh PS, Samarasekera DD, etal. A multi-institutional survey on faculty development needs, priorities and preferences in medical education in an Asian medical school. Med Educ Online. 2009;14:16 http://www.tandfonline.com/action/doSearch?Contrib=Bahar%5C-Ozvaris%2C+S http://www.ncbi.nlm.nih.gov/pubmed/?term=Desselle%20BC%5Bauth%5D http://www.ncbi.nlm.nih.gov/pubmed?term=Amin%20Z%5BAuthor%5D&cauthor=true&cauthor_uid=20165530 http://www.ncbi.nlm.nih.gov/pubmed?term=Eng%20KH%5BAuthor%5D&cauthor=true&cauthor_uid=20165530 http://www.ncbi.nlm.nih.gov/pubmed?term=Seng%20CY%5BAuthor%5D&cauthor=true&cauthor_uid=20165530 http://www.ncbi.nlm.nih.gov/pubmed?term=Hoon%20TC%5BAuthor%5D&cauthor=true&cauthor_uid=20165530 http://www.ncbi.nlm.nih.gov/pubmed?term=Sun%20GP%5BAuthor%5D&cauthor=true&cauthor_uid=20165530 http://www.ncbi.nlm.nih.gov/pubmed?term=Samarasekera%20DD%5BAuthor%5D&cauthor=true&cauthor_uid=20165530 http://www.ncbi.nlm.nih.gov/pubmed?term=Samarasekera%20DD%5BAuthor%5D&cauthor=true&cauthor_uid=20165530 http://www.ncbi.nlm.nih.gov/pubmed/20165530 Journal of Islamabad Medical & Dental College (JIMDC); 2016:5(3):134-138 138 2. Bransford JD, Brown AL, Cocking RR, editors. How people learn: brain, mind, experience, and school. Washington, DC, US: National Academy Press; 2000. 3. Dennick R. Log-term retention of teaching skills after attending the teaching Improvement project: a longitudinal, self-evaluation study. Med Teach. 2003; 25(3): 314–8. 4. Finucane P, Allery LA, Hayes TM. Comparison of teacher at a traditional and an innovative medical school. Med Educ. 1995;29(2):104-9. 5. Lancaster JW, Stein SM, MacLean LG, Amburgh JV, Persky AM. Faculty development program models to advance teaching and learning within health science programs. Am J Pharm Educ. 2014;78(5):99. 6. Wadhwa A, Das L, Ratnapalan S. Faculty development effectiveness: insights from a program evaluation. J Biomed Educ. 2014; 2014:5. 7. Irby DM, Wilkerson L. Educational innovations in academic medicine and environmental trends. J Gen Intern Med. 2003;18(5):370–6. 8. Hernandez M. Using logic models and program theory to build outcome accountability. Educ Treat Children. 2000;23(1):24–40. 9. Mayeske GW, Lambur MT. How to design better programs: a staff centered stakeholder approach to program logic modeling. Crofton MD: Program Design Institute; 2001. 10. Rogan JM. How much curriculum change is approp riate? defining a zone of feasible innovation. Sci Educ. 2007;91(3):439–60. 11. Bahar-Ozvaris S, Aslan D, Sahin-Hodoglugil N, Sayek I. Faculty development program evaluation: from needs assessment to long-term effects, of the teaching skills improvement program. Teach Learn Med. 2004;16(4):368- 75 12. McQuiggan CA. Faculty development for online teaching as a catalyst for change. J Asynchronous Learn Netw. 2012;16(2):27-61 13. Desselle BC, English R, Hescock G, Hauser A, Roy M, Yang T, etal. Evaluation of a faculty development program aimed at increasing residents' active learning in lectures. J Grad Med Educ. 2012;4(4):516–20. Authorship Contribution: Author 1: Conception, planning, active participation in research, analysis of results and final review. Author 2: Conception, planning, analysis and interpretation of results, discussion and final review. Author 3: Analysis and interpretation of results, discussion and final review. http://www.ncbi.nlm.nih.gov/pubmed/?term=Lancaster%20JW%5Bauth%5D http://www.ncbi.nlm.nih.gov/pubmed/?term=Stein%20SM%5Bauth%5D http://www.ncbi.nlm.nih.gov/pubmed/?term=MacLean%20LG%5Bauth%5D http://www.ncbi.nlm.nih.gov/pubmed/?term=Van%20Amburgh%20J%5Bauth%5D http://www.ncbi.nlm.nih.gov/pubmed/?term=Persky%20AM%5Bauth%5D http://www.hindawi.com/93834740/ http://www.hindawi.com/25694309/ http://www.hindawi.com/73501759/ http://www.sciencedirect.com/science/article/pii/S0149718914000354#bib0055 http://www.sciencedirect.com/science/article/pii/S0149718914000354#bib0085 http://www.tandfonline.com/action/doSearch?Contrib=Bahar%5C-Ozvaris%2C+S http://www.tandfonline.com/action/doSearch?Contrib=Aslan%2C+D http://www.tandfonline.com/action/doSearch?Contrib=Sahin%5C-Hodoglugil%2C+N http://www.tandfonline.com/action/doSearch?Contrib=Sayek%2C+I http://www.ncbi.nlm.nih.gov/pubmed/?term=Desselle%20BC%5Bauth%5D http://www.ncbi.nlm.nih.gov/pubmed/?term=English%20R%5Bauth%5D http://www.ncbi.nlm.nih.gov/pubmed/?term=Hescock%20G%5Bauth%5D http://www.ncbi.nlm.nih.gov/pubmed/?term=Hauser%20A%5Bauth%5D http://www.ncbi.nlm.nih.gov/pubmed/?term=Roy%20M%5Bauth%5D http://www.ncbi.nlm.nih.gov/pubmed/?term=Yang%20T%5Bauth%5D http://www.ncbi.nlm.nih.gov/pubmed/?term=Chauvin%20SW%5Bauth%5D