A CLINICAL SUPERVISION AND PEER CONSULTATION APPLICATION IN PROBLEM BASED LEARNING PROCESS: Şenol, Y. & Bilgin Aksu, M. (2015). A clinical supervision and peer consultation practice in problem based learning process. International Online Journal of Education and Teaching (IOJET), 2(1). 2-10. http://iojet.org/index.php/IOJET/article/view/71/101 A CLINICAL SUPERVISION AND PEER CONSULTATION PRACTICE IN PROBLEM BASED LEARNING PROCESS Yeşim Şenol Akdeniz University yysenol@hotmail.com Mualla Bilgin Aksu Akdeniz University muallaaksu@akdeniz.edu.tr Biodata Yeşim ŞENOL is an associate professor of Medical Education at the University of Akdeniz in Antalya, Turkey. Her research interest include standardized patient, communication skills, program development and evaluation in medical education. Mualla BİLGİN AKSU is Professor of Educational Sciences in the Faculty of Education, as well as the Head of Educational Administration and Supervision Program for the Graduate Institute of Educational Sciences at the University of Akdeniz in Antalya, Turkey. Her research interests include job stress, clinical supervision, peer coaching, strategic planning and TQM, teacher leadership, ethical leadership, educational leadership and policies, novice teachers, and theory - practice relations. Copyright by Informascope. Material published and so copyrighted may not be published elsewhere without the written permission of IOJET. mailto:yysenol@hotmail.com mailto:muallaaksu@akdeniz.edu.tr International Online Journal of Education and Teaching (IOJET), 2(1). 2-10. 2 A CLINICAL SUPERVISION AND PEER CONSULTATION PRACTICE IN PROBLEM BASED LEARNING PROCESS Yeşim ŞENOL yysenol@hotmail.com Mualla Bilgin Aksu muallaaksu@akdeniz.edu.tr Abstract To test the effectiveness of clinical supervision as an assessment method in the practices of faculty members attending PBL courses, to determine the effectiveness of the courses and to discuss its conformity for faculties of medicine.Clinical supervision and peer consultation methods were used to evaluate the performance of the 34 faculty members who took charge in Problem Based Learning modules. The students, the faculty members themselves, the clinical supervisor and the peer consultants participated in the evaluation.The satisfactory and underdeveloped skills of the faculty members, which they need throughout the implementation steps of PBL, were determined. Overall, the faculty members included in the study were found to be successful. As for the quantitative results, they indicate the faculty members’ opinion that the method may ensure standardization and improve the quality of the education. However, there are also some faculty members who are dissatisfied with the method and think that it is time-consuming.The obtained data suggest that peer consultation and clinical supervision can be applied in medical schools and may help faculty members develop their professional skills. Keywords: Medical education, clinical supervision, problem based learning, faculty member 1.Introduction Faculty development is accepted as a significant component of medical education for an effective teacher. New approaches to learning and teaching techniques and requirement of the faculty members to education strategies make the organization of such programmes necessary. Faculty development programmes are used to assist teacher role of the faculty member, to supplement their deficiencies and to increase their performance (Steinert et al. 2006). Although numerous publications concerning faculty development programmes exist, efficiency of the programmes is not searched adequately (Steinert 2000). Most of the researches conducted in this field base on the inclination or satisfaction of the participant. Most frequently used assessment methods are end of session assessment, monitoring surveys, assessments before and after the course which measure knowledge and behavior changes, direct observation of teaching behavior, student assessment concerning the course performance and self-inclination of the faculty member (Steinert et al. 2006). In this field, it is necessary to make assessment more carefully, and to take into consideration of other assessment methods since the faculty development has started. In addition, the assessment focus must be extended to provide the behavioural change in actual life. Clinical supervision practices are performed in educational sciences in order to study the actual life behavior changes of teachers. By clinical supervision, faculty member’s process of mailto:yysenol@hotmail.com mailto:muallaaksu@akdeniz.edu.tr Şenol & Bilgin Aksu 3 executing teaching-learning activities may be assessed in all aspects with intra-class observation, student assessment and video recording (Sergiovanni & Starratt 2002). Supervision in the field of medicine may be mentioned under the names such as clinical supervision, educational supervision, counseling and coaching and various forms. Boundary among these definitions is not very distinct (Launer 2006). “Clinical supervision” is generally used in mental health and nursing fields (Buus & Gonge, 2009) and post-graduate medical practices (Busari & Koot 2007; Cottrell et al. 2002; Sox et al. 1998). A limited number of articles are written on the subject of clinical supervision. There are only a few experimental works. There are some opinions that clinical supervision needs to connect with education strategies including Problem Based Learning (PBL), skills training and mentor system in medical education (Kilminster et al. 2007). Faculty development programmes are applied in Akdeniz University, Faculty of Medicine since late 1990 to increase the educational activities of faculty members. Satisfaction surveys are obtained from students and faculty members in the assessment of the effectiveness of the programmes. Up to date, any assessment or supervision process is not performed to assess behaviour changes. The purpose of this study, conducted for the first time in Turkey in a medicine faculty, is to test the effectiveness of clinical supervision as an assessment method in the practices of faculty members attending PBL courses, to determine the effectiveness of the courses and to discuss its conformity for faculties of medicine. In order to reach these targets, assessment of educational competency of the faculty members serving as education moderator in PBL practices involved in Akdeniz University Faculty of Medicine by clinical supervision and peer consultation method and determination of opinions related to this assessment method is aimed. Problem sentence of the research: What is the educational competency level of the tutor according to the clinical supervision and peer consultancy assessment of the faculty members serving in PBLs of Akdeniz University Faculty of Medicine? As a result, the following questions were advanced to guide this inquiry: 1. Is there a significant difference among the opinion of students, faculty members, clinical supervisor and peer consultants in PBL process related to educational competency of faculty members? 2. What is the assessment of attending faculty members on the clinical supervision and peer consultancy programme applied in PBL in Akdeniz University Faculty of Medicine? 3. What are the feelings and opinions of the faculty members before and after clinical supervision and peer consultancy practice in PBL sessions? 4. What are the opinions of the faculty members on the advantages and disadvantages of clinical supervision? 2.Methods The research is designed as a cross-sectional study where quantitative and qualitative methods are employed together. 2.1.Study population and setting In this research, clinical supervision and peer consultation practice was made for 34 faculty members serving in PBL module involved in Akdeniz University Faculty of Medicine in 2008-2009 academic year. 380 students have attended PBL sessions within this process. Students, faculty members themselves, clinical supervisor and peer consultants have participated in the assessment process. Peer consultants were involved with the purpose of controlling the clinical supervisors. Peer consultants were selected from among the faculty members trained on PBL and who wished to be an observer. Clinical supervisors are International Online Journal of Education and Teaching (IOJET), 2(1). 2-10. 4 constituted from the faculty members giving PBL training and being a moderator in PBLs at least for 10 times. 2.2.Data Collection Research instrument was developed in order to measure the in-group effectiveness of faculty members. The research instrument that has the same questions for all the participants was developed by the steps of PBL session. . The research instrument consisted of 21 (twenty one) items. The instrument was responded within three options: (1) agree, (2) neutral, and (3) disagree”. Because the number of peer consultants and clinical supervisors were very few, and data obtained from them were similar, the related data for statistical analyses were combined in the name of supervisor. Data were collected from the participants at the end of PBL practices. The instrument used as an assessment form was applied to the students after the faculty member has just left the session to prevent the students from being attracted by the faculty member. . Faculty members have handed their own assessment forms to the researcher in post-PBL meetings after filling them. On the other hand, the clinical supervisor and peer consultant have filled the forms independently while the sessions are ongoing. Qualitative data were collected by structured observation form and faculty member interview methods. Each faculty member and peer consultant were informed about the practices before clinical supervision process; video recording was indicated and they were asked to sign an approval form. Students were informed what is requested from them before the start of sessions on. Peer consultants and clinical supervisors have attended and assessed three sessions and delivered their assessment forms at the end of sessions. Nine students, a clinical supervisor, a peer consultant for each faculty member and the faculty member himself/herself have given feedbacks. A video recording was performed in order to be able to reaccess the practice. Video records were watched within 15 days together with the faculty member, then the feedback of the students, peers and clinical supervisors were shared in their offices. After the process, interviews were made with 34 lecturers, their comments related to the practice were obtained and their feelings during the practice were asked. Interviews have taken for about 30 minutes. The records of interview were typed as a text and approval of the faculty member was obtained after reading the written material. 2.3.Data analysis Data are analyzed in two ways: First, chi-square test was used in the assessment of quantitative data. Second, opinions of the faculty members were transferred to the electronic media as it is and made a text document. Lecturers are randomly numbered from 1 to 34. Coding process was completed by realizing the concepts encountered in any part of the interviews and which reflect the meaning and depth in the best way. Themes were found to explain data in general level with the headings as codes and interview form, and codes required to exist together were assessed and thematic coding was determined. 3.Results Results are given in the following lines within two sub-title: quantitative findings and qualitative findings 3.1.Quantitative findings Statistically significant differences are found among students, faculty members and supervisors in seven of 21 items where PBL steps are monitored. These items are “looking up Şenol & Bilgin Aksu 5 for the unknown words after the scenario is read”, “providing the determination of problems”, “writing the hypotheses”, “recording the unknown subjects as learning target”, “feedback of tutor concerning the process and practice”, “asking questions in relevant intensity and locations”, “case of limiting learning targets by tutors” (Table 1). Table 1. Ratio of “I agree” replies of evaluators to PBL steps (see p. 11) 3.2.Qualitative findings Findings are given under two major themes: (1) feelings and thoughts of faculty members before the practice, and (2) their opinions related to clinical supervision after the practice. 3.2.1.Feelings and thoughts of faculty members before the practice Faculty members have indicated that the practice is generally useful, but it causes stress and nervousness. It is observed that the experience of faculty members about PBL practice influences the stress and nerves. It is determined that inexperienced faculty members who have just been trained for PBL feel more nervous than the other counterparts. Opinions of the faculty members before the practice are as follows: “I have attended the course recently. I have forgotten some steps as the practice is made after summer holiday. I have benefited from our pre-practice work. However, I felt nerves since I have been involved in such a practice for the first time.” “A different practice which we are not used to; one necessarily becomes nervous. However, I think it will be useful if it always performs .” On the other hand, faculty members experienced in PBL practice have indicated that they had experienced no stress and nervousness prior to clinical supervision. Opinions of the attendants in their own words are as “I don’t feel nervous. It was not different from the other practices for me.” All faculty members prior to observation have indicated that the practice is useful and the guidance provided by the clinical supervisor contributes them to remember and realize their deficiencies. On the other hand, according to some faculty members, the practice is time consuming. Some considerations of the faculty members on this subject were as follows: “Reminding of all steps prior to practice and emphasizing on them in the meetings have provided us with review of the practice once more. I am satisfied with it. I am satisfied with observing my deficiencies”. “A very time consuming practice. I don’t think I can always spare my time”. 3.2.2. Their opinions related to clinical supervision after the practice. 3.2.2.1.Opinions of faculty members related to PBL practices: One of the assessments of faculty members as tutors related to their competence is “I couldn’t give equal rights to the students for speaking in the PBL periods. Some talked more and some talked less.” 3.2.2.2.Opinions related to video recording, peer consultancy and clinical supervisor Twenty faculty members have indicated that video recording caused both themselves and the students to lose concentration, and irritated the students. Faculty members have expressed their opinions on this subject as follows: “Were we obliged to perform this practice? Was there no way out of video recording?”, “I had difficulty in completing the PBL and felt I am continuously watched. I lost my concentration sometimes”, “I had attended a course five months ago. During my performance, video recording and two evaluators being present made me stressed. Being in the effort to remember the rules, and being watched increased my stress”. International Online Journal of Education and Teaching (IOJET), 2(1). 2-10. 6 On the other hand, satisfied faculty members have indicated that they are used to observers due to their PBL experiences. Therefore, video recording has not influenced them and the students. Opinions of some faculty members indicating their satisfaction are as follows: “I perform this practice for a long time. It is very useful in my opinion. I have understood that I am wrong in some practices. I will try to correct them in next practices.” 3.2.2.3.Opinions of Faculty Members on Advantages of Clinical Supervision Some opinions of faculty members on the advantages of clinical supervision are given below: Faculty members indicate that the practice is useful and should be involved in the programme. Furthermore, it is indicated that, by way of clinical supervision, standardization would be ensured among the faculty members, deficiencies of the faculty members and PBLs would be identified, a good feedback and control mechanism would be provided and that the quality of education would increase. Expressions of the faculty members related to the advantages of clinical supervision are: “As a matter of fact it is very useful, but actually, it is also very difficult to apply it in our country”, “This practice should be involved in our faculty; both feedback and control mechanism for the faculty member. It causes better work”, “Fear of doing wrong made me think that I should do my best. It shall ensure me to obey the rules. Even knowing that there are cameras in the rooms increases the sense of the lecturers’ responsibilities.”, “Practice is necessary in order to provide the quality. I consider that educational quality shall increase and all students get the same information”. 3.2.2.4. Opinions of Faculty Members on Disadvantages of Clinical Supervision Disadvantages related to clinical supervision indicated by the faculty members are as follows: compulsory video recording, a stressful environment, time consuming work. Thoughts of faculty members are given below: “In my opinion it shouldn’t be. We already produce too much work in a stressing environment. We come here in our heavy work by allocating our time. I don’t enjoy being in a stressful environment and being continuously watched”, “Feeling of supervision is not too enjoyable. University must have an independent climate. It may destroy the creativity and independency of the faculty member. We have an intense working tempo. Continuously being watched and scored might reduce the eagerness of the faculty members”, “Very time consuming. I don’t think any lecturers from the faculty of medicine, especially in surgery branches, want to have such effort”. 4.Discussion This study aims at developing teaching skills of the faculty members and supervising their positions in PBL. In our country, programmes for training the faculty members are given great importance in recent years. This programme is given in almost all medical schools and assessment is made by student feedbacks. Student feedbacks are collected at the end of the course, at the end of probation and in general collected by survey method. However, the actions behind the closed doors of the black box called classroom and whether these match with the contents of the programme are not known. On the other hand, in clinical supervision and peer consultation, the faculty member’s process of carrying out teaching – learning activities by in-class observation, student assessment and video recording can be assessed in every aspect (Sergiovanni & Starratt 2002). Therefore, this practice is expected to present useful results especially from the point of education quality. Results of the poll form containing PBL steps applied to the clinical supervisor, peer consultant, student and faculty member himself/herself are common in some items and different in others. These differences may originate from many reasons. In accordance with the nature of PBL, faculty members are obliged to work with little groups. Little group management is known to be much more difficult compared to the management of classical Şenol & Bilgin Aksu 7 class. In our study, students, peer consultant, clinical supervisor and tutors themselves have indicated that they are rather successful in this subject. Only four of the students have indicated that there isn’t a positive educational atmosphere. All assessors have given rather high points to the rule of deciding group rules, which is an important step to ensure educational atmosphere, with the participation of the majority of the group (Newble & Cannon 2000). Faculty members are rather successful in the determination of group rules containing items such as starting time of the sessions, selection of writers and readers, break time and its duration. In the researches (Newble & Cannon 2000), some difficulties often experienced by the students in the subjects of participation in discussions, understanding the tradition of group study and its acceptable behavioral manner, adequate knowledge to participate in the discussion,and assessment, are determined. It is seen in the study that faculty members are considered successful in these fields. Guidance to faculty members in courses and prior to clinical supervision is thought to be useful in this success. Courses are carried out in a structured way and instances related to practice are demonstrated. Tips are given during guidance for clinical supervision in order that tutors create a positive educational atmosphere, and a few warming activities and games are used to provide group dynamics. Students mainly think that hypotheses are written. On the other hand, peer consultant and clinical supervisor indicate that the hypotheses are not adequately written. This may be due to the case that students don’t like to write in the sessions and wish to finish and leave in a short time. Similarly, the students think that mechanisms are adequately explained and they have obtained educational targets. This ratio is smaller in the peer consultant and clinical supervisor. Having full knowledge of the subject, being informed about PBL guides and helping the faculty member prior to clinical supervision may cause the peer consultant and clinical supervisor to see the deficiencies better. It is a desirable property for the clinical supervisors to be specialized on the subject and have knowledge and skills on the subject to be supervised (Kilminster et al. 2007). Peer consultant and clinical supervisor think that the education moderator hasn’t asked a suitable question. High points from students and faculty members to this step may originate from their awareness of inadequate fields which should be developed. It also makes one to consider that guidance is inadequate in these fields. Whereas it is recommended to the faculty member and moderator to have full knowledge of the subjects not involved in the target, to follow up the training programme and to have good knowledge of the practice areas while the subjects that lead to the target are discussed (Cooper 2003; Das et al.& 2002). Another area in which there are differences between evaluators is the step of ensuring attendance of everyone in the sessions. About half of the faculty members have found themselves unsuccessful on this subject. In this field, peer consultant and clinical supervisor have found the faculty member more efficient compared to other evaluators. Faculty members who don’t accept themselves competent are those new faculty members. This competency is considered to improve in time. In terms of self-development of the faculty member in this area, tutors should establish a suitable environment for discussions. Results are used to assist the diagnosis of problems experienced during education rather than judging the faculty member. In this sense, clinical supervisor and peer consultant have undertaken the task of a mirror by clinical supervision to show the faculty member his/her performance. For this purpose, observation and feedback processes are used during this study to assist the faculty members and the specific conflicts between what they have done and they try to do are determined. International Online Journal of Education and Teaching (IOJET), 2(1). 2-10. 8 In the interview made with the faculty member concerning the practice, faculty members have indicated that the practice is exciting and that it has created motivation in the development of their educational skills. Similar results were achieved in previous studies. In the studies, it is found at the end of practices that trust for trying new ideas and techniques increases (Mcmahon & Patton 2000). In Missouri University, faculty members have found this programme suitable for their professional development similar to the results of this study and they have changed their teaching strategies by making use of the assessments (Happner & Johnston 1994). Special attention is paid to cause the faculty members and students to know exactly what would be done in PBL sessions, to want to be there, to be interested in the problems related to the programme for the realization of the aim of the study and this is noted to obtain the desired results. However, although they are informed about the practice, this method applied first in medicine schools has startled the concerned faculty members and students. Unease is realized rather due to video recording. Unease and stress is more widespread among the faculty members who have attended the course, but not managed PBL before. In one to one interviews, faculty members have indicated that clinical supervision is stressful and exciting, and especially the use of video increases excitement. Similar results in other studies measuring the in-class efficiency of teachers attract attention. It is found that teachers experience excitement in video usage which captures the feeling of class environment. It is indicated that course tools installed before the start of lecture may be useful for this problem (Acheson & Gall 1997) and a custom may be provided with the increase in the practice. Faculty members have indicated that they became aware of their inadequate aspects with the feedback of the clinical supervision and the peer consultation. In the previous interviews, faculty members have also indicated some thoughts that support the results obtained from the evaluation forms of the students, clinical supervisors and peer consultantsIn the last part of the instrument used for the qualitative data, there are opinions of the faculty members related to the advantages and disadvantages of the clinical supervision and its practice in medical schools. during the clinical supervision, it is considered that faculty members will take their tasks seriously, standardization may be provided in education and education quality will be increased. However, there are faculty members that are not satisfied with this practice and that considered it a time consuming method. The reason for this thought may be the multitask of faculty members of medical schools such as teaching, research study, and health service. 5.Conclusion This study reveals both qualitative and quantitative effects of peer consultation and clinical supervision practices on faculty members in medical school. The results of the study exposes that peer consultation and clinical supervision have significant contributions to the professional skills of the faculty members. In addition, it is considered that clinical supervision practice by faculty members would assist to increase the quality of medical education. Şenol & Bilgin Aksu 9 References Acheson, K.A., Gall, M.D. (1997) Techniques in the Clinical Supervision of Teachers, Peer consultation. (Newyork, Longman). Busari, J.O., Koot, BG. (2007) Quality of clinical supervision as perceived by attending doctors in university and district teaching hospital. Medical Education, 41, pp.957- 964. Buus, N., Gonge, H. (2009) Empirical studies of clinical supervision in psychiatric nursing: a systematic literature review and methodological critique. International Journal of Mental Health Nursing, 18, pp. 250-264. Cooper, M.H. (2003) An Exploration of tutors’ experiences of facilitating Problem- Based Learning, Part 2- implications for the facilitation of Problem-Based Learning, Nurse Education Today, 23, pp. 65-75. Cottrell, D., Kilminster, S., Jooly, B., Grant, J. (2002) What is effective supervision and how does it happen? A critical incident study. Medical Education, 36, pp.1042-1049. Das, M., Mpofu, D.J.S., Hasan, M.Y., Steward, T.S. (2002) Student perceptions of tutor skills in problem-based learning tutorials. Medical Education, 36, pp. 272-278. Heppner, P.P., Johnston, J.A. (1994) Peer consultation: faculty and students working together to improve teaching. Journal of Counseling & Development, 72, pp. 492- 494. Kilminster, S., Cottrell, D., Grant, J., Jolly, B. (2007) AMEE Guide No.27: Effective educational and clinical supervision, Medical Teacher, 29, pp. 2-19. Launer, J. (2006) Supervision, Mentoring And Coaching: One- To One Learning Encounters in Medical Education, Association for The Study Of Medical Education. McMahon, M., Patton, W. (2000) Conversations on clinical supervision: benefits perceived by school counsellors. British Journal of Guidance and Counselling, 28, pp. 339-351. Newble, D., Cannon, R.A. (2000) A Handbook for Medical Teachers. (Kluwer academic publication). Sergiovanni, T., Starratt, R. (2002) Supervision: A Redefinition (New York, McGraw- Hill). Sox, C.M., Burstin, H.R., Orav, E.J., Conn, A., Setnick, G., Rucker, D.W., Dasse, P., Brennan, T.A. (1998) the effect of supervision of residents on quality of care in five university affiliated Emergency Department. Academic Medicine, 73, pp. 776-82. Steinert, Y. (2000) Faculty development in the new millennium: key challenges and future directions. Medical Teacher, 22, pp. 44-50 Steinert, Y., Mann, K., Centono, A., Dolmans, D., Spencer, J., Gelula, M., Prideaux, D. (2006) A systematic review of faculty development initiatives designed to improve teaching effectiveness in medical education: BEME Guide no:8. Medical Teacher, 28, pp. 497-526. International Online Journal of Education and Teaching (IOJET), 2(1). 2-10. 10 FINDINGS Table 1: Ratio of “I agree” replies of evaluators to PBL steps student himself/herself supervisor Χ2 p Tutor has created a positive education environment 284 94.6 32 94.1 68 100.0 4.59 0.321 Decision of group rules with the majority of group 291 97.0 33 97.1 60 88.2 10.58 0.316 Looking up for the unknown words after the scenario is read 243 81.0 30 88.2 31 45.5* 49.90 0.000 Providing the determination of problems 274 91.3* 20 88.2 55 80.8 28.84 0.000 Determination of causes leading to problems by brain storming 265 88.4 29 85.3 59 86.8 3.91 0.417 Writing the hypotheses 254 84.7 25 73.5 46 67.6* 15.12 0.004 Explanation of hypotheses’ mechanisms 257 85.7* 27 78.4 48 70.6 10.07 0.039 Recording the unknown subjects as learning target 277 92.3 30 88.2 58 85.3 10.93 0.027 Writing the surveys first requested 230 78.0 30 85.3 55 80.9 1.88 0.753 Distribution of source list 263 87.6 30 88.2 60 88.2 2.08 0.720 Feedback of education moderator and students concerning the process and application 259 86.4 33 97.1* 57 83.8 12.16 0.016 Providing the presentation of education targets by the students at the beginning of second and third session 283 94.3 33 97.1 60 88.3 4.74 0.314 Providing the narrating of subjects considered to be imperfectly narrated by asking questions 265 88.3 33 97.1 56 82.4 6.74 0.149 Providing reach to diagnosis 258 86.0 33 97.1 64 94.1 6.99 0.316 Causing to summarize at the end of sessions 214 71.3 31 91.2 51 75.0 6.51 0.164 Facilitative and orienting function undertaken by tutor 270 90.0 32 94.1 63 92.6 3.22 0.522 Asking questions in relevant intensity and locations 267 89.0 29 85.3 47 69.1* 20.02 0.000 Ensuring attendance of everyone in session 236 78.6 24 70.6 55 80.9 9.28 0.054 Positive tutor behavior 287 95.7 32 94.1 66 97.1 4.73 0.315 Education moderator not informing in the process of sessions 156 52.0 24 70.6 44 64.7 14.72 0.053 Case of limiting learning targets by tutor 261 87.0* 18 53.0 42 61.8 39.36 0.000 *:p<0.05