148 October 2016, Vol. 8, No. 2 AJHPE Research Since the 1990s, mentorship programmes as a means to support student doctors have become increasingly common.[1,2].Although Levison et al. [3] described mentoring as a voluntary relationship between a more experienced professional and an apprentice, many of the mentorship programmes for student doctors are strongly encouraged or compulsory.[2] Unlike informal mentoring, which attracts high achievers, participants in formal mentorship programmes are more representative of the general student population.[4,5] Therefore, formal mentorship programmes ensure that a broader range of student doctors become mentees, who have the opportunity to discuss career planning, work-life balance, personal problems, and study skills with their mentors.[5,6] Mentorship of student doctors in a community of practice is more likely to transpire when there is a rapport between the mentor and the mentee.[7,8] Mentors and student doctors who have experienced the same curriculum find it easier to build a rapport.[8] Rapport is also improved when the demographic profile, personality traits, experiences, and personal and professional interests of the mentor correspond to those of the mentee.[5,9] This has led to the use of online matching systems that recommend poten- tial faculty mentors to student doctors.[10,11] Online systems are also used when mentors in developing countries are unavailable.[12] The appraisal process at the University of Leeds, UK is a formal process for mentorship of a large cohort.[2,6,13] It starts with an appraisal scheduled in the academic calendar, where faculty members guide 1st- and 2nd-year student doctors to reflect on their progress.[6,13] At the end of the appraisal, students fill in an appraisal record form where they state their personal goals, which will be discussed at the second appraisal meeting during the following year. Students who need additional support are referred to the dean of students.[13] The University of the Witwatersrand, Johannesburg, South Africa first implemented an adapted appraisal in 2008 for 3rd-year students. The 3rd year (the Graduate Entry Medical Programme 1 (GEMP1)) is a particularly crucial time for student doctors, as they make a transition to the hospital and shadow doctors. Graduate entrants also have to adjust to a new degree and in some cases to a new university. It was important to investigate the students’ perceptions of the appraisal process, as an adapted appraisal, which uses fewer resources, has not been researched. Furthermore, current research suggests that only assigned mentors who receive training and support manage to develop a rapport with student doctors.[13,14] The purpose of this research was to understand students’ experiences of the adapted appraisal in order to provide suggestions to other medical schools hoping to establish similar programmes with restricted resources. Therefore, the research questions for this study are: • What are students’ experiences of an adapted appraisal that uses fewer resources? • How can students’ feedback on the challenges faced during the appraisal process and the literature be merged to produce research-efficient solu- tions to these challenges? Unlike the University of Leeds, the University of the Witwatersrand does not have the time or the personnel to schedule appraisal days during term time. While the University of the Witwatersrand’s student support officer and staff from the Centre for Health Sciences Education (CHSE) co-ordinate the process, students and appraisers are responsible for arranging the initial appraisal and follow-up meetings. In 2011, the appraisal process began with a brief information session for the appraisers; approximately half of the 32 appraisers, all of whom participated in this process voluntarily, attended the information session, which was facilitated by Dr L Green-Thompson, one of the staff members who originally initiated the appraisal, and the student support officer. As no follow-up appraisal was scheduled, students did not fill in an appraisal record form. Instead, they were asked to complete a form before the appraisal as an aid to reflection on work-life balance, study habits, and career choice, which each appraiser then used to engage students in a 30-minute conversation. Appraisers who believed that students required additional support referred them to the student support office. Background. An appraisal model, a type of formal mentorship programme for a cohort of student doctors, is used at the University of Leeds, UK. The University of the Witwatersrand, Johannesburg, South Africa implemented an adapted version of the appraisal process that uses fewer resources. Objective. To explore students’ experiences of the appraisal process in order to provide information to medical schools with limited resources, which seek to develop or refine their own mentorship processes. Methods. A questionnaire containing Likert-type scales and open-ended questions was distributed to students. Students’ responses were analysed using descriptive statistics and content analysis. Results. Eighty-seven percent of the students had met with their appraiser once, and only 36% felt that they had built a rapport with their appraiser. Students were more willing to discuss academic problems (87%) and less willing to discuss personal matters (51%) with their appraiser. Conclusion. Despite failing to build a rapport with their appraisers, students indicated that the chance to discuss academic and personal problems, and their appraisers’ advice on study and career matters, had been beneficial. To improve the rapport between students and appraisers a number of suggestions that require few or no additional resources are made. Afr J Health Professions Educ 2016;8(2):148-151. DOI:10.7196/AJHPE.2016.v8i2.526 Student feedback on an adapted appraisal model in resource-limited settings L Arnold, BA, MSc Postgraduate Centre, University of Johannesburg, South Africa Corresponding author: L Arnold (larnold@uj.ac.za) This open-access article is distributed under Creative Commons licence CC-BY-NC 4.0. October 2016, Vol. 8, No. 2 AJHPE 149 Research The 17 appraisers who were university staff conducted appraisals in their offices, while the clinicians convened appraisals at teaching hospitals. The form students filled in before the appraisal had a space for completing details of a follow-up meeting; students could also contact their appraiser to suggest another meeting in 2011. While the appraisal at the University of Leeds took place once each year for 2 years, it was envisaged that students at the University of the Witwatersrand, especially those who were struggling with personal or academic problems, would visit their appraiser more often. If students had not met their appraiser by the end of the first quarter, an inquiry by the student support officer followed to ensure that students met with their appraiser at least once. Methods Data instrument The questionnaire contained open-ended questions and close-ended statements. The students responded to the statements using a 5-point Likert-type scale, where options ranged from ‘strongly agree’ (1) to ‘strongly disagree’ (5). The statements were based on the questionnaire used by Murdoch-Eaton and Levene,[13] but a few of these statements were adapted for the local context. For instance, statements referring to student-set goals were removed, as in the adapted appraisal follow-up meetings where these goals could be discussed were not prearranged. Open-ended questions were added in consultation with CHSE staff, including Prof. D Manning and Dr L Green-Thompson, who initiated the appraisal in 2008. These questions were added so that students could share their individual feedback on the appraisal process. Ethics Ethical approval for this study (ref. no. M111187) was obtained from the University of the Witwatersrand’s Human Research Ethics Committee. Data collection In 2012 the questionnaire was distributed to all former GEMP1 students during a problem-based learning tutorial. Students could then choose to complete the anonymous questionnaire and place it in a box outside their classrooms. This method of data collection was used so that students would not feel obligated to complete the questionnaire in the researcher’s presence. Sample size Of 296 former GEMP1 students at the University of the Witwatersrand, 93 (31%) completed the questionnaire. Data analysis Responses to the close-ended questions were typed into an Excel spreadsheet before the percentages of students who strongly agreed and agreed with each statement were calculated. As very little research is available on the appraisal process, an induc- tive approach to thematic analysis, where the coding of data into themes and subthemes without reference to a prior coding framework, was used. The data were typed into Excel verbatim. After examining the data, an initial list of codes that emerged from the data for each question was generated before related codes were arranged into organising themes and subthemes.[15] After a review of the themes and subthemes the researcher asked a colleague, Prof. P McInerney, who is well acquainted with the appraisal process, whether the themes and subthemes could be used to explain the raw data. The researcher then clarified the relationship between themes and subthemes through a ‘thematic network’.[15] After an examination of these thematic networks only the themes that were able to serve as an organising principle for a set of general ideas, with- out becoming repetitive or vague, were accepted.[15] Once the essence of each theme was understood, a name that described the organising principle behind each theme was chosen. Where the researcher had difficulty finalising the thematic networks or choosing an appropri- ate name for a theme, the advice of Ms A Magida, a researcher at the CHSE, was sought. Results Quantitative results Ninety-three (31%) of the 239 students completed the questionnaire. The majority (81) of the students had met with their appraiser once, and the remaining 12 students had met with their appraiser two to four times. Two of the 93 students did not complete the remaining questions and were removed from the dataset. The responses of the remaining 91 students for the six statements that were focused on the benefits of the appraisal process are shown in Fig. 1. A high proportion of students agreed that they were comfortable expressing their opinions (92%) with their appraiser, or able to discuss academic matters (87%). Slightly more than half of the students (51%) agreed that they could discuss personal matters with the appraiser. Less than half of students agreed that they had built a rapport with their appraiser (36%), changed their attitude towards their study habits (49%), or changed their attitude towards their career (40%). The responses to the four statements on the administration of the appraisal process are shown in Fig. 2. Most of the students (81%) agreed that there was enough time for the appraisal discussion, and that the appraisal 92 87 51 36 49 40 100 90 80 0 70 60 50 40 30 20 10 St u d en ts ' r es p o n se s: 's tr o n g ly a g re e' a n d 'a g re e' , % I w as co m fo rta bl e t alk in g/ ex pr es sin g m y o pi ni on s I w as ab le to d isc us s m y ac ad em ic pr ob lem s a nd is su es I w as ab le to d isc us s pe rso na l m at te rs I fe el th at I d ev elo pe d a ra pp or t w ith m y a pp ra ise r Ap pr ais er ’s c om m en ts ha ve ca us ed m e t o alt er m y a tti tu de s t ow ar ds m y s tu dy h ab its Ap pr ais er ’s c om m en ts ha ve ca us ed m e t o alt er m y a tti tu de s t ow ar ds m y c ar ee r Fig. 1. Perceived benefits of participating in the appraisal process. 150 October 2016, Vol. 8, No. 2 AJHPE Research process had gone well (69%). Fewer students (64%) agreed that they were well informed about the objectives of the appraisal and that the topics in the appraisal form were relevant. Qualitative results Answers to the open-ended questions were grouped into two themes: ‘Benefits of taking part in this process’, and ‘The way forward for the appraisal process’ (Tables 1 and 2). Discussion Only 36% of the students believed they had built a rapport with their appraiser. This lack of rapport could have stemmed from the fact that 87% of the students had met with their appraiser once, as evidenced by comments such as: ‘We only met once. There wasn’t much of a relationship.’ This lack of rapport also meant that students were less likely to change their attitudes towards their career or study habits after an appraisal, as shown by one student’s comment: ‘A stranger’s opinion of my marks makes no difference to my study ethic.’ However, one student commented that his/her study style was now focused less on his/her grades and more on understanding the work, and another wrote that he/ she had received ‘good advice from a more senior professional especially as I have no family members in the medical profession to discuss future opportunities with’. Only 51% of students felt comfortable talking about personal problems with their appraiser. This means that an appraiser may not know when a student needs to be referred for counselling. Although the following comment, ‘Feeling uncomfortable to discuss personal matter [sic] with the appraiser but having realised the appraisers help that I might be in deep trouble, I went to see a psychologist and that helped me a lot,’ shows that an appraisal can help students realise that they need counselling. While students were reluctant to discuss personal matters with their appraisers, certain appraisers were commended for the ‘genuine’ concern that they had shown towards their students, and their ability to give students ‘a voice to express how I felt’. Moreover, some students felt ‘reassured’ by the feeling that their appraiser was someone whom they could turn to if they experienced any problems in future. In light of previous findings that a shared curriculum helps to build rapport,[8] it is not surprising that students who were assigned to a former GEMP1 student said that their appraiser understood their situation. As the appraisal was a reflective process, comments such as: ‘It keeps you in check knowing that someone is monitoring your progress,’ were unexpected. Although students filled in their grades on the form before the appraisal, the appraiser had no access to their actual marks. A potential concern is that students could feel less receptive to discuss their problems with their appraiser if they mistakenly believe that they are being monitored. The value that students placed on the appraisal varied widely − while some suggested discontinuing or ‘scrapping’ the process altogether, others requested compulsory appraisal meetings scheduled in the timetable. Some students resented being urged to attend the appraisal, and felt that they should decide if the appraisal would benefit them or that it was only necessary for struggling students, while others lamented that ‘I should have gone more frequently’. Conclusions and recommendations Although it was expected that students who struggled with academic or personal problems would meet with their appraisers more than once, 87% of students had only met their appraiser once. This created a situation where: (i) only 36% of students felt they had developed a rapport with their appraiser; and (ii) students were reluctant to discuss personal matters with their appraiser or take their appraiser’s advice. Despite these negative findings, as with student doctors in other mentorship programmes, these students received advice on academic and personal problems.[6] A few students incorrectly assumed that the appraisers were there to monitor their progress. While these students felt more motivated by the thought of someone monitoring their progress, it can be argued that this Table 1. Benefits of taking part in the appraisal process Theme Category No benefits No relationship Not needed Unwilling to take advice Academic Study tips Encouragement after poor marks An accountability partner Career advice from someone in the profession Personal My appraiser understood what I was going through He/she was genuinely concerned about me A voice to express how I felt A person I can approach in future Table 2. The way forward for the appraisal process Theme Category Way forward Scrap it It should be voluntary Compulsory process for some Students should take more responsibility Administrator should play a more central role 81 69 64 64 90 80 0 70 60 50 40 30 20 10 St u d en ts ' r es p o n se s: 's tr o n g ly a g re e' a n d 'a g re e' , % Th er e w as su � cie nt ti m e f or th e ap pr ais al di sc us sio n Th e a pp ra isa l p ro ce ss w en t w ell I w as w ell in fo rm ed ab ou t t he ob jec tiv es of th e a pp ra isa l Th e t op ics in th e a pp ra isa l fo rm we re re lev an t Fig. 2. Feedback on the administration of the appraisal. October 2016, Vol. 8, No. 2 AJHPE 151 Research may impede them from developing a rapport with their appraiser. Before future appraisals, faculty should stress the self-reflective nature of the appraisal so that students understand the appraiser’s role more clearly. Students’ feedback on the form is completed before the appraisal and could be used to improve the form’s perceived relevance. While the university cannot allocate a significant amount of additional resources to select, or train and support faculty mentors, it is conceivable that the student support officer could be trained to manage this responsibility more effectively, or that their workload could be reduced during the appraisal period so that they are able to schedule meetings between appraisers and students. While accessing additional funds for a scheduled appraisal (e.g. as at the University of Leeds) is unlikely, funds could be used to develop an online system that matches appraisers and students. If this system were designed to facilitate long-distance mentoring, the pool of appraisers could be expanded. These measures could help to strengthen the relationship between the appraiser and the student. Limitations of the study and directions for future research This study was retrospective; only students who successfully completed the year at the end of 2011 received the questionnaire. If students had received the questionnaire in 2011, the views of students who would fail or drop out of the course would have been included. The viewpoints of the appraisers could also have added a different perspective to this study. Acknowledgements. The author would like to thank Prof. D Manning and Dr L Green-Thompson, who initiated the appraisal process and have continued to establish, maintain and refine the process to what it currently entails. I also wish to thank Prof. P McInerney and Ms A Magida for their assistance with the content analysis, and Dr P Lamberti for her editorial support. References 1. Frei E, Stamm M, Buddeberg-Fischer B. Mentoring programmes for medical students. A review of the PubMed literature 2000 - 2008. BMC Med Educ 2010;10(32):1-14. DOI:10.1186/1472-6920-10-32 2. Mann MP. Faculty mentors for medical students: A critical review. Med Teach 1992;14(4):311-319. DOI:10.3109/01421599209018849 3. Levison DJ, Darrow CN, Klein CG, Levison MH, Mckee B. The Seasons of a Man’s Life. New York: Knopf, 1978. 4. Kman NE, Bernard AW, Khandelwal S, Nagel RW, Martin DR. A tiered mentorship program improves number of students with an identified mentor. Teach Learn Med 2013;25(4):319-325. DOI:10.1080/10401334.2013.827976 5. Dimitriadis K, von der Borch P, Störmann S, et al. Characteristics of mentoring relationships formed by medical students and faculty. Med Educ Online 2012;17(10):1-12. DOI:10.3402/meo.v17i0.17242 6. Murdoch-Eaton DG, Pell G, Roberts T. Changing approach to undergraduate studies documented during annual appraisal of medical students. Med Teach 2007;29(2-3):111-118. DOI:10.1080/01421590601178006 7. Ramanan RA, Taylor WC, Davis RB, Phillips RS. Mentoring matters. Mentoring and career preparation in internal medicine residency training. J Gen Intern Med 2006;21(4):340-345. DOI:10.1111/j.1525-1497.2006.00346.x 8. McLean M. Does the curriculum matter in peer mentoring? From mentee to mentor in problem‐based learning: A unique case study. Mentoring Tutoring: Partnership Learn 2004;12(2):173-186. DOI:10.1080/1361126042000239929 9. Seal K, Mutha S. Enhancing faculty mentoring of medical students. Teach Learn Med 1996;8(3):174-178. DOI:10.1080/10401339609539791 10. Von der Borch P, Dimitriadis K, Störmann S, et al. A novel large-scale mentoring program for medical students based on a quantitative and qualitative needs analysis. GMS Z Med Ausbild 2011;28(2):1-11. DOI:10.3205/zma000738 11. Störmann S, von der Borch P, Dimitriadis K. Online matchmaking enables large-scale individual mentoring. Med Educ 2010;44(5):492-493. DOI:10.1111/j.1365-2923.2010.03663.x 12. Mbuagbaw L, Thabane L. How to set up a long-distance mentoring program: A framework and case description of mentorship in HIV clinical trials. J Multidiscip Healthc 2013;6:17-23. DOI:10.2147/JMDH.S397 13. Murdoch-Eaton DG, Levene MI. Formal appraisal of undergraduate medical students: Is it worth the effort? Med Teach 2004;26(1):28-32. DOI:10.1080/0142159032000150502 14. Allen TD, Eby LT, Lentz E. Mentorship behaviours and mentorship quality associated with formal mentoring programs: Closing the gap between research and practice. J Appl Psychol 2006;91(3):567-578. DOI:10.1037/0021-9010.91.3.567 15. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006;3(2):77-101. DOI:10.1191/ 1478088706qp063oa dx.doi.org/10.1186/1472-6920-10-32 http://dx.doi.org/10.3109/01421599209018849 http://dx.doi.org/10.1080/10401334.2013.827976 http://dx.doi.org/10.3402/meo.v17i0.17242 http://dx.doi.org/10.1080/01421590601178006 http://dx.doi.org/10.1111/j.1525-1497.2006.00346.x dx.doi.org/10.1080/1361126042000239929 http://dx.doi.org/10.1080/10401339609539791 http://dx.doi.org/10.3205/zma000738 http://dx.doi.org/10.1111/j.1365-2923.2010.03663.x http://dx.doi.org/10.2147/JMDH.S397 http://dx.doi.org/10.1080/0142159032000150502 http://dx.doi.org/10.1037/0021-9010.91.3.567 http://dx.doi.org/10.1191/­1478088706qp063oa http://dx.doi.org/10.1191/­1478088706qp063oa