Research 78 June 2017, Vol. 9, No. 2 AJHPE Reflective portfolios are a collection of evidence that attests to achievement as well as personal and professional development through critical analysis and self-reflection.[1] Professional portfolios may be required for purposes such as seeking promotion and documenting continuing professional development, and for accreditation. In health professional education, portfolios have been used to support reflective practice, summative assessment, critical thinking, self-growth, learning and professionalism. In addition, they have been effective in aiding knowledge management processes and in bridging the gap between learning and practice.[2-4] Perceived benefits to students include greater learning autonomy and heightened understanding of exit learning outcomes.[5] Although portfolios have demonstrated positive effects in undergraduate programmes, evaluations of its use in postgraduate education have shown mixed results.[6] There is limited description of the contribution of portfolios to postgraduate health professional education, particularly within specialised public health programmes, such as maternal and child health (MCH) and in African and South African (SA) settings. The Master’s degree in Child Health (MSc) and Master’s degree in Public Health (MPH (MCH)) at the University of the Witwatersrand, Johannesburg, SA incorporated reflective portfolios as a course activity and assessment tool in 2010. We conducted this study to analyse the contribution of academic portfolios in the development of students’ reflective and critical thinking abilities, and its utility in assisting faculty to monitor and influence students’ learning and attainment of key competencies. Methods The University of the Witwatersrand offers two part-time 2-year Master’s programmes related to MCH; one in Child Health (MSc) and another in MCH (MPH). Both programmes are designed to prepare health professionals working in government departments, non-profit organisations, the private sector and at academic institutions for leadership positions in the field of MCH. A combination of lectures, student-led seminars and discussions, online learning, group work sessions and site visits are employed across the 8 - 12-course modules to achieve course competencies. In 2010, portfolios were introduced as an additional strategy for students to reflect on their learning experiences. Students are encouraged to review their engagement with course material and activities, and to showcase their academic, professional and personal growth. Creative presentation of ideas and learning constructs is promoted. Students submit an individual portfolio after each course module and an integrated portfolio at the end of each of the 2 years of study. Each portfolio is marked by two or three staff members to enhance reliability. A marking rubric is used to provide grades and feedback to students (Appendix 1). This study involved a qualitative, retrospective record review of postgraduate student portfolios submitted as part of their MSc/MPH coursework. We analysed all 35 integrated portfolios available since 2010 (when they were introduced) until 2014 (three cohorts of students). The final integrated, rather than individual module, portfolios were selected because they better reflected students’ overall experience. Narratives/text segments were analysed qualitatively by thematic content analysis, a method used to identify, analyse and report patterns (themes) within data.[7] Analysis was supported by MAXQDA version 11 (VERBI Soft ware GmbH, Germany).[8] Deductive and inductive codes were used. We identified deductive codes by reviewing literature on the role of portfolios in health professional education, realising codes such as ‘achievement of Background. Portfolios are increasingly used across a range of disciplines in health professional education to support reflective practice and to help assess students’ academic and professional development. However, their value in postgraduate education is uncertain. Objectives. To identify the role of portfolios in the development and assessment of professional competencies in postgraduate maternal and child public health education. Methods. A qualitative retrospective review of 35 student portfolios was conducted. Thematic content analysis of portfolios was done, identifying emerging themes and analysing patterns. Results. Two major themes were explored – the benefit of the portfolio to the student and to faculty. For students, portfolios promoted reflective abilities and critical thinking and assisted them in planning learning needs. For faculty, the portfolios assisted in monitoring students’ growth over time, identified learning gaps, helped to establish if expected learning outcomes were being attained and provided feedback on students’ application of academic learning to professional practice. Portfolios also offered students an opportunity to provide critical feedback on curricula content and course pedagogic activities. Conclusion. Portfolios are an under-utilised assessment and self-development tool in postgraduate training. They allow students to self-assess their attainment of personal learning needs, professional growth and competency achievement and provide faculty with useful feedback on curriculum content, educational activities and competency attainment. Afr J Health Professions Educ 2017;9(2):78-82. DOI:10.7196/AJHPE.2017.v9i2.796 Reflective portfolios support learning, personal growth and competency achievement in postgraduate public health education H Pandya, BDS, MPH; W Slemming, BSc (Physiotherapy), MPH; H Saloojee, MB ChB, FCPaed (SA), MSc Division of Community Paediatrics, Department of Paediatrics and Child Health, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa Corresponding author: H Pandya (drhimanip@gmail.com) This open-access article is distributed under Creative Commons licence CC-BY-NC 4.0. Research June 2017, Vol. 9, No. 2 AJHPE 79 learning outcomes’, ‘reflective and critical thinking ability’ and ‘application of learning to practice’. New inductive codes such as ‘feedback on pedagogic techniques’ and ‘plan learning needs’ emerged while analysing the portfolio scripts. Finally, all codes were collated to develop two overarching themes, i.e. student benefits and faculty benefits. Ethical clearance for the study was obtained from the Human Research Ethics Committee (Medical) of the University of the Witwatersrand (ref. no. M150750). Results Findings are presented under two major headings: the benefit of portfolios to students and to faculty. Portfolios substantially contributed to enhancing students’ critical thinking and reflective abilities and in planning their learning needs during the course of the degree. Portfolios assisted the faculty in assessing gaps in students’ learning, and in monitoring and appreciating their personal growth, achievement of learning outcomes and application of learning to professional practice. Fig.1 presents a framework that summarises the contribution of portfolios to both students and faculty in postgraduate public health education. It depicts the relationship between major themes and codes, with relevant examples for each code. Student benefits of developing a portfolio Portfolios promote students’ reflective abilities and stimulate critical thinking Students began relating topics and issues covered in modules to events in their daily lives and workplace and utilised the portfolio to reflect on this: ‘I was impressed most by the psychiatric disorder session; I could link this to street children in the Democratic Republic of Congo (where resided). Most of them were soldiers and rejected by their family by mothers saying they are not normal, they are sorcerers. The life in the street is very difficult and I cannot imagine the life of girls begging and sleeping on the street and how fragile girls are in the street.’ (MPH, 2011) ‘I wrote an assignment on mental health of South African children with a new found understanding of and respect for the field. I always thought mental health services were for the most severe cases and the others would simply sort themselves out over time. However, millions of children throughout South Africa and the world suffer in silence from the myriad challenges, i.e. poverty, parental loss, abuse, etc., they are forced to face every day.’ (MPH, 2011) Students were exposed to MCH programmes and policies, and controversial issues, such as HIV infant feeding, abortion and reproductive health, were debated during modules. Portfolios contained evidence that students re-examined their own biases and pre-existing notions based on exposure to this additional knowledge and exposure: ‘There is a notice on the entrance to the family planning section of the clinic we visited that says: No male allowed. The lack of men’s participation in reproductive health services and family planning means that they do not benefit from any information given by heath providers regarding sexuality, pregnancy and their roles in these.’ (MSc, 2011) ‘On that day, only 20 people came for family planning services. Again I started asking myself. Why is it that there are so many people seeking termination of pregnancy (TOP) services and not family planning services? Are people using TOP as family planning method? This made me realise that designing interventions for public health problems is not easy, it needs careful planning, involvement of stakeholders and beneficiaries and there is need to constantly assess the impact of the designed intervention.’ (MPH, 2011) Portfolios help students to plan their learning needs and reflect on performance Through portfolio writing, students systematically compared their learning with explicit MCH competencies they were expected to attain, assessed Fig. 1. Framework showing contribution of a portfolio in postgraduate public health education. To students To Faculty Promote re�ective abilities and critical thinking Plan learning needs Assess gaps in student learning Assess students’ growth Assess application of learning in professional practice Feedback on curricula content and pedagogic activities – Course subject matter – Gaps in public health policies and programmes – Attainment of competencies – Modi�cation of learning approaches – Research design – Presentation skills – Data interpretation – Budgeting – Improved communication, leadership and management – Professional goal attainment – Change management at work place – Feedback on group work and experiential visits to clinics Assessment Feedback from students Examples Research 80 June 2017, Vol. 9, No. 2 AJHPE gaps and indicated how they planned to modify their learning approaches. They also reflected on their performance in assignments and examinations during the modules, analysed reasons for poor performance and developed strategies to improve their performance: ‘I have compared my learning to the MCH competency framework for those areas relevant to all modules. I have decided to do this on an ongoing basis … since I will need to pursue a certain amount of self- directed learning to ensure that I am up to speed in all areas of my studies.’ (MSc, 2010) ‘I realise that information comes from reading or seeking to know more. I will need to increase my level of reading of academic papers as well as challenge myself to think critically.’ (MPH, 2011) Students did not report any negative perceptions about the portfolio, such as fear of confidentiality being breached, distraction from other coursework activities or the repercussions of expressing negative views. Many students found the activity quite challenging at first, because it forced them to think creatively and critically, which deviated from their previous academic experiences. However, this challenge was overcome as more portfolio entries were completed and as they received positive feedback. Confidence in the lack of negative consequences grew as students recognised that unfavourable reflections on course activities were viewed positively rather than being discouraged by staff: ‘I was introduced to the concept of the “portfolio” – this filled me with a real sense of dread. It encouraged us to “think out of the box” and be creative. For scientifically-minded and very right-brained individuals, “creative” is often something that does not come easily! As more modules rolled around, I think that I understood the portfolio-thing a bit better and felt that I was synthesising many of the concepts learnt during the lecture block weeks.’ (MPH, 2013) Benefits of the portfolio to faculty Portfolios help faculty to assess gaps in students’ learning Student portfolios helped us, as faculty members/course organisers, to identify gaps in our students’ academic and professional knowledge and skills. Common gaps reported by students included poor epidemiology, research, health budgeting and strategic planning skills and inadequate knowledge of socioeconomic and community components of public health (specifically reported by clinician students): ‘As a clinician, I am able to perform exceptionally well. As a leader, a researcher, a manager and analyser of systems, I found myself in an uncomfortable place where I could understand what was needed to be done but felt lacking in terms of the knowledge and tools to reach my goals.’ (MSc, 2012) ‘I must admit that working out budgets was never one of my strong points. I always thought as a public health manager, I will just employ someone qualified to deal with that part. During the discussion session, it became clearer to me that as a leader it is wiser to be involved and understand the financial aspect of any MCH programme implemented.’ (MSc, 2010) Portfolios demonstrate students’ growth and achievement of learning outcomes Students used portfolios to reflect on ways in which they matured over the 2 years of the Master’s programme. They reported an improvement in their knowledge of MCH topics, communication skills (oral presentation and writing) and leadership/management skills. Some explained how their newly acquired competencies improved their performance as health professionals: ‘After this module, I had learned a lot about programme planning and managing. All my life I have been taught to implement, not to plan, the experience that I gained over the years, to sit in the consultation room and see patients. At the end of the month I would write reports as part of my responsibility, not knowing that programme planning is based on numbers.’ (MPH, 2011) ‘During the course I was exposed to group presentations due to which my communication skills have improved and facing an audience is no longer as frightening as before. I conduct most of the mortality and morbidity meetings in my department and there is no doubt now that a big stage will be next.’ (MSc, 2011) Portfolios display how students apply learning to professional practice Portfolios offered a platform for students to identify and reflect on their professional goals and objectives and to indicate how they transferred their academic learning to professional practice. Students outlined their immediate and longer-term goals and described how they would achieve these. Students reflected on changes they implemented in their professional practice as a result of their learning: ‘I took back to hospital the use of zinc as one step to improve the outcome of diarrhoea in our ward and now all doctors prescribe it. The pharmacy has joined us in an effort to have zinc available all the time. We have already seen a change in the duration of paediatric patients’ stay in hospital in the last few weeks.’ (MSc, 2011) ‘By the time I wrote my third and fourth portfolio, it was great work. I drew up my personal experience and all the principles of writing which I learnt a long time ago and this really helped me. Currently, the reports that I write for my work look much better than before.’ (MPH, 2011) Portfolios provide feedback on curricula content and pedagogic activities Students reflected on various teaching and learning techniques experienced in the programme. Portfolios captured critical feedback from students not often acquired through other feedback mechanisms. Further, portfolios fostered the development of a professional identity: ‘It came to me as a pleasant surprise to find engagement, interrogation and stern but helpful questions and advice employed as tools of learning. As I already understood, this was just one of the skills that would be necessary for me to be successful as not only a Master’s candidate but also as a leader, researcher and public health professional.’ (MSc, 2014) ‘I was looking forward to the group work sessions, as these were important highlights for me during this module. The group work sessions were slowly training me to be the leader or manager I had in mind.’ (MSc, 2010) Discussion This study demonstrates that reflective portfolios can substantially contribute to postgraduate and public health education. While the study echoes many of the findings from the existing literature on the benefits of portfolios in health professional education, it is the first to do so specifically in the context of maternal and child public health education and in an African setting. Key findings emanating from this study include the following: • Portfolios are similarly useful in postgraduate public health education, specifically MCH, as in other disciplines of health professional education. Research June 2017, Vol. 9, No. 2 AJHPE 81 • Portfolios can assess students’ learning needs, growth and competency achievement and provide useful feedback on pedagogic activities and curriculum content. • Portfolio guidelines and marking rubrics can contribute greatly in directing students’ reflections and ensuring that a full range of activities and developmental areas are considered. • Portfolios of postgraduate public health students working within health systems while studying contain crucial evidence of ways in which academic learning is applied in professional practice. With the complexity of emerging public health issues, it is imperative that Master’s graduates not only demonstrate mastery of theoretical content, but also critical thinking and application in practice. They are expected to function as reflective practitioners capable of evaluating policies and programmes and fixing gaps in the public health system. We believe that portfolio writing enhanced students’ reflective ability and critical thinking skills. This conclusion concurs with that in a study indicating that portfolios contributed substantially in stimulating critical thinking in social work education.[9] There is mixed evidence as to whether portfolios aid or hinder reflection in postgraduate education, as it depends on individual learning preferences.[2] We found that writing a portfolio generally enabled students to relate the knowledge gained during the coursework to their real life experiences and to reflect on an ongoing basis. We identified no demographic, cultural or learning traits that restricted this conclusion being drawn. We established that students utilised the portfolio as a platform to iden- tify their learning gaps, plan learning needs and organise their academic journey during the course of a Master’s programme. In a systematic review of the effectiveness of portfolios for postgraduate assessment and education, portfolio users were shown to take increased responsibility for their own learning and be less passive learners.[2] Keim et al.[10] showed that, compared with a control group, portfolio users produced more learning needs assess- ments and learning plans. Portfolios have been recognised as useful tools for faculty to conduct students’ assessments and identify gaps in their learning. However, there is wide variation in the literature in the level of reliability of portfolios for assessment. It is recommended that portfolios should not be used for summative judgements but for more qualitative and less structured student assessments.[2] We assigned summative judgements (20% of the year mark) to our portfolios, primarily to ensure that the task was taken seriously. The study was not designed to test the reliability of our assessment, but we noted good congruence between the three evaluators. Portfolio reflections influenced various curriculum change decisions in our programme, including the delivery and structure of curriculum content, facilitator changes and a greater focus on pedagogic techniques preferred by students, such as experiential site visits. Massive open online courses (MOOCs) were introduced as core ‘pre-readings’ based on portfolio comments suggesting that non-clinician students were struggling with basic clinical concepts during modules. Comments also directed course co-ordinators to the type of changes needed in student assessment and course evaluation tools. Portfolio reflections provided qualitative or descriptive insight into intangible aspects of students’ growth, which could not be assessed by conventional modes of assessment, such as written examinations and assignments. Portfolio entries demonstrated ways in which students grew and evolved over the duration of the MSc/MPH programme. The portfolio guidelines (and assessment rubric) also demanded student engagement with activities that may otherwise have been ignored, such as continuous reflection on, and self-evaluation of, course competency achievement. Lastly, a particular strength of the portfolio was the students’ reflection on the transfer of academic learning to their respective workplaces. Literature suggests that portfolios support application of learning to practice in health professional education, especially if their use is continued at the workplace.[2] A portfolio can promote holistic learning by serving as a reflective bridge between the student, the workplace and the academy.[11] Our study expands this evidence base with a specific focus on postgraduate public health educa- tion in MCH. As this was a qualitative study based on narratives reported in portfolios, we cannot quantify the proportion of students to whom the positive findings were applicable. This limits the generalisability of the results. We acknowledge that some students had better reflective abilities than others. Students’ claims could not be verified by using other evaluative methods. Further, as the study did not follow up students after they graduated, we cannot verify if the students’ intentions and self-reported competencies were actually realised or put into practice after degree completion. As the portfolio contributed to summative assessment, students may have presented undue positive or favourable comments to curry favour from staff. To dissuade students’ from doing this, we rewarded critical, rather than overly positive comments. We believe this approach was successful. Future research should explore if there is a gap between self-reported learning, competency achievement and actual practice. It would be worthwhile ascertaining if graduates continue reflective activities, such as diaries, blogs or similar activities, in a professional setting when portfolio writing is no longer a compulsory course activity. Conclusion The continuing development and improvisation of higher education in specialised fields of health, such as MCH, demand more valid and reliable assessment of knowledge, competency and skill attainment, as well as attitude and behaviour assessment. A reflective portfolio can successfully serve this purpose for both students and faculty. Portfolios written by MSc/ MPH students contained crucial evidence of reflective practice, critical thinking, self-growth, professionalism, knowledge management processes and heightened appreciation of exit competency outcomes. The portfolio also enabled us, as faculty, to attain a better understanding of student experiences and exit outcomes accomplishment. Acknowledgement. We thank our students who agreed to have their portfolio entries analysed and publically shared on the promise of anonymity. 1. McMullan M, Endacott R, Gray MA, et al. Portfolios and assessment of competence: A review of the literature. J Adv Nurs 2003;41(3):283-294. http://dx.doi.org/10.1046/j.1365-2648.2003.02528.x 2. Tochel C, Haig A, Hesketh A, et al. The effectiveness of portfolios for post-graduate assessment and education: BEME Guide No. 12. Med Teach 2009;31(4):299-318. http://dx.doi.org/10.1080/01421590902883056 3. Sauer KA. Use of reflective portfolios in health sciences education. Am J Pharm Educ 2007;71(2):1. http://dx.doi. org/10.5688/aj710234 4. David MFB, Davis M, Harden R, Howie P, Ker J, Pippard M. AMEE Guide No. 24: Portfolios as a method of student assessment. Med Teach 2001;23(6):535-551. http://dx.doi.org/10.1080/01421590120090952 5. Davis MH, Ponnamperuma GG, Ker JS. Student perceptions of a portfolio assessment process. Med Educ 2009;43(1):89-98. http://dx.doi.org/10.1111/j.1365-2923.2008.03250.x 6. Driessen E, van Tartwijk J, van der Vleuten C, Wass V. Portfolios in medical education: Why do they meet with mixed success? A systematic review. Med Educ 2007;41(12):1224-1233. http://dx.doi.org/10.1111/j.1365- 2923.2007.02944.x 7. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006;3(2):77-101. http://dx.doi. org/10.1191/1478088706qp063oa 8. MAXQDA. The art of data analysis. 2016. http://maxqda.com/legalinfo (accessed 4 April 2016). 9. Coleman H, Rogers G, King J. Using portfolios to stimulate critical thinking in social work education. Social Work Educ 2002;21(5):583-595. http://dx.doi.org/10.1080/0261547022000015258 10. Keim KS, Gates GE, Johnson CA. Dietetics professionals have a positive perception of professional development. J Am Diet Assoc 2001;101(7):820-824. http://dx.doi.org/10.1016/s0002-8223(01)00202-4 11. Brown JO. The portfolio: A reflective bridge connecting the learner, higher education, and the workplace. J Contin Higher Educ 2001;49(2):2-13. http://dx.doi.org/10.1080/07377366.2001.10400426 http://dx.doi.org/10.5688/aj710234 http://dx.doi.org/10.5688/aj710234 http://dx.doi.org/10.1111/j.1365-2923.2007.02944.x http://dx.doi.org/10.1111/j.1365-2923.2007.02944.x http://dx.doi.org/10.1191/1478088706qp063oa http://dx.doi.org/10.1191/1478088706qp063oa http://dx.doi.org/10.1016/s0002-8223(01)00202-4 Research 82 June 2017, Vol. 9, No. 2 AJHPE Appendix 1. Marking rubric for MSc/MPH portfolios Student Detail Name Module Characterisation of dimensions of portfolio design Style Descriptive ٱ ٱ ٱ ٱ ٱ Reflective Structure Informal ٱ ٱ ٱ ٱ ٱ Formal Confidentiality Personal ٱ ٱ ٱ ٱ ٱ Public (closed) (open) Content Focused ٱ ٱ ٱ ٱ ٱ Comprehensive (e.g. critical incidents) (e.g. range of activities) Presentation Grade Comment • Evidence of guidance framework (index, conclusion, captions, etc.) • Organisation • Visual appeal • Spelling/grammar • Referencing Content Grade Comment • Conciseness/length • Creativity • Effort • Professionalism • Additions (e.g. articles) Reflection on Grade Comment • Critical incidents • Educational activities • Programme objectives • Response or solutions to issues • Competencies achieved • Self growth (e.g. leadership, embracing diversity, communication, etc.) Evaluator’s comments • Overall decision: (%) • Most enjoyed or impressed by: • Could have been improved by: • General comment/s Evaluator’s name Date Grading Percent Category A+ > 80 Exceptional A 76 - 80 Excellent B 70 - 75 Superior C+ 65 - 69 Good C 60 - 64 Satisfactory D 50 - 59 Minor deficiencies F < 50 Major deficiencies (fail) N - Not done