Case Studies Compass: Journal of Learning and Teaching, Vol 12, No 1, 2019 Peer-assisted learning to support attainment in Pharmacy Rebecca Maccabe, Ricarda Micallef, Mark Carew Kingston University London Abstract Academic mentoring (AM) is embedded in two modules of the Pharmacy course, using a peer-assisted learning (PAL) design. Training and payment are given to student mentors who work with academics and the AM team to create student-centred active learning material. In 2017/18, student attendance at PAL sessions was high, with 87% of the first- and second-year Pharmacy cohorts attending a session. Thirteen mentors supported both modules for a semester each. The impact of PAL in these sessions was measured by quantitative and qualitative analyses, using institutional data on retention, progression, attainment and self-report data. These data showed that students who engaged in PAL were: from traditionally under-represented groups in higher education (HE); specifically- disadvantaged students; from a Black and Minority Ethnic background (BME). PAL-engaging students had higher progression, retention and module pass rates. A survey of fifty mentees found unanimous agreement for the proposal that PAL was valuable and it confirmed that mentors were regarded as enthusiastic, organised and prepared. The formalisation of the AM and PAL process, together with its co-production between mentors and mentees, has increased the professionalism of the scheme and should be supported in the future. Introduction Context Kingston University is a post-1992 university in South West London, with a diverse student population of about 17,500 students, 56% of whom are from a Black and Minority Ethnic (BME) background. The institution aims to ensure success for all students and has a strong reputation in the sector for its successful work on the BME attainment gap (McDuff et al., 2018). Academic Mentoring Programme The purpose of academic mentoring (AM) at Kingston University is to improve the retention, progression, attainment and employability of all students (Williams and Reddy, 2016; Carbone et al., 2015). At this institution, AM is focused particularly on the attainment of BME students and those from a low socio-economic background, with funding for the project coming from the Access and Participation Plan. AM is embedded in twenty-four courses across four faculties and 150 mentors are recruited annually to facilitate the learning of over 1200 first- and second-year students. Pharmacy is the only course where AM operates in both the first and second year and it has the highest level of student engagement in the AM portfolio. The AM programme adopts, depending on the discipline, four structures: peer- assisted learning (PAL); mentors embedded into learning and teaching; a buddy scheme; e- mentoring. PAL involves the facilitating, by experienced student mentors or peers, of study Case Studies Compass: Journal of Learning and Teaching, Vol 12, No 1, 2019 sessions for students in the year(s) below. PAL sessions in Pharmacy modules are structured so that peer mentoring is closely aligned to learning, teaching and assessment (Metzger et al., 2013). Because mentors facilitate peer-learning environments, they must be confident in their subject knowledge, demonstrate leadership skills and be able to communicate effectively and work productively with a diverse student body. Mentor recruitment is in two stages: the first is based on recommendations from academic staff to ensure that students have the relevant academic credentials; the second involves mentor participation at an assessment centre to evaluate the key competencies listed above. In September, mentors receive formal training, which includes topics such as mental health awareness, unconscious bias and workshops from Teach First, a charity which recruits and develops trainee teachers to create educational equality in England and Wales. Research has shown that mentors also greatly benefit from the PAL experience (Williams and Reddy, op.cit.; Hill et al., 2010). PAL in Pharmacy Pharmacy in the United Kingdom (UK) is a four-year undergraduate course culminating in a Master of Pharmacy (MPharm) degree. Upon completion of a twelve-month pre-registration placement and the final examination, the candidate may qualify as a pharmacist. The institution is one of thirty-one schools of Pharmacy in the UK accredited by the General Pharmaceutical Council (GPhC), the regulator of Pharmacy in the UK, to deliver the MPharm degree programme. PAL sessions are becoming more widespread in Pharmacy teaching and are designed to support the student experience and attainment (Aburahma and Mohamed, 2017; Metzger et al., 2013). PAL at the institution has been used in the second-year Pharmacy Law, Ethics and Practice module (PY5020) for six years. The PAL sessions were originally introduced to support learning in this module, which has higher demands than other modules – notably a higher pass mark of 50%, compared to 40% in other second-year modules. In 2016, PAL sessions were also introduced in the first-year Cell Biology and Physiology module The Human Body (PY4010). Both this and PY5020 are year-long, thirty-credit modules. Prior to this time, the PAL mentors who delivered the sessions were third- and fourth-year volunteers. From 2016 onwards, the mentoring process became more formalised and module leaders worked collaboratively with the AM team to recruit, train and support mentors. PAL sessions in PY5020 are student-led; mentors deliver the small group sessions without the presence of an academic. Multiple scenarios are prepared by the mentors, with outside feedback and support from the module leader, and the mentors facilitate learning by moving around a cabaret-style classroom layout and interacting with the mentees. In PY4010, PAL sessions are delivered weekly, to supplement learning and teaching. These sessions are used, for example, to consolidate lecture material and the mentors also model effective ways of learning to aid knowledge retention. The sessions for both modules are student-centred and promote active learning. This approach helps to create a collaborative community of learners in Pharmacy, with common activities that include the practice of Observed Structured Clinical Examination (OSCE) scenarios, formative quizzes, concept-mapping, wider reading around the discipline and revision strategies. Case Studies Compass: Journal of Learning and Teaching, Vol 12, No 1, 2019 PAL and the inclusive curriculum The aim of the sessions is to support students who work in partnership with mentors to co- create knowledge within an inclusive curriculum. Although the PAL sessions are in the academic timetable, attendance at PAL sessions is voluntary. Students may opt out of attendance if they wish; voluntary attendance is therefore seen by the students as a positive choice, rather than as compulsory participation in a remedial scheme. With the support of the module leader, students co-design and deliver PAL sessions and, since the mentors help to shape learning and teaching practice on the basis of their own experiences, the learning is relevant, tailored and inclusive. AM can play a valuable role in forming learning communities that bring students from different levels to learn together, an approach with particular relevance to the Teaching Excellence Framework. The BME attainment gap is a long-standing concern across HE, although there is still limited evidence of what creates this at an institutional level (Richardson, 2015 and McDuff et al., op.cit.). The gap in Pharmacy continues to exist even after graduation, where there is a significant difference in pass rates between BME and white students in the GPhC pre- registration exam (James, 2018). A GPhC-commissioned report described feelings of isolation and exclusion as difficulties for BME students (OPM, 2016). Initiatives such as PAL sessions can help to alleviate these problems through the formation of supportive peer- learning networks. Analysis at local level shows that those students who do not achieve at the early stages of their degree are more likely to fail at first attempt of the registration assessment. Support early on in a degree course, for example with PAL sessions, may therefore support transition into the workplace. Description of the sample Table 1. PY4010 2016/17 2017/18 Mentors 14 (second year) 13 (9 third year, 4 final year) Mentees 76 103 Cohort size 124 118 PY5020 2016/17 2017/18 Mentors 13 (third year) As above Mentees 98 116 Cohort Size 161 133 The number of Pharmacy mentors has decreased from twenty-seven in 2016/17 to thirteen in 2017/18. Over that time period, the number of students engaging with PAL increased across both years, thus showing better use of finite resource. In order to support both modules, mentors were asked to support one module per semester, working in smaller teams. For PY5020 PAL sessions, the cohort was split into two groups, with the same content delivered twice. For the PY4010 module, the whole cohort was timetabled to attend the same PAL session. Case Studies Compass: Journal of Learning and Teaching, Vol 12, No 1, 2019 Method of evaluation The overarching research questions for this study were: 1) Do student participants in Pharmacy PAL have higher progression, retention and module pass rates than those students who do not engage in the programme? 2) Is PAL in Pharmacy mutually beneficial for mentors and mentees? To measure the impact of PAL in Pharmacy, PAL attendance data for PY4010 and PY5020 in 2016/7 and 2017/8 were recorded and analysed against institutional data on retention, progression and module pass rates. Attendance data were reported for all students who attended PAL sessions. Data from the under-represented groups of interest in this study were then analysed. To measure the effectiveness of PAL, an online survey – consisting of five closed questions for mentees on a Likert scale and two open questions for mentors, listed below – was disseminated electronically. The Likert scale ranged from ‘strongly agree’ to ‘strongly disagree’; responses were subsequently grouped into positive (‘strongly agree’, ‘agree’) and negative (‘not sure’, ‘disagree’, ‘strongly disagree’) statements. In previous years, to measure the impact of PAL, more generic questions were used, in terms of ‘student confidence’, ‘resilience’ and ‘sense of belonging’. Since student feedback indicated that the questions were not relevant to the programme, the questions for this study were tailored to relate directly to student expectations of PAL. Table 2. Survey questions for mentees Survey questions for mentors PAL has been valuable. In one sentence, summarise your experience as an Academic Mentor. I feel more confident and engaged in lecture material as a result of PAL. In a short paragraph/sentence, reflect on the skills you have acquired through PAL. Owing to PAL, I feel more confident in completing assignments. My mentors were enthusiastic. My mentors were mentors organised and prepared. The surveys were created on Google Forms and distributed in a PAL session using iPads (and email thereafter). The response rate for mentees was 23% (n=50/219) and for mentors 46% (n=6/13). The survey questions for mentees focused on the quality and relevance of the sessions. The questions for mentors centred on the skills obtained from the role. Presentation and analysis of the data Overall, the data showed that those students who engaged in PAL had higher progression, retention and module pass rates than those students who did not engage in the programme, with the most positive impact being on student progression rates for BME and ‘disadvantaged’ students. Students who attended two or more mentoring sessions were defined as ‘engaged’, whilst those who did not attend or attended only one session were Case Studies Compass: Journal of Learning and Teaching, Vol 12, No 1, 2019 defined as ‘not engaged’. Engagement was calculated by the average session attendance in 2016/17 for PY4010 and PY5020 and this figure was also used for 2017/8 to ensure consistency across the academic years. Table 3. Progression at first attempt for BME students Not engaged Engaged progressed/ total % progressed progressed/total % progressed NNP (95% CI) PY4010 2016/17 36/67 53.7% 27/35 77.1% 5 (2.4- 19.7) 2017/18 12/21 57.1% 56/77 72.7% 7 (*) PY5020 2016/17 51/84 60.7% 41/48 85.4% 5 (2.6- 9.8) 2017/18 14/33 42.4% 65/79 82.3% 3 (1.7- 4.8) BME student progression at first attempt on PY4010 and PY5020 for 2016/17 and 2017/18 by their engagement in PAL. Table 4. Module pass rates for BME students Not engaged Engaged passed/total % passed passed/total % passed NNP (95% CI) PY4010 2016/17 60/67 89.6% 35/35 100% 10 (5.6 – 32) 2017/18 16/21 76.2% 65/77 84.4% 8 (*) PY5020 2016/17 78/84 92.9% 48/48 100% 14 (7.9 – 61.1) 2017/18 29/33 87.9% 77/79 97.5% 11 (*) BME student module pass rates on PY4010 and PY5020 for 2016/17 and 2017/18 by their engagement in PAL. Tables 3 and 4 show that there was an observable difference between the module pass rates and progression rates of BME students who engaged in PAL, compared to those who were defined as ‘not engaged’. Institutional data showed that PAL positively improved progression and module pass rates for both white and BME students. However, because 90% of the Pharmacy cohort were BME, and PAL was funded through the Access and Participation Plan, the focus for discussion is on the BME student data. The data were analysed further, with a calculation of Number Needed to Progress (NNP). The NNP was a metric designed to be analogous to the NNT (Number Needed to Treat) in medical research. A NNT of 5 means one person out of five receives the benefits of the treatment. In our case, a NNP of 5 would mean that one student out of five would receive the Case Studies Compass: Journal of Learning and Teaching, Vol 12, No 1, 2019 benefit of PAL and pass a module or progress the year. NNPs were calculated using the NNT calculator at https://www.graphpad.com/quickcalcs/NNT1.cfm. NNP values ranged from 3 to 14. In three cohorts (marked with an asterisk), the 95% Confidence Intervals (CIs) were associated with a negative NNP and therefore difficult to interpret. NNP values ranged from 3-7 for progression at first attempt and 8-14 for module pass rates. PAL may be more effective, therefore, in the overall progression of a student at first attempt. The data on module pass rate include the resit attempt, where effects of PAL may diminish as more students pass. The PY5020 module also had a high pass rate and so there was less room for the effect of an intervention. Table 5. Progression at first attempt Not engaged Engaged progressed/total % progressed progressed/total % progressed PY4010 (male) 2016/17 14/29 48.3% 13/16 81.3% 2017/18 4/9 44.4% 18/25 72.0% PY5020 (female) 2016/17 43/66 65.2% 30/32 93.8% 2017/18 8/20 40.0% 51/61 83.6% Progression at first attempt on PY4010 (male students) and PY5020 (female students) for 2016/17 and 2017/18 by their engagement in PAL. Engaging in PAL increased progression and module pass rates for both male and female students in Pharmacy, with the biggest impact being on male progression in PY4010 and female progression in PY5020, as shown by Table 5. Disabled students (accounting for 7- 15% of students across both modules) who were defined as ‘engaged’ had higher progression rates on the whole than their ‘disengaged’ counterparts. The data also revealed a noticeable difference between the progression rates of ‘disadvantaged’ students who engaged in PAL compared to those who were defined as ‘not engaged’. Table 6. Progression at first attempt for disadvantaged students Not engaged Engaged progressed/ total % progressed progressed/total % progressed NNP (95% CI) PY4010 2016/7 29/51 56.9% 20/25 80.0% 2 (1.3- 3.1) 2017/8 8/18 44.4% 45/57 79.0% 3 (1.7- 7.5) PY5020 2016/7 33/63 52.4% 26/31 83.9% 14 (*) 2017/8 12/29 41.4% 42/50 84.0% 18 (*) Disadvantaged student progression at first attempt on PY4010 and PY5020 for 2016/17 and 2017/18 by their engagement in PAL. https://www.graphpad.com/quickcalcs/NNT1.cfm Case Studies Compass: Journal of Learning and Teaching, Vol 12, No 1, 2019 Students were defined as being ‘disadvantaged’ if they met one or more of the multiple indicators of disadvantage (low participation neighbourhood, indices of multiple deprivation and socio-economic status). As shown by Table 6, the NNP data was very positive for PY4010, indicating that, in 2016/17, one student out of two received the benefit of PAL in terms of higher progression rates and, in 2017/18, one student out of three received the benefit of PAL. Retention figures were calculated for full-time first degree student entrants taking the PY4010 module in 2016/17 (N=64). Students were defined as ‘retained’ if, in the following academic year, they were still studying at the institution or had received an award. Of students engaging in PAL 100% were retained (N=19). Students who did not engage in PAL had a retention rate of 88.9% (N=45). Retention at the institution is calculated between Level 4 and Level 5; it therefore applies only to students taking PY4010. Since the implementation of PAL in 2016/17 into PY4010, the overall student progression at first attempt has increased. Table 7. Overall student progression at first attempt progressed/total % progressed PY4010 2015/16 81/138 58.7% 2016/17 75/124 60.5% 2017/18 79/118 67% Student progression at first attempt for PY4010 pre and post PAL intervention. Table 7 shows that progression rates for students were lower in 2015/16 prior to PAL intervention and, whilst it cannot be concluded that PAL alone increased progression rates, it can be viewed as a factor contributing to improving this key metric. The survey findings echo the success of PAL in Pharmacy; of fifty mentee respondents, 100% agreed that PAL had been valuable, that mentors were enthusiastic, organised and prepared. Additionally, 96% of the respondents felt more confident about completing assignments as a result of PAL, indicating its alignment to learning, teaching and assessment. Qualitative data were collected from the mentors and three broad themes emerged through thematic analysis, with the mentors commenting on their increased confidence, communication and time-management skills as a result of the mentor role. One mentor revealed the impact on his own academic studies: “PAL has also helped to consolidate my own knowledge and understanding through mentoring others.” When asked to summarise their PAL experience in one sentence, the mentors offered: “amazingly unique”; “an exciting and rewarding experience”; “gained a lot of experience and confidence”; “helped to break down barriers, build friendships and improve confidence through enhanced learning.” The feedback shows that PAL is mutually beneficial for mentors and mentees, echoing previous studies (Williams and Reddy, op.cit; Hill et al., op.cit.). Case Studies Compass: Journal of Learning and Teaching, Vol 12, No 1, 2019 The success of PAL in Pharmacy has gained exposure in the sector. The AM team and two Pharmacy mentors were invited to present at a Staff and Educational Development Association (SEDA) conference on PAL, discussing what works and what does not. The audience commended the PAL model in Pharmacy, because it is clearly embedded in the curriculum, it is actively championed by the academic department and it combines the resources of both mentors and the module leader. Since presenting at SEDA, numerous institutions have modelled their schemes on the co-production example in Pharmacy. Limitations Whilst the evidence suggests that those students who engage in PAL have higher progression, retention and module pass rates than their ‘disengaged’ counterparts, some limitations need to be taken into account. The findings may be somewhat limited by the fact that the study considered PAL in only one academic course at one institution – the low survey response rates may have created a non-response bias. Furthermore, as Pharmacy students are training to be professionals, they are also expected to abide by GPhC Standards, which may, in part, be the reason for their motivation and enthusiasm. Despite these promising results in Pharmacy, questions remain in terms of the effectiveness of PAL in other disciplines. Other institutions may wish to explore this in terms of further research. Additionally, it could be argued that the positive results were down to other influences beyond peer support, as it is often the case that students who engage in PAL engage in all aspects of university life and therefore are more inclined to achieve and progress. This is an important issue for future research. Conclusion This study suggests that PAL is a successful initiative for the Pharmacy course and should be further supported in the future, so as to benefit both mentor and mentee. The AM programme should be extended to other courses at the institution, following exemplary practice in Pharmacy, and further research ought to be undertaken to investigate whether PAL should be an institution-wide initiative. Although PAL is a fruitful area for further work, it must be noted that there would be significant financial costs associated with programme expansion as the mentors are paid at Kingston University. Reference list Aburahma, M. and Mohamed, H. (2017) ‘Peer teaching as an educational tool in Pharmacy schools; fruitful or futile’ Currents in Pharmacy Teaching and Learning 9, 1170-1179. Available at: https://doi.org/10.1016/j.cptl.2017.07.026 (Accessed: 02 August 2018). Carbone, A., Ross, B., Phelan, L., Lindsay, K., Drew, S., Stoney, S. and Cottman, C. (2015) ‘Course evaluation matters: improving students’ learning experiences with a peer-assisted teaching programme.’ Assessment and Evaluation in Higher Education. 40(2),165-180. Available at: https://www.tandfonline.com/doi/abs/10.1080/02602938.2014.895894 (Accessed: 02 August 2018). https://doi.org/10.1016/j.cptl.2017.07.026 https://www.tandfonline.com/doi/abs/10.1080/02602938.2014.895894 Case Studies Compass: Journal of Learning and Teaching, Vol 12, No 1, 2019 Hill, E., Liuzzi, F. and Giles, J. (2010) ‘Peer-assisted learning from three perspectives: student, tutor and co-ordinator.’ The Clinical Teacher, 7, 244-246. Available at: https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1743-498X.2010.00399.x (Accessed: 02 August 2018). McDuff, N., Tatam, J., Beacock, O. and Ross, F. (2018) ‘Closing the attainment gap for students from black and minority ethnic backgrounds through institutional change.’ Widening Participation and Lifelong Learning. 20(1), 79-101. Available at: https://doi.org/10.5456/WPLL.20.1.79 (Accessed: 21 August 2018). Metzger, A., Stoner, S. and Hanes, S. (2013) ‘Essential Elements for a Pharmacy Practice Mentoring Program.’ American Journal of Pharmaceutical Education. 77(3), Article 23. Available at: https://www.ajpe.org/doi/full/10.5688/ajpe77223 (Accessed: 02 August 2018). OPM (2016) Qualitative research into Registration Assessment performance among Black- African candidates. London. Available at: https://www.pharmacyregulation.org/sites/default/files/qual-research-into-ra-performance- among-black-african_candidates_final_0.pdf (Accessed: 22 August 2018). Richardson, J. (2015) ‘The under-attainment of ethnic minority studentsin UK higher education: what we know and what we don’t know.’ Journal of Further and Higher Education 39(2) 278-291. Available at: https://www.tandfonline.com/doi/abs/10.1080/0309877X.2013.858680 (Accessed: 02 August 2018). Williams, B. and Reddy, P. (2016) ‘Does peer-assisted learning improve academic performance? A scoping review.’ Nurse Education Today, 4, 23-29 .Available at: https://doi.org/10.1016/j.nedt.2016.03.024 (Accessed: 02 August 2018). https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1743-498X.2010.00399.x https://doi.org/10.5456/WPLL.20.1.79 https://www.ajpe.org/doi/full/10.5688/ajpe77223 https://www.pharmacyregulation.org/sites/default/files/qual-research-into-ra-performance-among-black-african_candidates_final_0.pdf https://www.pharmacyregulation.org/sites/default/files/qual-research-into-ra-performance-among-black-african_candidates_final_0.pdf https://www.tandfonline.com/doi/abs/10.1080/0309877X.2013.858680 https://doi.org/10.1016/j.nedt.2016.03.024