Journal of Learning Development in Higher Education ISSN: 1759-667X Issue 16: December 2019 ________________________________________________________________________ Investigating impact: Exploring the effect of ‘open’ support on student success Jennie Rose Steres Blake University of Manchester Nicola Grayson University of Manchester Sami Karamalla-Gaiballa University of Manchester Abstract Traditional investigations into the impact of skills support on student success tend to focus on embedded or curriculum linked modes of delivery. The subject of this investigation concerns a study of the impact of ‘open’ support delivered through the University of Manchester library’s My Learning Essentials skills programme (MLE). MLE is a blended service providing both face-to-face and online support through two dominant pathways: one which is embedded in the curriculum and one which is ‘open to all’ regardless of degree programme or level of study. The ‘open’ nature of this type of support and the variety amongst the student population who engage with it means that measuring the impact on areas such as attainment has always been difficult. This article will present the results of a small study that investigated a specific cohort of undergraduate students in order to assess whether connections could be drawn between attendance at MLE ‘open’ workshops and degree classification. Although the cohort investigated was quite small, there is evidence of significant positive impact on student attainment as a result of engagement with the MLE programme. The data was run through a regression analysis that controlled for factors that could influence attainment and compared attendees of MLE open workshops with those who did not attend. Beyond the results of the regression analysis the study reveals interesting data around student uptake of MLE as a service and presents the methodology used, the results gained, and the lessons learned throughout the process. Blake, Grayson, Karamalla-Gaiballa Investigating impact: Exploring the effect of ‘open’ support on student success papers Journal of Learning Development in Higher Education, Issue 16: December 2019 2 Key words: academic development; intervention; impact; open; regression; attainment; undergraduate. Introduction With nearly 40,000 students enrolled in its degree programmes, the University of Manchester is one of the largest single-site universities in the UK. The student body is diverse, as are the requirements for its degree programmes, and the demands for academic support reflect these needs. This is compounded by recent changes to disability support and the recognised needs of students with learning difficulties (Mortimore and Crozier, 2006). Any support that seeks to address the campus as a whole must therefore be flexible and accessible (without presenting itself as generic or remedial) and able to respond to demands put upon it by Manchester’s student body. The support must recognise the needs of the learners and utilise the available research to inform practice, but due to the size and diversity of the student population it needs to serve (and the size of the campus), the support offered cannot depend on resource-heavy interventions such as one-to-one drop ins or individual proofreading support (Howard-Jones et al., 2018). One of the central support programmes is the University of Manchester library’s My Learning Essentials (MLE). MLE is now just over six years old and follows a blended learning model. It encompasses face-to-face workshops, limited one-to-one support and open online resources that address a broad range of topics. Topics covered range from the traditional information literacy areas such as searching and referencing, to academic skills such as writing and critical analysis, through to well-being support that focuses on perfectionism, procrastination, and managing academic pressure. It is important to note that while the well-being workshops focus on areas such as procrastination or academic pressure, they follow the MLE model of addressing these topics through the development of specific skills and strategies and tend to focus on the academic implications of these behaviours. The face-to-face and online support embraces interactivity and promotes key aspects of learning as a process of partnership and co-creation (Blake and Illingworth, 2015). Both also incorporate aspects of Nancy Fisher and Douglas Frey’s ‘release of responsibility’ Blake, Grayson, Karamalla-Gaiballa Investigating impact: Exploring the effect of ‘open’ support on student success papers Journal of Learning Development in Higher Education, Issue 16: December 2019 3 model, which sees learning as a process that ends with participants confidently employing new knowledge independently (Frey and Fisher, 2013). As a whole, the MLE programme is responsive and commits to a deliberate integration of the student voice both through an emphasis on co-creation in learning and through the integration of student feedback to directly shape the design of learning resources and the delivery of support (Grayson et al., 2018). MLE has two distinct avenues through which students and staff access its resources. ‘Embedded’ support is delivered through face-to-face teaching and online resources, which are worked into and aligned with courses (or modules) at the request of academic staff. The embedded support can be online only or ‘blended’ and typically involves a partnership between library staff (who design and deliver the sessions) and academics who are teaching on the course/module. Participants accessing embedded support are almost always from the same subject area and working at the same level of study. In contrast, the ‘open’ support comprises online resources that are freely available to all and can be self-selected through the library website, short drop-in support which is open to all students, and face-to-face workshops. In the face-to-face open workshops, students from any degree programme and working at any level of study attend together, and the focus is on enabling participants to apply new or improved skills to their own study. The focus for this small study is the MLE open workshops (excluding the one-to-one drop-in support). Although we were keen to see the results, we also felt it was important to explore different methods of evaluating an open and self-selecting resource. In the end, this study specifically seeks to explore how impact can be investigated when dealing with ‘open’ learning support in contrast to traditional modes of investigating impac,t which tend to focus on the delivery of support which is ‘embedded’. It will present an understanding of the attendee profile data of participants who engage with the open learning service and outline the impact of participants’ use of multiple face-to-face MLE workshops. This study is very much seen as an exploratory first step, not only to understand the service and its impact on the participants engaging with it, but also to examine the different methodologies and models that may be used to assess study support. It is our hope that the results of this study will be useful for those assessing similar work and that our methods can make a positive contribution to the ongoing conversation of how to define, present, and articulate the worth of skills support and, in turn, of learning development. Blake, Grayson, Karamalla-Gaiballa Investigating impact: Exploring the effect of ‘open’ support on student success papers Journal of Learning Development in Higher Education, Issue 16: December 2019 4 The tension between openness and demonstrable impact In order to understand the contextual landscape of the data and gain a holistic understanding of the MLE service, a small analysis was conducted on those who had self- selected to attend the open workshops. We examined a specific cohort of participants and conducted a regression analysis to compare MLE attendees with non-attendees in relation to attainment. The objective in doing this was to identify any correlation between the use of MLE as a service and degree classification. There is a significant tension between delivering an ‘open to all’ service and gathering adequate data in order to understand the reach, breadth, and impact of such a service. As the wealth of support offered through MLE open workshops differs in content (e.g., it can be focused on academic skills, information literacy, referencing, or well-being support) and students from any cohort and at any level of study can attend, it is a significant challenge to try to gain insights and draw connections that might be more easily drawn in relation to, e.g., attendance at ‘embedded’ sessions. With support embedded in the academic programme, participants can be tracked longitudinally to examine any links between the support provided and specific assessment or learning outcomes (Wingate, 2006). However, with the open support workshops offered by MLE, students self-identify their learning needs and then choose to engage in community with others from different schools and who are working at various levels (undergraduate, post-graduate, and PhD). While essential to reaching the students and staff at the university, this structure does not necessarily enable MLE as a service to easily access the type of data and information that could allow for any definitive causal or correlative impact work. Since the launch of MLE in 2013, the open strand has gathered attendance data in the form of student ID numbers alongside attendee behaviour data, which can be accessed from our booking system. The numbers are anonymised and, by using them as our data points, we are able to examine other aspects of the participants’ make-up (such as gender) without personally identifying the student. Access to this information has allowed us to see who is attending the open workshops (in terms of their faculty/school and level of Blake, Grayson, Karamalla-Gaiballa Investigating impact: Exploring the effect of ‘open’ support on student success papers Journal of Learning Development in Higher Education, Issue 16: December 2019 5 study), which sessions these particular students are selecting and how often they are attending. In addition, we seek to understand the participants’ responses to the workshops themselves and to do this we designed a 90-second ‘quick impact’ survey, which participants complete directly after attending a session (which has a high response rate of approximately 66%). The short survey is followed up with a longer survey sent by email (which has a lower response rate of around 17%) but allows participants to give more detailed response and features a formative feedback mechanism to enable attendees to feed into the support on offer. We do this to reflect known best practice and so that they can let us know if there is anything else that they need support with (Hattie and Timperley, 2007). However, both these surveys look mainly at engagement on the day or self-reported change. In 2016, to improve the quality of our analysis, we reviewed the mechanisms used to gather impact and feedback of MLE and a series of changes were made. The ‘quick impact’ survey was aligned with the Kirkpatrick model for evaluating training programmes (Kirkpatrick and Kirkpatrick, 2006), the longer email survey was revised to support a more detailed understanding of participants’ responses, and a series of interviews were conducted with PASS (Peer Assisted Study Support) leaders who attended MLE sessions as part of their training. PASS leaders are drawn from undergraduate programmes (years two and above) to support year one students in time-tabled sessions within their schools and were selected for the interviews because of their close connection to the wider support network on campus. All this work enriched our understanding of the service but did little to help us create predictive models or see if that was even possible given the constraints of our data. Seeking to address that lack, we undertook a deeper analysis of the attendee profile data. The review of the MLE feedback and data collection led to partnerships with two key central services: Widening Participation and the Directorate of Planning Support. The partnership with Manchester’s Widening Participation and Recruitment (WP) team meant that we were able to use Discoverer (a tool used internally to analyse datasets) and this enabled us to access more detailed information about the MLE attendee population, e.g.: year of study, mature student status, international status, and whether participants had Blake, Grayson, Karamalla-Gaiballa Investigating impact: Exploring the effect of ‘open’ support on student success papers Journal of Learning Development in Higher Education, Issue 16: December 2019 6 engaged with any widening participation initiatives at a university or school level. As the Directorate of Planning Support has access to official Higher Education Statistics Agency (HESA) return data (which is the official and centralised data-set for higher education institutions in the UK and thus provides raw data in relation to exams and student attainment), we worked with them to analyse the larger MLE attendee data-set to understand its significance and to investigate ways of measuring the impact of MLE as a supportive intervention. Historically, we had struggled to find a methodology that would allow us to make strong statements about the impact of the open workshops on student attainment. As is common across the sector, it has become increasingly important to show robust analysis of services and resources. Through the ‘quick impact’ surveys we asked how participants felt the session had run on the day and whether they had any suggestions for improvement, and these questions are useful for service improvement and management of the programme. However, the surveys did little to tell us what was actually happening to participants after they attended our sessions. Therefore, we could not claim that MLE was helping them to achieve any personal or academic goals; this issue is well-recognised in relation to measuring support of any kind, from the academic support offered via MLE to continuing professional development (Spowart et al., 2017). The new partnerships we developed with the Directorate of Planning Support and the WP team meant that we were able to run three years’ worth of attendance data through Discoverer using their more comprehensive datasets and further understand our attendee population. In addition, by using the HESA data, we were able to connect the student ID numbers of MLE attendees to the degree classifications they obtained. Establishing a link between our attendee population and their results then enabled us to work with the Directorate of Planning Support to conduct a correlation analysis using regression modelling. Working in partnership to quantify impact In an effort to move beyond how participants felt on the day, we decided to focus on the attendance data gathered for each workshop. In order to truly understand the attendance data we were gathering, we worked with the Directorate of Planning Support to devise an appropriate methodology that we could build on in future years and that would signal the Blake, Grayson, Karamalla-Gaiballa Investigating impact: Exploring the effect of ‘open’ support on student success papers Journal of Learning Development in Higher Education, Issue 16: December 2019 7 start of a longitudinal study on the impact of the MLE open programme. The methodology we decided upon constituted a significant reduction in our sample size (reducing it from 8,244 attendees which we saw over the first three years of the programme to 856 attendees who fit the analysable profiles) and the rationale behind this decision was to establish a sustainable process for exploring the impact of students’ attendance at MLE open workshops on their results. However, the historical data excluded from the scope of this report is still available (so that further analysis may be carried out at a later date). The primary objective behind the work done to analyse the attendee data has been to investigate the following questions:  Who we are supporting?  What are the implications?  Can we tie the support to learning, retention, and results?  How do we know? The study uses the first three years’ worth of data gathered from the MLE open programme workshops to present a picture of who is attending (via the demographic details of the attendee population in comparison to the wider university population) in order to assess what effects MLE may be having on attendees (particularly in respect to attainment). We focused on these two aspects in order to understand the ways in which the participants of MLE workshops reflect the make-up of the current student body and to begin to sketch a line between attendance of the skills support workshops and student achievement. The study focuses on:  One cohort of full-time undergraduate students (UGs) enrolled in 2013/14 for the duration of their three-year course.  Three cohorts of full-time postgraduate taught Masters and PGCE students (PGTs) studying for one year (for 2013-14, 2014-15, and 2015-16). Blake, Grayson, Karamalla-Gaiballa Investigating impact: Exploring the effect of ‘open’ support on student success papers Journal of Learning Development in Higher Education, Issue 16: December 2019 8  Two cohorts of full-time postgraduate research students (PGRs) on three-year courses tracking their usage in the first two years of their studies (those who started 2013-14 and 2014-15). Though our initial sample was significantly reduced, using this more limited sample enables the claims we can make about MLE to have more credibility. For example, if we are looking at a full cohort of UGs it is easier to compare those who attended MLE workshops with those who did not, and through the use of regression modelling (which helps to control for other influencing factors such as tariff entry data, disability status, domicile, gender, and mature status) we are able to see whether attendance of MLE workshops results in a higher or lower probability of attendees gaining a first class degree. Because the open nature of the workshops meant typical longitudinal studies would be difficult, we were unable to base our methodology strictly on previous studies. We chose to use a regression analysis because we wanted to explore what it could tell us about the relationships between the variables, and because it allowed us to account for variables that are known to have an impact on attainment. We also sought to better understand the attendees of the workshops by creating attendee profiles. In this case, the three groups analysed were:  Full time UGs enrolled in 2013-14: 362 MLE attendees were compared to the wider university population of 3,669 in this category (Table 1).  Full time PGT Masters (and PGCE) students: 276 MLE attendees (total for three cohorts) were compared to a wider university population of 4,200 (Table 2).  Full time PGR students: 218 MLE attendees (total for two cohorts) were compared to a wider university population of 1,109 (Table 3). First degree 2013/14 entrants. Table 1. MLE users in the first degree entrants 2013 population. Categories Count % of MLE % of UoM population Multiple User 139 38.4% 3.8% Single User 223 61.6% 6.1% MLE User Total 362 9.9% Non User 3307 90.1% University Total 3669 100.0% Blake, Grayson, Karamalla-Gaiballa Investigating impact: Exploring the effect of ‘open’ support on student success papers Journal of Learning Development in Higher Education, Issue 16: December 2019 9 Postgraduate taught full-time students of 2013/14, 2014/15, and 2015/16. Table 2. Postgraduate taught MLE users in the three years. Type of MLE User 2013/14-Count 2013/14-% 2014/15-Count 2014/15-% 2015/16-Count 2015/16-% Multiple User 153 3.6% 185 3.9% 251 5.0% Single User 123 2.9% 172 3.6% 278 5.6% MLE Total 276 6.6% 357 7.5% 529 10.6% Non-User 3924 93.4% 4400 92.5% 4456 89.4% UoM Total 4200 100.0% 4757 100.0% 4985 100.0% Table 3. Postgraduate research MLE users in the two years analysed. Type of MLE User 2013/14 2014/15 Total 2013/14 2014/15 Total Multiple User 46 67 113 8.6% 11.6% 10.2% Single User 49 56 105 9.2% 9.7% 9.5% MLE Total 95 123 218 17.9% 21.3% 19.7% Non-User 437 454 891 82.1% 78.7% 80.3% UoM Total 532 577 1109 100.0% 100.0% 100.0% The MLE population in each area was compared to the wider university population in order to establish whether cohorts such as mature students, international students, and others were proportionally represented in the former. The comparison enabled us to demonstrate that specific groups are receiving support through MLE and that the programme is fulfilling its role as an intervention with strategic importance for the library and the wider university (particularly in respect to groups which the university has identified as a concern in relation to differential attainment). It provides a valuable insight into who accesses the support as well as giving us a chance to demonstrate the impact the programme is having. Matching behaviour to student profiles allows for a more nuanced view of the quantitative data so that we understand not only who is using the face-to-face element of the open programme but also how it is being used. Our analysis can thereby move from summative statements such as ‘all faculties and degree programmes use MLE’ to a much more useful narrative around the fact that certain cohorts, from certain areas, use MLE in a specific way. The first picture is a gratifying one, but the second allows for a much deeper analysis of the effects of the service on the participants. Results of the analysis Blake, Grayson, Karamalla-Gaiballa Investigating impact: Exploring the effect of ‘open’ support on student success papers Journal of Learning Development in Higher Education, Issue 16: December 2019 10 The data for the cohort of undergraduates from 2013-16 was run against the HESA return data and regression modelling was used to control for factors influencing attainment so that MLE attendees could be compared to non-attendees to assess whether attendance at MLE workshops has any influence on degree outcome. The regression analysis focused solely on the undergraduate cohort. In respect to undergraduates, 9.9% of those completing during the cohort analysed accessed the MLE workshops. Those from the School of Social Sciences (SoSS) and Alliance Manchester Business School (AMBS) are over-represented proportionally in the MLE attendee population with the School of Arts Languages and Cultures (SALC) as the most under-represented school. Well-being sessions were the most common workshops (mostly being accessed in the third year of study) but this behaviour coincides with changes in the nature of the support on offer, e.g., when the counselling service became a bigger part of the MLE delivery team in 2014/15. Both international and EU (European Union) students are over-represented in the MLE population; international students are over-represented as multiple users and EU students as both single and multiple users. Students with a known disability are slightly under-represented (6.4% of MLE users compared to 6.9% of wider university population), but mature students are over- represented and more likely to be multiple users. Non-white students are also over- represented as attendees and all ethnic minorities (especially black students) are more likely to be multiple users of MLE. In order to display the results of the regression model, effects plots were produced to illustrate the strength of the impact of an independent variable on the dependent variable, which was two different measurements across two models: 1) Graduating with a first-class degree or not. 2) Graduating with a good degree or lower degree. When looking at figure 1 and figure 2 (see below), there is a point representing the probability of gaining a first-class/good degree and a vertical line which represents how confidently the model can predict the effect that the independent variable has on the offer rate. This is known as the 95% confidence interval, meaning we can be 95% confident the Blake, Grayson, Karamalla-Gaiballa Investigating impact: Exploring the effect of ‘open’ support on student success papers Journal of Learning Development in Higher Education, Issue 16: December 2019 11 actual probability of graduating with a first-class degree/or good degree is within the range of the line. Smaller sample sizes produce wider confidence intervals. After controlling for other factors (such as academic school, disability status, gender, UCAS Tariff, age, and domicile group) the effects plot shows that multiple users of MLE workshops have a higher probability of graduating with a first-class degree than non-users and the difference between the two groups is one which can be claimed to be statistically significant (figure 1). Figure 1. Probability of gaining a first. Figure 2. Probability of gaining a good degree. A regression analysis was also conducted to assess whether multiple users of MLE have a higher probability of gaining a good degree: although there is an increased probability amongst repeat users compared to non-attendees, in this instance, due to the sample size, the difference cannot be claimed to be statistically significant (figure 2). In future attempts at this sort of analysis, we intend to make an effort to include part-time students and investigate whether the presence of careers workshops (which moved out of MLE in 2014) and the inclusion of workshops led by the Disability Advisory and Support Blake, Grayson, Karamalla-Gaiballa Investigating impact: Exploring the effect of ‘open’ support on student success papers Journal of Learning Development in Higher Education, Issue 16: December 2019 12 Service (DASS) (which were designed with the MLE team and began in 2014/15) shift the results. We envisage that there will now be a proportional over-representation of students with a known disability attending the programme, as in 2014/15 DASS began delivering compulsory workshops for students needing to access the service and for those interested in using assistive technology. In addition, we recognise that the cohorts that do not use the face-to-face workshops may be accessing MLE online resources, and thus we need to make sure that our statements around correlation are specific to the face-to-face workshops. Moving forward with greater understanding This research was always intended to act as an investigation into the potential for analysing the MLE open programme as well as considering the impact of the programme itself. With all the difficulties inherent in making correlative (let alone causal) statements around the benefits of skills support (particularly ‘open’ skills support), we wanted to ensure that we were using the data we could gather as effectively as possible. The results of the regression modelling are promising as we can claim that it is likely that MLE plays a positive role in supporting its users to obtain a first-class degree, however, we cannot prove causality. The hope is that with the addition of data for three more years (and thus an increased sample size), we will be able to demonstrate an ongoing correlation supported by a wider sample and will be able to claim statistical significance in relation to an increased probability of those who use MLE gaining a good degree (as well as those gaining a first-class degree). A continuation of this analysis will be a true test of the programme’s contribution to improving students’ performance and attainment. In addition, the work we have done, and our greater understanding of the MLE attendee profile population, has opened up the potential to link MLE to work currently being done to address differential attainment at the university. Answering the questions The impact of open learning support on attainment can be traced to some extent through correlation analyses, but when you have a commitment to resources which are fully open Blake, Grayson, Karamalla-Gaiballa Investigating impact: Exploring the effect of ‘open’ support on student success papers Journal of Learning Development in Higher Education, Issue 16: December 2019 13 (and also online), guaranteeing that some groups have never used the online resources is difficult. The results of this study are limited to the delivery of face-to-face workshops, but we cannot claim with confidence that those who did not attend the workshops had not engaged with other open, online support offered by MLE; it is not possible for us to have a clear ‘control’ group with which to compare those who attended the workshops. In respect to the question ‘who are we supporting?’, the collection and analysis of attendee profile data has enabled us to draw conclusions about who attends and engages with the MLE programme so that we can now answer this question to a much greater extent. In respect to ‘what are the implications?’, and ‘can we tie the support to learning, results, and retention?’, the regression analysis has demonstrated that the probability of obtaining a first-class degree was higher for those who were repeat attendees of the MLE programme so we now understand more about who we are supporting, and what some of the implications are. Furthermore, we now have evidence that MLE is supporting groups that the university is targeting in order to try and address the attainment gap. Our contribution to the support needed in this area demonstrates the value of our work to the wider university (and the UK higher education sector) as we can clearly demonstrate that students who fall into some of the groups most at risk in relation to the attainment gap are engaging with and benefitting from open skills support where they can work in community with one another to develop new skills and further enhance existing ones. The partnerships we have built with the WP team and the Directorate of Planning Support have given us greater insight into the cohorts we are supporting through MLE; the implications of these findings mean that we can now examine areas of under-representation and take steps to address them. In addition, we can effectively demonstrate our value to the academic schools, the student body, and the university as a whole in respect to the benefits gained by those who are self-selecting to attend the support we offer. Which leaves the question ‘how do we know?’. Through the ‘quick impact’ survey, the longer email survey and the deeper analysis of our attendee profile data combined, we can connect the support we offer through MLE to learning and to results, but there is still a lot of work to do to ensure that we are supporting the university and the groups most at risk particularly in regard to retention. Although the links we have drawn and the connections we can demonstrate (so far) rest firmly within the bounds of correlation (and not causality), the strength of the correlations we can demonstrate and our sample sizes are both on the Blake, Grayson, Karamalla-Gaiballa Investigating impact: Exploring the effect of ‘open’ support on student success papers Journal of Learning Development in Higher Education, Issue 16: December 2019 14 rise, and it is our sincere hope that others wishing to create similar programmes might use this study as a supporting factor in the justification of their potential to significantly impact attainment. Our final question remains unanswered for now but we have nonetheless taken some positive steps in relation to measuring our worth and the worth of the work we do for our students. For the future Even with a small sample size, the results of this study have enabled us to demonstrate that MLE is an important part of the support offered to students at the University of Manchester. It has brought us, as learning developers into conversations around differential attainment, teaching excellence, widening access, progression, and curriculum development. It has brought the work that we do out of the library so that we now participate in a number of other key initiatives both university and sector-wide. Because of its commitment to enfranchising the student voice and because of the rigorous work we are doing to evaluate and research the effects of the service, MLE is now part of the university’s official access and participation plan for Widening Participation work. We have begun to move forward with our impact work and we are looking at other methodologies for evaluation. For example, we have used our more recent data to investigate other ways to draw conclusions about correlation between use of My Learning Essentials and other variables, including a Spearman’s rank correlation, a numerical measure of statistical dependence. The initial results are promising and have allowed us to make further supporting statements in relation to the strength of the connection between student attainment and the use of the MLE open workshops. If we had remained content to simply assess how things went or even to understanding merely who attended the MLE open workshops, we could paint an incomplete, if pleasant, picture of what actually happens via our skills support programme. We would have been able to make important claims about who we support and what they think about our work, but we would not have been able to take the next steps and make larger, more generalisable statements about the service as a whole and about the impact an ‘open’ skills programme where students self-select the support they require can have. Further work is needed, and it is hoped that we (and others) will continue this work, but the initial Blake, Grayson, Karamalla-Gaiballa Investigating impact: Exploring the effect of ‘open’ support on student success papers Journal of Learning Development in Higher Education, Issue 16: December 2019 15 steps we have taken have had a positive impact both on how the MLE open support is received by students (and staff) and on how evaluation and impact measures are valued across the university (e.g., our work has prompted similar studies to be undertaken in other areas and we are often asked to consult and support those running the studies). We see the analysis we have conducted as an important addition that supplements and strengthens the story-telling and qualitative work we regularly engage in around skills support and learning development. We need to be gathering data in relation to all those who attend or engage with our support (whether ‘open’ or ‘embedded’), and, where possible, design our programmes to support data collection so that we can share gains we have made and insights we have discovered. We need to work in partnership across our universities and across the HE sector to pool our skills, understanding, and resources, and make the most of our institutional expertise (particularly with respect to data analysts) by working in partnership with other departments to assess whether the systems they use can help us to demonstrate our value and to understand the effect that the support we deliver is having on those who attend it beyond what they report through feedback. Learning developers can make a positive contribution to the conversations around metrics that are currently happening (in relation to resources and funding) and in order to strengthen our position we need to share what we have learnt and collaborate across institutions. If we work together to connect and share evaluation measures across the sector, we can collectively show the impact of the type of support we offer, allowing us to demonstrate the extent to which it is valued in community. Often student support seems relegated to something that is ‘nice to have’ or is ‘only for certain cohorts’. We wish to challenge these assumptions by arguing that any student, regardless of their background or degree programme, should have access to support and services which can make their time at university a success; we can play a role in maximising their chances of success by giving them the power to self-select the support they need, when they need it. References Blake, J. and Illingworth, S. (2015) ‘Interactive and interdisciplinary student work: a facilitative methodology to encourage lifelong learning’, Widening Participation and Lifelong Learning, 17(2), pp. 108–118. Blake, Grayson, Karamalla-Gaiballa Investigating impact: Exploring the effect of ‘open’ support on student success papers Journal of Learning Development in Higher Education, Issue 16: December 2019 16 Fisher, D. and Frey, N. (2013) Better learning through structured teaching: A framework for the gradual release of responsibility. 2nd edn. Alexandria, VA: ASCD. Grayson, N. J., Blake, J. and Stock, M. (2018) ‘The co-creation of exam support: Students as partners in the research, planning, design and quality assurance of learning resources’, The Journal of Educational Innovation, Partnership and Change, 4(1). Availabe at: doi: 10.21100/jeipc.v4i1.767 (Accessed: 10 December 2019). Hattie, J. and Timperley H. (2007) 'The power of feedback', Review of Educational Research, 77(1), pp. 81-112. Howard-Jones, P., Ioannou, K., Bailey, R., Prior, J., Yau, S. H., & Jay, T. (2018). 'Applying the science of learning in the classroom', Impact: Journal of the charted college of teachers, 18(2), p. 19 Kirkpatrick, D. L. and Kirkpatrick, J. D. (2006) Evaluating training programs: the four levels. 3rd edn. San Francisco: Berrett-Koehler. Mortimore, T. and Crozier, W. R. (2006) ‘Dyslexia and difficulties with study skills in higher education’, Studies in Higher Education, 31(2), pp. 235–251. Spowart, L., Winter, J., Turner, R., Muneer R., Mckenna C. and Kneale P. (2017) ‘Evidencing the impact of teaching-related CPD: beyond the “Happy Sheets”’, International Journal for Academic Development, 22(4), pp. 360–372. Wingate, U. (2006) ‘Doing away with “study skills”’, Teaching in Higher Education, 11(4), pp. 457–469. Author Details Jennie Blake is the Learning Development Manager at the University of Manchester Library. She is a Senior Fellow of the Higher Education Academy and a National Teaching Blake, Grayson, Karamalla-Gaiballa Investigating impact: Exploring the effect of ‘open’ support on student success papers Journal of Learning Development in Higher Education, Issue 16: December 2019 17 Fellow and has taught and researched in education for nearly twenty years. She is particularly interested in the power of collaboration and inclusive and accessible materials and teaching to widen participation and support student success. Nicola Grayson is a Learning Developer with expertise in measuring impact and researcher development. She is a Senior Fellow of the Higher Education Academy and works as an Associate Lecturer teaching Philosophy in addition to supporting the design, delivery, and evaluation of the University of Manchester Library’s learning programmes. Sami Karamalla-Gaiballa is a data analyst in the Equality, Diversity and Inclusion team at the University of Manchester. He focuses on statistical analysis and building models which help identify areas of underrepresentation of protected characteristics, potential bias in decision making and evaluation of equality policy in relation to student attainment as well as participation, employees’ progression, and research strategy. Investigating impact: Exploring the effect of ‘open’ support on student success Abstract Introduction The tension between openness and demonstrable impact Working in partnership to quantify impact Results of the analysis Moving forward with greater understanding Answering the questions For the future References Author Details