Reviewed Articles - Teaching and Learning in Clinic ENHANCING EMPLOYABILITY THROUGH STUDENT ENGAGEMENT IN PRO BONO PROJECTS Sarah Blandy, University of Sheffield, UK*[footnoteRef:1] [1: *Sarah Blandy is Professor of Law in the School of Law, University of Sheffield. The author would like to offer thanks for the funding provided by the University of Sheffield; to Amy Murphy and Emilie Sylvester, the final year students who administered the surveys and made a preliminary analysis of the results; to all the members of staff in the School of Law who assisted in compiling lists and encouraging survey responses; and not least to the students and alumni who completed the survey.] Abstract This paper discusses the findings of a survey carried out by the School of Law at the University of Sheffield, placing it in the context of international research on links between student participation in pro bono projects, and employability. The aim of this survey was to establish whether students’ pro bono experiences assist them in obtaining training and employment. Over the summer of 2016 a survey was sent to current students and to alumni who were (or had been) volunteering at one of the two longest-established pro bono projects run by the School of Law. The paper explains how the survey was designed, conducted and analysed, and discusses the methodological issues which arose. Although the original aims of the research were not achieved, and perhaps could never have been, the responses to the surveys yielded very useful and rich data. No direct questions were asked about skills development, but the respondents’ unanticipated and unsolicited qualitative comments can be positively mapped onto the key skills and attributes that constitute ‘employability’. The findings set out here therefore add to the small amount of existing literature about student perceptions of how their experiences as pro bono volunteers assist them through placement, training and employment application processes. INTRODUCTION An increasing amount of time, effort and resources is invested in university law clinics. In 2014 a survey carried out for LawWorks[footnoteRef:2] showed that over 70% of UK law schools were then running some form of pro bono legal clinic involving around 10,000 students; and 35% offered more than one type of clinic (Dignan et al, 2017, pp. 3 and 4). This activity was broadly defined as ‘a structure that delivers pro bono work that is organised (but not necessarily delivered) by a law school’ (ibid; p. 3). Although university law clinics in the UK are unregulated (Thomas, 2017), they aspire to the highest professional standards. The work of 79% of these clinics is supervised by ‘a qualified barrister or solicitor with a full practising certificate – be it a member of academic staff or a lawyer external to the law school’ (Dignan et al, 2017, p. 8). [2: LawWorks is a UK charity that aims ‘to connect people in need of legal advice with the skills and expertise of lawyers willing to meet those needs for free, by supporting a network of local independent pro bono clinics’. https://www.lawworks.org.uk/ [accessed 29 August 2018].] The School of Law at the University of Sheffield, a Russell Group[footnoteRef:3] university in the UK, is no exception to this trend. Its student pro bono opportunities have been expanded and enhanced over recent years, at least partly because of an underlying assumption that these opportunities enhance students’ career prospects. This paper reports on a survey carried out in 2016 which aimed to explore the links between student experiences of pro bono volunteering and their employability. The term ‘employability’ has been defined as a ‘set of achievements, understandings and personal attributes that make individuals more likely to gain employment and to be successful in their chosen occupations’ (Yorke, 2006: 8). Although, as explained later, Sheffield law school offers a wide pro bono programme, the survey only included those current students and alumni who were volunteering, or had done so, at the two longest-established pro bono projects. This was in order to capture any differences amongst those with similar experiences, over ten years. [3: The Russell Group comprises 24 leading, research-intensive UK universities; see https://russellgroup.ac.uk/ [accessed 10 February 2019].] The paper is structured as follows: further details are given about Sheffield university’s pro bono programme and the opportunities it offers to law students, setting this programme within UK and international contexts. Then the existing research literature on links between CLE or pro bono and employment is reviewed, followed by an outline of research and guidance about skills and attributes linked to employability. The paper then discusses the methodological issues raised when designing research into the impact of pro bono involvement on student employability, before turning to the Sheffield survey and a presentation of its research methods, challenges and findings. In the final section of the paper some conclusions are reached. PRO BONO OPPORTUNITIES AT SHEFFIELD Although a large majority of UK law schools offer their students the opportunity of involvement with practical law projects, the form that these projects take is varied. All university law pro bono projects must strike a balance between offering an acceptable standard of service to clients, and enhancing the learning opportunities for their students. However, there is no consensus amongst university law clinics as to whether students’ CLE activities should be part of their assessed degree programme, or whether involvement should be voluntary. Similarly, practice and opinions are divided as to whether any module associated with pro bono work should be compulsory or elective. The terms used are also varied. University-run projects are often referred to as ‘clinical legal education’ (henceforth ‘CLE’), because they are embedded in the curriculum, and fully assessed. For a full discussion of the differences between CLE and pro bono, see Corker (2005) and Cantatore (2015). However, the Sheffield pro bono programme does not conform to all the characteristics identified by these authors, in particular, the emphasis on ‘community service’. At Sheffield the term ‘pro bono’ has always been used, rather than CLE. In 2008 the two first Sheffield pro bono projects were established: the FreeLaw clinic, which offers advice on civil legal matters, and the Miscarriages of Justice Review Centre (the ‘MJRC’, formerly known as the Innocence Project). The School now allocates around two hundred places a year at ten pro bono projects through a very competitive selection process for student volunteers (both undergraduate and postgraduate). Four projects are run in-house and the School has established formal links with six externally-run projects, which include the Citizens Advice Bureaux and the Sheffield Refugee Law and Justice project. All the School’s pro bono projects are extra-curricular. However, an optional assessed module has been run for final year undergraduate students volunteering at FreeLaw or MJRC. From 2016-17 this module has been made available to students volunteering at other pro bono projects run by or associated with the School of Law. The module is assessed in part through portfolios in which students reflect critically on the skills developed through their pro bono experiences, a widely accepted form of assessment amongst university law schools that offer pro bono projects or CLE (Hyams, 2010).[footnoteRef:4] [4: Students also have to make a presentation to their seminar group about the development of one of those skills; and write an academic essay about the context in which legal advice and support is provided.] It may be useful to provide some further detail about how FreeLaw and the MJRC operate, to make sense of the survey responses from current or former student volunteers at these two projects. The Sheffield model of pro bono differs from that of many other universities because it is student-run. At both MJRC and FreeLaw, two student managers have overall responsibility for the work of the other student volunteers under the guidance of the part-time Co-ordinators of each project, who are members of the Law School staff. The students are organised into groups, with more experienced pro bono volunteers acting as group leaders who are responsible for the management of their group’s cases. The criminal cases taken on by the MJRC concern convictions for serious crimes (mostly murder). Students work on their group’s case, reviewing the large amount of accompanying documentation, with the ultimate aim of persuading the Court of Appeal to review the case and consider overturning the client’s conviction. The MJRC project director attends weekly meetings of the student teams, and guides their research. In comparison with the MJRC, the Freelaw clinic deals with a large number of civil cases, but these tend to be much less complex. The clinic does not offer advice on financial issues, welfare benefits and immigration problems, but takes on a wide range of other non-urgent civil matters. Most UK university law clinics adopt one of two alternative models. The first model can be described as running a 'drop-in' clinic, in which clients attend without appointment and are interviewed by students who usually do not know the nature of the client’s problem in advance. In the second model, the clinic’s cases are 'pre-vetted': potential clients telephone or email in advance, often during a restricted period of time immediately before each semester, and a member of academic staff decides which cases are suitable to take on. In both of these models, students interviewing clients are often supervised by university staff or by external lawyers who assist the project on a volunteer basis. FreeLaw has adopted the ‘drop-in’ clinic model. However, no supervision is provided for student interviews with clients, either by members of staff or by external lawyers involved with the project. Students are instructed never to offer advice at the interview. They take down factual information from the client, and obtain further relevant details and documentation. The students then research the legal issues and may seek guidance from the Co-ordinator or other members of law school staff. Letters of advice drafted by students are uploaded to the Case Management System. One of the FreeLaw Directors, both of whom have practising certificates and are covered by professional negligence indeminity insurance, provide comments on accuracy, content, structure and clarity before giving final approval for the advice letter to be sent to the client. The aim is for this advice to be provided within fourteen days. RESEARCH INTO LINKS BETWEEN CLE / PRO BONO AND EMPLOYMENT One of the aims of university CLE and pro bono projects is to facilitate students’ development of ‘lawyering skills’ and professional responsibility (see for example Foley et al, 2012). However, surprisingly little research has been published on the issue of whether involvement in CLE or pro bono experience assists with developing students’ employability in reality, despite ‘the increasing importance seemingly attached to the … ever-present employability agenda’ by university law schools in the UK (Carney et al, 2014, p. 33). One explanation for this might be that, when asked about the relative importance of the reasons for running pro bono projects, nearly all (94%) of the law schools which took part in the 2104 LawWorks survey rated ‘educational value’ as a very important aim, as compared with 75% who rated ‘employability’ similarly highly (Dignan et al, 2017, p. 4). A systematic search of the past ten years’ issues of the International Journal of Clinical Legal Education found no articles on this topic which related to the UK, although articles had been published on the link between employability and professional skills development by law students in Australia (Cantatore, 2018) and in Nigeria (Mokidi and Agbebaku, 2012). The recent article by Cantatore (2018) sets out the findings from the author’s survey of two cohorts of law students at Bond University, Australia. The first group were involved for one semester in pro bono activities, and the second were not. This research provides interesting points of comparison with the Sheffield survey, as it uses the same definition of employability (Yorke, 2006) and measures responses against the skills listed in the Australian Graduate Employability Survey (Oliver et al, 2011) supplemented by work by Kuh (2001) in the US, and by Coates (2009) in Australia. Cantatore’s skills list is similar to the graduate skills and attributes compiled by the Higher Education Authority in the UK (see Buckley, 2015), which provides one of the measures referred to in this article. This list is set out in a subsequent section, and more detailed methodological comparisons with Cantatore (2015) will also be discussed subsequently. CLE is well-established in US universities[footnoteRef:5]; over many years research has been published asserting the benefits of CLE for student employability. More recently, research by Yackee (2015) surprisingly suggested the opposite. However, Kuehn (2015) has comprehensively critiqued Yackee’s statistical findings, pointing out a fundamental confusion between correlation [two factors appear to develop at the same time and follow the same trajectory] and causation [one factor has caused the other]. Kuehn then goes on to discuss other forms of evidence, including the fact that many employers in the US ‘identify law clinic experience as a positive factor in hiring’, and that new employees give weight to their clinic experience as a factor in successful job applications (ibid., p.660), which support the widely-held view that CLE enhances student employability. Kuehn’s work will be referred to again in the discussion of methodological issues. [5: It should be borne in mind that in the USA, Law courses are open only to postgraduates.] In the UK, currently there seems to be only anecdotal evidence that UK students’ employability is enhanced through pro bono projects, and through the self-reflection about skills development which is required by the associated modules offered at many UK universities, including Sheffield. Only one systematic attempt in the UK to collect evidence of the link between pro bono experience and employability has been identified, the survey carried out by the Personal Support Unit (PSU)[footnoteRef:6]. The PSU runs an ongoing national survey targeted at its ‘student alumni’ who have been volunteers. Analysis of responses to the PSU’s 2015 survey, which is the most recent available, found that 87% of the respondents were in employment. Of the 69% of former PSU volunteers now employed in the legal sector, all felt that their experience as a PSU volunteer had helped them to secure either pupillage, or a training contract or employment (PSU, 2015). [6: See https://www.thepsu.org/volunteer/student-alumni-survey [accessed 10 February 2018]. The PSU is a national charity which provides assistance at family and civil courts to litigants in person, witnesses, and other inexperienced court users. The PSU in Sheffield is one of the external pro bono projects which selects students from the School of Law as volunteers.] A valuable recent UK research project focused on the perceptions of legal employers about CLE, rather than on student experiences (Thomas, 2018). The graduate recruitment webpages of the top 50 UK law firms were analysed. None of these included CLE (or pro bono) as an example of experience sought in future trainee solicitors, although ‘nearly three-quarters’ of those firms mentioned ‘their own pro bono and/or corporate social responsibility activity’ (ibid. p.138). Telephone interviews were also carried out with recruiters from two barristers’ chambers, a law centre, six of the top 25 UK solicitors firms, seven international firms, ten regional firms, and five legal aid organisations. These revealed that employers ‘certainly were not familiar with the term “clinical legal education”’ (ibid., p.137). However, once provided with some information about CLE, the recruiters could easily identify skills and attributes likely to be developed in students taking part. The majority of interviewees seemed to understand CLE as similar to a placement at a law firm or chambers, but considered that taking part in CLE would provide a more valuable experience as students had direct contact with clients. Thomas concludes that the term ‘pro bono’ should be adopted instead of CLE, and that universities should do more to educate legal employers about their clinics and the professional standards adopted in them. She also suggests that students should be helped to reflect on their skills development through involvement in pro bono, and how to present this most effectively to prospective employers (Thomas, 2018). This overview of the research literature on the development of employability by students involved in CLE and pro bono indicates that there is scope for more research in this area, particularly in the UK. It is in this context that the Sheffield survey and findings are discussed in this paper. First, the definition of employability itself is considered. EMPLOYABILITY AND SKILLS DEVELOPMENT A general survey of UK employers found a gap between their expectations of trainees and employees and the reality, for both ‘hard’ and ‘soft’ skills: ‘Skills related to operational aspects of the role, as well as complex analytical skills, were the main technical and practical skills lacking. The main people and personal skills lacking pertained to time management, management and leadership, and sales and customer skills’. (UK Commission for Employment and Skills, 2016, p.4). Although presented at a more abstract level, a similar emphasis is apparent in the ranked ‘top ten skills’ which global employers will be seeking in 2020: 1. Complex problem solving 2. Critical thinking 3. Creativity 4. People management 5. Coordinating with others 6. Emotional intelligence 7. Judgement and decision-making 8. Service orientation 9. Negotiation 10. Cognitive flexibility (World Economic Forum, 2016; henceforth ‘WEF’). It is noticeable that in this list attributes such as creativity, and ‘people skills’ such as emotional intelligence and the ability to co-ordinate with others, take priority over hard skills. Indeed, in relation to cognitive skills, the most desired attribute is apparently flexibility rather than detailed subject knowledge. These observations of employers in general are important, particularly as law graduates do not all seek, or are able to obtain, employment in the legal sector. According to the UK Higher Education Academy, all university graduates should have acquired the following skills during the course of their degree: · Thinking critically and analytically · Writing clearly and effectively · Speaking clearly and effectively · Analysing numerical and statistical information · Independent learning · Innovation and creativity · Working effectively with others · Developing and clarifying personal values · Understanding people of other backgrounds · Exploring complex real-world problems (Buckley, 2015). Interestingly, although all the above skills can be seen as contributing to employability, the Higher Education Academy’s UK Engagement Survey treats ‘acquiring employability or career skills’ as an additional, stand-alone skill. Analysis of responses from more than 35,000 undergraduates to the 2017 survey found that: ‘The one exception to the [upward] trends on skills development is the decline in the proportion of students who are acquiring career skills, which has fallen from 51% last year to 49% this year’ (Neves, 2017, p.14; emphasis added). The HEA survey established that employability skills, as well as engagement in real-world problems, were better developed in students who participated in extra-curricular activities (Buckley, 2015, p.21). A similar effect was found in students who volunteer, leading to a particularly marked positive difference in ‘being an informed and active citizen, developing personal values, and acquiring employability skills’ (ibid., p.23). It seems therefore that volunteering at pro bono projects as an extra-curricular activity should enhance students’ acquisition of key skills. Further, according to the HEA survey (ibid.), students’ positive perceptions of their skills development were closely related to four other factors: · collaboration with other students; · interaction with teaching staff; · engagement in research; and · reflection on what they had learned. These are all features of pro bono volunteering at Sheffield. The way that FreeLaw and MJRC are organised requires collaboration with other students to work on the cases in groups. Student interaction with staff is an essential aspect of these pro bono projects, whereas the 2017 HEA survey found that only 19% of all students had worked with staff outside the requirements of their degree course, a reduction from previous years (Neves, 2017, p.26). Engaging in research, as student volunteers do when working on pro bono cases, is a particularly important activity which has ‘the strongest relationship with developing the skill of being innovative and creative’ (Buckley, 2015, p.17). Creativity is one of the top three skills identified by global employers, as listed above (WEF, 2016). The optional pro bono module offered at Sheffield requires critical self-reflection, and the students also benefit from individual supervision sessions with their tutor, which is an integral aspect of the teaching methods for that module. Employers in the legal sector are looking for ‘hard’ skills such as subject-specific knowledge, as well as the more general, transferable or ‘soft’ skills already mentioned. For example, Hammad Akhtar, a graduate recruitment and corporate partner at Ashurst solicitors, told The Guardian UK 300 (the group of top 300 graduate employers, as voted for by students) that he looked for ‘technical legal ability, commercial instinct and analytical ability… communication and team working skills … and motivation, determination and drive’ (TARGETJobs, 2018), an interesting combination of hard and soft skills and attributes. The UK’s Solicitors Regulation Authority reflects this combination in their Statement of Solicitor Competence, which is organised under the broad headings of ‘Ethics, professionalism and judgement; Technical legal practice; Working with other people; Managing themselves and their own work’ (Solicitors Regulation Authority, 2015). For many law students, engagement with the legal profession starts at university when they apply for (and undertake, if selected) vacation placements and mini-pupillages. Most law schools in the UK provide tailored careers advice, but information is not available about the content of this advice, nor about the proportion of law students who take advantage of it. Anecdotally, however, it is well-known that many students make use of the online advice that is available for those seeking training and employment in the legal sector. The following skills and attributes, presented as those sought by commercial law firms, appear in many similar online sources: · commercial awareness · attention to detail · time management · resilience and self confidence · organisational skills · genuine interest in and broad knowledge of the law · sensitivity and integrity · articulacy · discretion and trustworthiness (list compiled from Smith (2017); and https://www.monster.co.uk/). The literature on the benefits of CLE claims that it provides students with the opportunity to develop most of the skills and attributes discussed in this section (see for example Thanaraj, 2016). The aim of the Sheffield survey was to see if a link could be established between involvement in a pro bono project, development of these skills and future employment. The next section of the paper provides an overview of the methodological issues raised by this type of research. RESEARCH DESIGN AND METHODOLOGICAL ISSUES Kuehn (2015) discusses the methodological difficulties of designing research which might establish a statistically significant correlation between a student’s involvement in any given US university law clinic and their subsequent employment. He identifies problems of ‘conjecture about cause and effect, uncertainty about the underlying data, variability from year-to-year, conflicting control variables, and differences among schools and their employment markets’ (ibid.; 663). He suggests that instead of ‘attempting to create a nationwide predictive model, studies could best be done on a school-by-school basis by, for example, surveying likely employers to find out what educational experiences of students are most valued. Schools also could retrospectively look at various employment outcomes for graduates and any relationship to their experiences while students.' (ibid., 664) As noted above, Thomas (2018) has carried out a survey of prospective UK employers. The Sheffield survey was designed to meet Kuehn’s second point about graduate employment and the link between these outcomes and experiences as students. In addition we wished to find out more about the experiences of current students who apply for vacation placements and training contracts while still at university, and the outcomes of those applications. Therefore there were two cohorts whom we wished to reach in the Sheffield survey: the first consisting of all students currently volunteering at FreeLaw or MCRJ; the second of alumni who had volunteered at either project during the previous ten years. Once ethical approval had been obtained, two students were recruited to assist with administering the survey. SurveyMonkey[footnoteRef:7] was chosen for the survey instrument as it provides a free online survey with built-in tools for analysing the data and for creating graphic representations of the quantitative results. Contact details were compiled from various sources, resulting in 105 names on the list for the first cohort (current students), and 136 for the second cohort of alumni. A standard form of words was prepared, introducing the project and asking respondents to complete the survey, with links to SurveyMonkey and to the information sheet for the project. This message was emailed to the first cohort, and sent through LinkedIn to the second cohort, during summer 2016. The surveys remained open for three months, from May to July. The survey’s substantive questions asked each participant whether they (had) volunteered at Freelaw or at MJRC, and if so for how many years. The respondents were asked for their views on whether ‘volunteering at the pro bono project [or taking the pro bono optional module] while at the School of Law was a help in gaining training /employment in your chosen field [and/or] in securing any training / employment’. Respondents were invited to explain each of their answers in a ‘free-text box’ on the survey form. [7: See https://www.surveymonkey.com/ [accessed 10 February 2019].] Problems with the research design for some participants in the first cohort became apparent when analysing the responses. For example, some respondents replied ‘no’ to the question about training, but said in answer to the question about employment that volunteering with FreeLaw had helped them to obtain a vacation scheme place. There was obviously some ambiguity about whether vacation schemes were perceived as training or as employment, which had not been foreseen when drafting the questions. With hindsight, the substantive questions for the first cohort should have asked separately about vacation placements and training contracts, and asked separate questions about training and employment outside the legal sector. Further, some current students seemed to have answered the question ‘Has pro bono volunteering helped in securing employment?’ on the basis of what they believed would be the case in future, rather than on the basis of their actual experience. Similarly, some participants in the first cohort gave positive responses to the question about whether the module had assisted them in securing training or employment, when it was clear from a previous answer that they had not yet taken the module. The final response rates were 22% for the first cohort (23 full responses) and 21% for the second cohort (29 full responses). The response rate for the first cohort was disappointing, as these were current students who received two further emails prompting them to complete the survey before its closing date. However, the survey was not well-timed for this cohort as it coincided with revision and assessment periods followed by the vacation. The low response rate for the second cohort was anticipated, as they were alumni who had left university some time ago. It might have been possible, with considerable time and effort, to obtain responses to the Sheffield survey from almost all the listed individuals in both the first and second cohorts. However, given the resources available, this was not feasible. The research design allowed for the results to be analysed by various variables: according to whether each respondent had been involved in the MJRC or in FreeLaw; the length of time they had been a volunteer; and (for the second cohort) their year of graduation. However, as the overall numbers were so low, this depth of analysis was not considered worthwhile. In both cohorts there were more survey participants from FreeLaw than from MJRC; there are over twice as many student places available each year at FreeLaw (currently 80 places) as at MJRC (currently 45 places). In the survey’s first cohort there were 16 participants who were FreeLaw volunteers and 7 MJRC volunteers. As this cohort’s experiences are more recent we thought it may be of interest to distinguish their responses between FreeLaw and MJRC, but it seemed irrelevant to distinguish between the second cohort respondents by project, as so few of them had volunteered at the MJRC. The most significant contrast with the surveys carried out by the PSU (2015) and by Cantatore (2018), was that the Sheffield survey did not include any direct questions about development of skills or attributes. The PSU survey asks ‘What benefits did volunteering with the PSU have on your life?’, to be answered on a yes/no basis against eight pre-set responses. These are set out below, with the percentage of respondents who agreed that their experience of volunteering with PSU had: · Improved client skills (82%) · Raised awareness of the diverse range of people who require legal advice (76%) · Improved listening skills (71%) · Improved understanding of the legal system (71%) · Practical application of what I learnt on my law course (68%) · Raised awareness of how important it is to get legal advice (66%) · Provided good first-hand examples of real life experience for job applications and/or interviews (62 %) · Provided direct exposure to a range of court cases (59%) (PSU, 2015). The PSU survey report does not disclose response rates, making it difficult to assess the significance of, for example, 82% of the total when this could represent 100 or 1,000 respondents. Further, surveys from the years subsequent to 2015 have not been made publicly available although the survey is still ongoing, so no conclusions can be drawn about any trajectory over time. It is also instructive to compare the research design of the project described by Cantatore (2018). Her survey was administered twice, in weeks four and 12 of the semester in which students were taking part in pro bono projects, to capture an increase or decrease in each respondent’s perception of their level of competence in relation to specified key skills (the list discussed earlier). There were 33 respondents, participating in four different pro bono projects. The same survey was administered to a control group of 34 students who were not taking part in pro bono projects. These two comparisons, over time and between groups, enable Cantatore to demonstrate that the pro bono group of students showed a far higher average increase in self-assessed competence than the control group, over the same period of time. The students involved with the Commercial Law Clinic are reported as showing a higher average increase over time than volunteers at the other projects. However, this last finding cannot be considered robust due to the low numbers (either four or five) participating in each of the other projects. The use of a control group is excellent research practice, but in reality it would be very difficult to set up a genuine control group for such research. The students who participate in CLE or pro bono projects which are not a compulsory element of their degree programme, tend to be more motivated than their peers. If there is in addition a selective recruitment process, as at Sheffield, the chosen students will already have acquired more impressive and relevant skills, including the ability to learn quickly when given the opportunity. These factors may at least partly explain the difference in average increases in self-assessed competence, between the pro bono students and members of the self-selected control group in Cantatore’s research (2015). A genuine control group could only be achieved at Sheffield by recruiting twice as many students as needed for the pro bono programme over a period of several years, without allowing this to become public knowledge. Each year the group of selected students would then be randomly divided into two. The first half would be allocated to their chosen pro bono project, and the second half would be denied that opportunity.This would of course be wholly unethical. All students, from both groups in each year, would be followed for the next ten years to research the comparative impact on their employability of (non-)involvement in pro bono programmes at university. Cantatore (2018) incorporated a more qualitative element in her research. Once students had completed their semester at a pro bono clinic, they were invited to complete an anonymous SurveyMonkey survey about their experiences ‘and to provide detailed feedback if they choose to do so’ (ibid., 164). Although neither the response rate to the survey nor how many students chose to provide feedback in this way is made explicit in the paper, the survey provides useful information about student perceptions of their own skills development. Several illustrative extracts from students’ responses are included in the article, and cross-referenced to skills such as ‘knowledge, writing, speaking and problem-solving’ (ibid., 167-168). All the Sheffield survey respondents were also self-selected – as indeed were the PSU survey respondents (PSU, 2015). Self-selected research participants are often those who are motivated to complete the survey because they hold strong views on the topic, and/or because they feel indebted in some way to the people or institution conducting the survey; the research outcomes may therefore be biased (Sterba and Foster, 2008). Thus the Sheffield survey captured the views of a minority of students who have volunteered at the law school’s pro bono projects over the past ten years. The survey respondents were likely to be students and alumni who felt they had gained from the experience of pro bono volunteering. This does not mean that their views are not valuable, rather that they are not necessarily representative of all past and current student volunteers at FreeLaw and MJRC. In particular it should be noted that the views expressed in respondents’ own words and quoted in this report have been taken primarily from those who felt that pro bono volunteering had assisted them in their progression towards a career, or in obtaining promotion within that career. It could therefore be assumed that these students and alumni are the success stories. Bearing that caveat in mind, a number of important findings can nevertheless be drawn from the survey results about the link between pro bono volunteering at Sheffield, and employability. ANALYSIS OF THE SHEFFIELD SURVEY RESPONSES For the methodological reasons discussed above, reliable quantitative data could not be derived from this survey. Instead, its value derives from the very rich data collected on respondents’ views about their pro bono experiences. In the absence of direct questions about skills and attributes associated with employability, it was very striking that most respondents, from both cohorts, chose to expand on this aspect of pro bono volunteering in their explanations. They used their own words without any suggestions or prompting, in contrast to the PSU survey where participants must agree or disagree with set statements, and to the quantitative element of Cantatore’s research in which students were asked to assess their competence against a stated list of skills and attributes. The comments reproduced here come from the survey participants who specifically gave their consent for their words to be published, anonymously. The quotes can be identified by the following abbreviations: ‘FL1’: current FreeLaw volunteer, in the first cohort of the survey; ‘MJRC1’: current MJRC volunteer, in the first cohort of the survey; ‘PB2’: previous volunteer at either of the two pro bono projects, now graduated; in the second cohort of the survey. It was clear from many of the survey responses that pro bono volunteering allows students to discover and experience for themselves the long-established difference between ‘law in books’ and ‘law in action’ (Pound, 1910). Current students are very appreciative of the opportunity to get involved in ‘authentic legal cases’ (FL1). One respondent wrote: ‘This is serious work, working with proper documents, dealing with actual people's lives’ (MJRC1). Another commented positively that ‘it [MJRC] creates quite an "employment" atmosphere’, explicitly making the link between pro bono experience and employability. These views mirror those expressed by Cantatore’s research participants (Cantatore, 2015 and 2018). The pro bono projects at Sheffield provide a rare opportunity for those students selected as student managers and group leaders to develop skills in managing people: ‘I’ve learnt to delegate appropriately and take control in situations when necessary, but still have awareness for the voices and opinions around me’ (MJRC1). An emphasis on teamwork and organisational skills was particularly noticeable in respondents from the first cohort who were MJRC volunteers. One respondent commented on having grasped the importance of ‘relying on others and working with others to ensure a task is done. You need to learn to value the opinions of others and listen to what they have to say, accepting that your way may not always be the best (increasing trust).’ (MJRC1). Many in the second cohort of respondents are now in employment and can look back with some perspective on the skills they developed through pro bono volunteering. One described these as ‘very transferable’ (PB2), while another participant commented that ‘I took these skills with me to my following roles after university’ (PB2). The value of the ‘opportunity to get advice from practitioners’ (PB2) as a pro bono volunteer was cited by a second cohort respondent, but not by any of the first cohort participants who may not yet appreciate the benefits, which the HEA survey results indicate are particularly significant in developing employability skills (Buckley, 2015). Some second cohort participants compared their pro bono experience positively with low-level posts in the legal sector; for example, one respondent commented on the value of having ‘experience in case management which you don't get as a legal secretary or junior paralegal’ (PB2). Such posts can be an important stepping stone towards obtaining a training contract. Analysis of the free-text answers made it possible to map the skills and attributes which were mentioned onto the lists previously discussed (WEF, 2016; Buckley, 2015; Smith (2017). The participants in the Sheffield survey may not have used exactly the same term for each particular skill, so some interpretation has been necessary in the mapping process, and of course there is no suggestion that any one participant mentioned all the listed skills. Table 1: Comparison of skills mentioned by participants in the Sheffield survey, with skills listed by the World Economic Forum (WEF), Online legal careers advice, and the UK Higher Education Authority (HEA) WEF Survey Online careers advice Survey HEA Survey Complex problem solving Commercial awareness x Thinking critically and analytically Critical thinking Attention to detail Writing clearly and effectively Creativity Time management Speaking clearly and effectively People management (for some) Resilience and self-confidence Analysing numerical and statistical information x Coordinating with others Organisational skills Independent learning Emotional intelligence Genuine interest in, and broad knowledge of the law Innovation and creativity Judgement and decision-making Sensitivity and integrity Working effectively with others Service orientation Articulacy Developing and clarifying personal values Unclear WEF Survey Online careers advice Survey HEA Survey Negotiation Unclear Discretion and trustworthiness Understanding people of other backgrounds Unclear but probably Cognitive flexibility Exploring complex real-world problems Becoming an informed and active citizen Unclear but probably Definite evidence from survey responses 9/10 8/9 7/11 Probabable 1/10 3/11 Total 10/10 8/9 10/11 ‘Analysing numerical and statistical information’ and ‘Commercial awareness’ were the only two skills and attributes listed above which were not specifically mentioned in survey responses. This is understandable given that student volunteers at the Commercial law clinic run at Sheffield were not asked to take part in the survey. However, the respondents from FreeLaw and MJRC were aware that they were acquiring ‘important skills that can be transferred into a commercial law environment, e.g. filing, teamwork, leadership and working to a deadline’ (MJRC1). In terms of developing self-confidence, cognitive flexibility and problem-solving skills, one respondent noted that ‘FreeLaw opens you up to various areas of law and pushes you to work outside your comfort zone’ (FL1). In addition to the skills listed, the survey respondents also identified a range of ‘hard’ skills relevant to legal careers, such as legal research, dealing with clients, interviewing, drafting letters and legal documents. LINKING SKILLS TO EMPLOYABILITY As explained above, it was not possible - given the low response rates and the sometimes contradictory responses from participants in the first cohort - to derive any statistically reliable correlation between volunteering at either pro bono project and the respondents’ subsequent training or employment. However, quantitative results from the first cohort survey results show that 91% (21 out of 23) considered that pro bono volunteering had assisted them in securing placements, training, employment or promotion. For the second cohort the comparable rate was 86% (25 out of 29). This difference could be explained by improvements in the organisation and delivery of the pro bono projects by the law school year on year, or it might be a result of the optimistic expectations of current students in the first cohort who had not read the question accurately. Survey respondents were asked about their future, current and past placements, training and employment, although a significant proportion chose not to answer this question. In the first cohort, several top UK and international law firms were mentioned. Respondents in the second cohort reported on a range of employment. Three were working as qualified solicitors; thirteen as trainee solicitors, and five in a paralegal capacity - of whom two would be shortly moving on to training contracts. Another respondent from this cohort was a postgraduate researcher; one was working as a policy adviser, and another as a teacher. The law firms mentioned by participants from the second cohort included four of the five law firms with London headquarters, informally known as the Magic Circle: the top performers in terms of profitability, revenues and international work. Several survey participants felt that their pro bono experiences had been the key to their success, stating for example that volunteering had ‘helped me with my professional development in securing and being successful in criminal law placements’ (MJRC1); ‘it's got me a vacation scheme at my dream firm. It helped with the assessment centre’ (FL1); and ‘volunteering at the FreeLaw clinic was invaluable in securing a training contract’ (PB2). Participants’ perceptions of how employers view pro bono volunteering Sheffield, like many law schools, has good links with employers.Rgular discussions are held to identity what employers are looking for in the recruitment process. The survey respondents provided first-hand accounts of assessment processes and interviews for placements and/or for positions as trainees and employees, from ‘the other side of the table’. Many participants in the survey offered their views on how much employers appear to value pro bono experience; for example: ‘I have had many training contract interviews recently and the interviewers all asked me about my time volunteering at the FreeLaw clinic and were very interested in the sorts of things I did. My general feeling was that they found it very impressive’ (PB2). More specifically, it seems that ‘employers love people with client experience’ (FL1), treating this as a proxy for having acquired relevant hard skills. Further, ‘employers are always interested in these soft skills’ which are associated with involvement in pro bono projects (MJRC1). The comment that ‘voluntary work is usually looked at positively’ (PB2) chimes with other explanations that pro bono volunteering ‘demonstrates interest and passion to employers’(MJRC1), and that it ‘will always stand out on a CV as it shows not only a genuine interest but willingness to help others with no financial gain. This selflessness is really attractive to employers’ (MJRC1). Survey participants also thought that the type of legal work undertaken at FreeLaw and MJRC would be of interest to employers: ‘Most firms carry out an element of pro bono work (albeit usually small) so it is good to show that you would be interested in this’ (MJRC1). These accounts accord with the findings of the employers’ survey discussed earlier, even though none of the recruiters interviewed ‘mentioned that participating in clinical legal education on a voluntary basis might be indicative of personal values that would be attractive in potential recruits’ (Thomas, 2018, p.140). The employers’ enthusiasm and interest which the survey participants reported must therefore be dependent on effective presentation of the applicant’s pro bono experience, as it is apparent that CLE is generally poorly understood by employers. For example, Thomas’ interviewees quickly identified that interpersonal skills and communication (with clients) would be developed through CLE, but none of them identified teamwork (ibid.,) which is an essential skill in pro bono project volunteering. How pro bono experience is used in securing placements, training, employment and promotions This section of the paper focuses on participants’ accounts of how they made use of their experiences at FreeLaw and MJRC at the various stages of the recruitment process. It was difficult to make a selection of these qualitative responses, as there were so many which made excellent points. Those reproduced here are representative of similar views expressed by several respondents. Applying for vacation schemes, or for a training contract, are the commonest first points of entry into the legal profession. Many participants from the first cohort reported that pro bono volunteering helped them to ‘perform in assessment centres and apply skills in problem scenarios’ (FL1). One respondent wrote that their pro bono experience had supplied ‘examples in my answers to evidence my skills and my commitment to the legal profession’ (FL1). In terms of broader employability, another respondent wrote that ‘having evidence to support your soft skills is great and the MJRC gives me this’ (MJRC1). Second cohort participants also had experience of applying for training contracts, for post-qualification posts, and for promotions. One respondent from this cohort, who had been a student manager, explained that ‘I have used the leadership and management experience gained through working with Free Law at every external and internal interview since University’ (PB2). Another participant wrote that ‘I have also applied for teaching work, and my pro bono experiences were just as beneficial here’ (FL1), although most survey respondents were focused on a legal career. Recruitment processes in this sector often require candidates to address the Solicitors Regulation Authority’s Statement of Solicitor Competence (SRA, 2015). One respondent described the skills and experiences gained through pro bono volunteering as: ‘essential for both legal job interviews and graduate scheme interviews. In each process you are required to undergo a competency interview or answer competency-based questions on application forms. I am confident that at least 50% of all the competency questions I have ever answered have been from situations during my time at Freelaw’ (PB2) A widely-held view, that volunteering at FreeLaw or MJRC helped respondents to stand out from the many other good graduates who are their competitors for positions in the legal profession, was expressed by many respondents. One participant said that volunteering at the MJRC ‘helped me demonstrate an ability beyond that provided by the traditional law degree … showing that I was a good candidate for my job’ (PB2). Another respondent commented that ‘when applying for training contracts it is very difficult to distinguish yourself, it is also difficult to answer problem-based questions without any experience. Therefore having the opportunity to talk about real life situations really helps’ (PB2). Pro bono volunteering was also seen as confidence-building, which helps alumni succeed in a crowded job market. In the words of this respondent, ‘I was able to talk with more legitimacy about what I enjoyed about the law in practice and what I was good at / weaker at. My whole application had more gravitas’ (PB2). These very positive experiences described by the law school survey’s respondents seem significantly different from those taking part in the PSU survey, in which less than two-thirds (62%) of respondents felt that volunteering at PSU had provided them with ‘good first-hand examples of real life experience for job applications and/or interviews’ (PSU, 2015). In contrast, for example, this participant to the law school’s survey was typical in reporting that ‘I was able to give comprehensive answers to all the questions asked in interview [for a training contract] just by using FreeLaw’ (PB2). Value added by taking the Pro Bono module Sixteen respondents from the first cohort (70% of the total) and twenty-three (80%) of the respondents in the second cohort had taken the optional module, which is only available to final year pro bono volunteers. When these respondents were asked if taking the module had assisted them in obtaining training or employment, 61% of the second cohort agreed. For the first cohort, this figure was 75%. This difference in the positive link between the module and employability may be due to changes introduced in the teaching and assessment of the module over the last few years, which include a greater focus on critical assessment of skills development and self-reflection. On the other hand (as previously discussed) the difference might be a result of the optimistic expectations of current students in the first cohort who had not read the survey question accurately. Nevertheless, the detailed views which participants volunteered, about the benefits of taking the module, proved extremely interesting. Nearly all respondents who expressed an opinion included an explanation that the module’s requirement for critical self-reflection had proved particularly helpful. This participant, for example, considered that the module provided: ‘an opportunity to show future employers that I was able to critically evaluate myself. Personal reflection is an important skill that employers are increasingly looking for, therefore I think the module was useful in securing a training contract’ (PB2). Many of the respondents considered that the module had been useful because students were required to identify and discuss, in their assessed portfolios, the competencies and skills developed through pro bono involvement. This exercise was reported to have assisted in many aspects of the recruitment process. One participant wrote that ‘self-reflection has made me realize how much work I have done and what I am capable of and what I need to work on. I am now more confident in writing cover letters about myself’ (MJRC1). Another survey participant explained that, without having to complete the portfolio, ‘I probably wouldn't have thought to write down all the great examples of leadership and learning that I experienced’ (PB2). The practical impact of the portfolio assessment was expressed by this respondent, who felt that it had made the interview process less stressful, because ‘by completing the module I had information at the forefront of my mind to answer questions’ (PB2). Interestingly, although the low numbers mean it is not possible to place any statistical reliance on this result, those respondents in the first cohort who were taking the module expressed more positive views about the general value of pro bono volunteering in enhancing employability, in comparison with their peers who were not taking the module. This may be due to the individual contact with tutors which, as explained above, is a feature of the module, and is a key indicator of student engagement and perception of skills enhancement (Buckley, 2015). Some survey participants did not feel that the module had assisted their employability, but none of these offered any further explanation in their survey responses. The following comment speaks for most survey respondents: ‘taking the module demonstrated my commitment to developing practical legal skills as opposed to just learning black-letter law’ (FL1). The views and experiences of participants in both cohorts who have taken the optional module give a very useful insight into why and how they felt it enhanced their employability. CONCLUSIONS This research project was not able to demonstrate statistically a direct correlation between pro bono volunteering and students’ employability, or their subsequent employment. The problems of achieving a robust research design to address those issues appear insurmountable, not least the difficulties of establishing a comparable control group. However, given the resources which universities now commit to CLE or pro bono programmes, and the assumptions that students’ employability skills and attributes are enhanced through participation in them, it is important to research connections with employability. There have been few studies in this field so far, and those (including the Sheffield survey) have been small-scale and inexpensive to conduct (see Cantatore, 2018, and Thomas, 2018) but have nevertheless established many valuable points. The additional value of the Sheffield study lies in the unprompted qualitative data included in the survey responses. With0ut being provided with a list of skills, survey respondents said they had developed particular skills and attributes through pro bono volunteering, which can be matched against the lists developed by the World Economic Forum, careers advisers for the commercial law sector, and the HEA UK Engagement Survey. These serendipitous findings (Foster and Ford, 2003) from the survey comments make it clear that pro bono volunteering offers students the opportunity to develop both hard and soft skills, beyond the academic curriculum. The majority of respondents felt that the experience of pro bono volunteering had enhanced their employability, in the accepted sense of a ‘set of achievements, understandings and personal attributes that make individuals more likely to gain employment and to be successful in their chosen occupations’ (Yorke, 2006: 8). The survey respondents’ accounts of applying for vacation schemes, training contracts and employment in the legal sector, indicates that their experience of engaging with ‘real world’ legal issues is recognised by employers, marking out pro bono volunteers from their peers. Further, students who have developed skills and attributes through engagement with pro bono projects understand how to use and present these in recruitment processes, both in the legal sector and more widely, with a considerable degree of success. This seems particularly marked for the Sheffield students who took the optional module, which requires critical self-awareness of skills development. The survey also confirms the benefits, in terms of developing employability skills, of the student-run pro bono model developed at Sheffield which gives volunteers greater responsibility and opportunities to develop both hard and soft skills. This may help them to stand out from their peers at other UK law schools which also run pro bono clinics and similar CLE projects, albeit on a ‘pre-vetted’ basis or with more staff involvement. Echoing Cantatore (2018), there is a need for more research into the various aspects of developing employability skills in Law students. However, caution is needed when presenting research findings. The methods used must be carefully explained, so that the data can be distinguished from other research carried out or which is being planned. Findings from one small-scale survey should not be taken to be applicable to other contexts. Each university running a law clinic has its own version of CLE or pro bono programme, and jurisdictional differences must also be taken into account. Some points of general applicability can nevertheless be derived from this research project. Students should be made more aware of the close connection between CLE or pro bono involvement and employability, which is particularly apparent from the data provided by second cohort respondents in the Sheffield survey. If the module associated with these activities is optional, more students should be encouraged to take it, as the survey data indicates it supports them in identifying and presenting the employability skills, both hard and soft, which they are acquiring. This is reportedly useful in the recruitment process, particularly in dealing with competency-based questions. Law schools should therefore be encouraged to continue, or increase, devoting resources to CLE and pro bono projects. References Buckley, A. (2015) UKES 2015: Students’ perceptions of skills development. York: Higher Education Academy. Accessible at: https://www.heacademy.ac.uk/knowledge-hub/uk-engagement-survey-2015 Cantatore, F. (2018) ‘The Impact of Pro Bono Law Clinics on Employability and Work Readiness in Law Students’, International Journal of Clinical Legal Education, 25:1, 147-172. Cantatore, F. (2015) ‘Boosting law graduate employability: Using a pro bono teaching clinic to facilitate experiential learning in commercial law subjects’, Legal Education Review 25: 1, 147-172. Carney, D., Dignan, F., Grimes, R., Kelly, G. and Parker, R. (2014) The LawWorks Law School Pro Bono and Clinic Report. LexisNexis. Coates, H. (2009) Engaging students for success: Australian student engagement report. Melourne: Australian Council for Educational Research. Dignan, F., Grimes, R. and Parker, R. (2017) ‘Pro Bono and Clinical Work in Law Schools: Summary and Analysis’, Asian Journal of Legal Education, 4:1, 1-16. Foley, T., Rowe, M., Holmes, V. and Tang, S. (2012) ‘Teaching professionalism in legal clinic – what new practitioners say is important’, International Journal of Clinical Legal Education, 17, 5-22. Foster, A. and Ford, N. (2003) ‘Serendipity and information seeking: an empirical study’, Journal of Documentation, 59: 3, 321-340, Hyams R. (2010) ‘Assessing Insight: Grading Reflective Journals in Clinical Legal Education’, James Cook University Law Review, 17, 25-46. Kuh, G. D. (2001) The national survey of student engagement: Conceptual framework and overview of psychometric properties. Bloomington, IN: Indiana Center for Postsecondary Research. Kuehn, R. R. (2015) ‘Measuring Clinical Legal Education’s Employment Outcomes’ Wisconsin Law Review 645-664. Mokidi, S. K., and Agbebaku, C. A. (2012) ‘Legal Clinics and Professional Skills Development in Nigeria’, International Journal of Clinical Legal Education, 17-18:1, 38-48. Neves, J. (2017) The UK Engagement Survey 2017: Student participation and skills gain. York: Higher Education Academy. Accessible at: https://www.heacademy.ac.uk/knowledge-hub/ukes-2017-report Oliver, B., Whelan, B., Hunt, L., Hammer, S., Jones, S., Pearce, A. and Henderson, F. (2011) Introducing the Graduate Employability Indicators. Canberra: Australian Learning and Teaching Council. Accessible at; https://www.assuringgraduatecapabilities.com/uploads/4/5/0/5/45053363/introducingthegei.pdf Personal Support Unit (2015) Training the Profession. [No longer available online]. Pound, R. (1910) ‘Law in Books and Law in Action’, American Law Review 44, 12-36. Smith, J. (2018) ‘7 skills for a successful law career’, Prospects. Accessible at: https://www.prospects.ac.uk/jobs-and-work-experience/job-sectors/law-sector/7-skills-for-a-successful-law-career Solicitors Regulation Authority (2015) Statement of Solicitor Competence. Accessible at: http://www.sra.org.uk/threshold and http://www.sra.org.uk/knowledge Sterba, S.K. and Foster, E.M. (2008) ‘Self-Selected Sample’, in Paul J. Lavrakas (ed) Encyclopedia of Survey Research Methods, Sage ResearchMethods. Accessible at: http://methods.sagepub.com/reference/encyclopedia-of-survey-research-methods/n525.xml TARGETJobs (2018) ‘What skills do law firms look for when recruiting graduates for trainee solicitor jobs?’ Accessible at: https://targetjobs.co.uk/career-sectors/law-solicitors/advice/291203-what-skills-do-law-firms-look-for-when-recruiting-graduates-for-trainee Thanaraj, A. (2016) ‘Understanding how a law clinic can contribute towards students' development of professional responsibility’, International Journal of Clinical Legal Education, 23:4, 89-135. Thomas, L. (2018, forthcoming)’’It Puts the Law They’ve Learnt in Theory into Practice’: Exploring Employer Understandings of Clinical Legal Education’ in Thomas, L., Vaughan, S., Malkani, B. and Lynch, T. (eds) Reimagining Clinical Legal Education. Oxford: Hart. Thomas, L. (2017) ‘Law clinics in England and Wales: a regulatory black hole’, The Law Teacher, 51:4, 469-485. UK Commission for Employment and Skills (2016) UK Employer Skills Survey 2015: Summary report. Available at: https://www.gov.uk/government/publications/ukces-employer-skills-survey-2015-uk-report Universities UK (2016) Higher Education in England: Provisions, Skills and Graduates. Accessible at: http://www.universitiesuk.ac.uk/policy-and-analysis/reports/Pages/higher-education-in-england-provision-skills-and-graduates.aspx World Economic Forum (2016) The Future of Jobs: Employment, Skills and Workforce Strategy for the Fourth Industrial Revolution. Geneva: World Economic Forum. Accessible at: https://www.weforum.org/reports/the-future-of-jobs Yackee, J.W. (2015) ‘Does Experiential Learning Improve JD Employment Outcomes?’, Wisconsin Law Review, 601-626. Yorke, M. (2006) Employability in higher education: what it is – what it is not. York: Higher Education Academy. Accessible at: https://www.heacademy.ac.uk/.../id116_employability_in_higher_education_336.pdf 45