257699B_IJCLE_July_09 Assessment– Are Grade Descriptors the Way Forward? Victoria Murray and Tamsin Nelson* The debate on how best to assess clinic, or indeed if it should be assessed at all has raged for decades and shows no sign of abating. The passage of time has been unable to resolve the question of assessment, no doubt due in part to the expansion and diversification of clinical legal education. The scope of clinic and its role in both society and as a teaching method is constantly evolving and assessment methods must develop to reflect the ever changing clinical profile. In an attempt to bring its assessment regime up to date, in 2007/2008 Northumbria University’s Student Law Office modified its assessment regime, replacing outmoded criteria with grade descriptors. This paper focuses on the use of grade descriptors and criterion referenced assessment in clinical legal education, addressing whether clinic should be assessed and which of the two methods is best suited to clinical legal education. The article draws on the experiences of clinicians and students to determine what issues this change in assessment regime has raised for the assessors and the student body. It concludes that it is appropriate to assess clinic by fully grading and suggests grade descriptors are the way forward. INTRODUCTION In 2007/2008 Northumbria University’s award winning Student Law Office (SLO)1 overhauled its assessment regime, replacing an outmoded list of criteria with grade descriptors. This paper provides an evaluation of these two assessment methodologies, with specific reference to their adoption in a clinical setting.2 After many years of criterion referenced assessment, grade descriptors were mooted and SLO clinicians agreed they should be adopted for the academic year 2007/2008. This change in assessment regime was welcomed not least because it was anticipated the grade descriptors would remedy what the supervisors perceived to be increasingly problematic assessment criteria. In * Senior Lecturers in the School of Law, Northumbria University, UK. The authors wish to express their gratitude to their clinical colleagues and students who contributed to a focus group and responded to questionnaires, which enriched our research. Our thanks also extend to Elaine Hall who greatly assisted with the student questionnaires. This paper was presented at the IJCLE Conference, University College Cork, Ireland, 2008. 1 In 2008 the Student Law Office was awarded the Attorney General’s Pro Bono award for Best Contribution by a Law School. 2 For a full consideration of assessment in the clinical environment see Stuckey, R. & Others. (2007) Best Practices for Legal Education. Available at http://cleaweb.org/documents/Best_Practices_For_Le gal_Education_7_x_10_pg_10_pt.pdf (accessed on 11 August 2008). International Journal of Clinical Legal Education July 2009 48 3 Full time students on the freestanding Legal Practice Course and Bar Vocational Course can also participate in the SLO from January to June. 4 Where a positive impression of the student has been formed which can inf luence the assessor to subconsciously distort information favourably this is known as the halo effect and conversely, where a negative impression has been formed this is referred to as the horns effect. Dunn, L., Morgan, C., O’Reilly, M & Parry, S. (2004). The Student Assessment Handbook. London, Routledge Falmer. page 255. 5 Stuckey, R (2006) ‘Can We Assess What We Purport To Teach In Clinical Law Courses?’ 9 International Journal Clinical Legal Education 9–28 at page 23. particular, supervisors hoped that the grade descriptors would engender more reliable and consistent marking owing to their explicit detail. In order to appreciate the findings from the research undertaken in the SLO, it is necessary to provide some contextual background information. The SLO is a year long compulsory clinical module undertaken in the final year of the law school’s exempting law degree (ELD).3 The ELD is fully integrated, combining the undergraduate law degree with a one year post graduate vocational course. Students graduating from the ELD are eligible to commence the training stage necessary to qualify as a solicitor or barrister. In 2007/2008 approximately 130 exempting degree students completed the Student Law Office module and clinical supervisors numbered 17. The SLO counts for 2 full modules, the largest undertaken on the degree and the final year marks contribute 40 per cent towards the student’s degree classification. The mark achieved for the SLO module can therefore play a pivotal role in determining a student’s overall degree classification. How Is Clinic Assessed? Clinic has many guises and consequently the modes of assessment are wide and diverse in order to reflect the particular clinical model in question. Many clinics assess on both a formative and summative basis. The formative aspect of assessment in clinic is intended to provide feedback and give the student direction on how they can improve their performance. The purpose of summative assessment is to formally assess the student’s output. Summative assessment may take the form of a numeric or letter grade, or may be on a pass/fail basis. Some modules are not assessed per se but result in award of credits. It is summative assessment that can cause the most difficulty. The problem with clinic is that as students are usually assessed by their clinical supervisor there can be a large element of subjectivity when assessing. Consequently, it can prove difficult to mark students objectively due to the close (or in some cases challenging) working relationship which has developed between supervisor and student throughout the year. This is known as the halo and horns effect.4 It is probably a question for another paper as to whether clinical assessment should include an element of subjectivity or if it should be, or is capable of being completely objective. There have been criticisms of having any subjective element in clinical assessment and in order to overcome these concerns, clinic can be assessed in a variety of different ways including obtaining feedback from clients, giving the student a point based score5 and using standardised clients. In order to assess the skills that a student has obtained whilst in clinic there is invariably a degree of subjectivity particularly if the person assessing is the student’s supervisor. One way to counteract this is for all students to write a standardised letter or critique a standard file which is assessed by an independent person. Students can also create a portfolio of work that is accumulated over the course of the year and assessed. Any of these methods of assessment can be marked on a pass/fail basis or graded. 49 Assessment– Are Grade Descriptors the Way Forward? In the SLO, students submit a portfolio evidencing the live client work they have undertaken throughout the academic year. This will typically comprise draft and final versions of correspondence, attendance notes, research reports, legal pleadings and interview plans. The portfolio contributes 70% toward the mark for the module with the remaining 30% attributed to 3 written pieces reflecting on skills in practice, the law in action and one other optional reflective account selected from a prescribed list of titles. The portfolio and reflective pieces are assessed by the solicitor who has supervised the student during the year and these are moderated by a supervisor who has no connection to the student. The question then is how the portfolio or any other means of assessment is best assessed. Should clinical work be assessed by way of general criteria or formalised grade descriptors? How Should Clinic Be Assessed – Grade Descriptors Versus Criteria For many years the Student Law Office assessed student performance utilising a list of criteria. The 13 point list essentially required the clinician to evaluate “to what extent” a student had performed across a range of areas. For example, to what extent did they: • Participate in and diligently conduct cases • Begin to develop an ability to manage and analyse factual information • Begin to develop an ability to plan the conduct of a case • Demonstrate an ability to critically consider and analyse the development of their legal skills It can be sen from the above that the criteria were scant in terms of context. Criterion referenced assessment The Carnegie Report hits the nail on the head when it identifies the key difficulty with criterion referenced assessment – ensuring consistency when grading.6 Clinicians have qualities which they look for in students, and they will rank some of those abilities and attributes more highly than others. For example, commercial clinicians might rank drafting skills above those of advocacy, and vice versa for criminal clinicians. Consequently, the question must be posed – using a list of criteria can we ever be entirely confident that we are marking to the same standards? How can we be certain that one clinician’s mark of 65 is comparable to another’s 65? Stuckey highlights further issues with criteria based assessment when he states, “when criteria are given to students, they tend to be checklists that cover the entire spectrum of lawyering activities without any description of different levels of proficiency.”7 This quotation highlights two pitfalls of assessing using criteria. The first is that students will see the criteria as an inventory and may simply tick off what they have achieved from the list without perhaps considering to what extent that skill has been developed. Furthermore, if the list is exhaustive students may not strive to achieve above and beyond the criteria specified. From a teaching perspective, the use of criteria may therefore stifle ambition to realise full potential and achievement. And what if a student 6 Sullivan, W.M. et al., (2007) Educating Lawyers: Preparation for the Profession of Law. San Francisco: Jossey– Bass Inc, page 170. 7 Stuckey, R, Best Practices, op. cit at page 238. International Journal of Clinical Legal Education July 2009 50 displays qualities outside the scope of what is provided for by the criteria; can this be rewarded if those qualities do not explicitly appear within the assessment criteria? The other problematic aspect which Stuckey identifies is the lack of guidance offered by written criteria. His assertion that they lack any description or meaningful instruction on performance levels is something of which SLO clinicians were acutely aware. The lack of explicit guidance on performance levels resulted in uncertainty that each and every single clinician was marking to the same standard. One might hope that any inconsistencies would be rectified in the moderating process however, it must be extraordinarily difficult for a second marker to evaluate the work without the benefit of having monitored the student’s progress throughout the year. For example, two portfolios of work might contain excellent pieces of work. What may not be evident to the second marker is that one student may have produced excellent first attempts requiring little amendment or input from the supervising clinician, whilst the other student may have needed several attempts before achieving the finished product. Furthermore, intangible attributes, such as initiative, will not necessarily be obvious to a second marker from viewing a collection of the student’s written work. Whilst the lack of guidance on the one hand may lead to inconsistent marking, some clinician’s may enjoy the room for discretion which this inevitably allows. The flexible nature of assessing via an imprecise list of criteria arguably fits the unpredictable and personal nature of clinic. That is to say, a rigid and static assessment regime may be suited to a controlled form of assessment such as an essay or exam question, but given live client work often takes unexpected turns, clinic should have an assessment method which allows for discretion and flexibility. A list of criteria certainly possesses this quality, but arguably at an unacceptable level. Furthermore, where criteria are too vague or lacking in detail, it is too tempting to rely on a subjective, rather than objective, assessment of the student performance, and subjectivity promotes inconsistent marking. It is clear, then, that there are issues with the use of a list of criteria as an assessment method from both student and teaching perspectives. Thus, the decision was taken to abandon the use of criteria and a new assessment regime of grade descriptors was introduced. Grade Descriptors8 As a result of the above concerns, grade descriptors were formulated detailing the level students would have to reach in order to achieve a 2.2 classification (50–59%), a 2.1 classification (60–69%) and a first (70%+). They also profiled a fail student (<50%) and a strong first student (>80%). The grade descriptors were based on discussions with clinical supervisors regarding the factors they concentrated on when assessing students. The descriptors concentrated on the student’s ability to demonstrate autonomous learning. This encompassed the student’s ability to identify and apply the law, plan and manage cases, and learn from past performance. The grade descriptors identify the performance indicators which a student has to achieve across all classification levels. 8 The full grade descriptors can be viewed at http://www.northumbria.ac.uk/sd/academic/law/slonew/assessment/ 51 Assessment– Are Grade Descriptors the Way Forward? For example: A minor failing of the grade descriptors was identified when conducting our research. In some cases a performance indicator was not present across all classifications.9 This only became apparent when a detailed comparison of the grade descriptors was completed. For example, Once a relevant performance indicator has been determined, care needs to be taken that it is tracked across the classifications. 9 This anomaly has now been rectified. International Journal of Clinical Legal Education July 2009 52 Fail student 2.2 student 2.1 student First class student Strong first class The student demonstrates little commitment or energy to achieving the best resolution for the client. The student will often demonstrate enthusiasm and empathy but commitment to the client’s case may be undermined by failure to do work to a sufficiently high standard. The student shows commitment to their clients and is able to demonstrate empathy for the client. There will be a high level of commitment to the client. The student’s communications with the client instil a high level of confidence about their ability to empathise with, understand and serve the client’s interests. Fail student 2.2 student 2.1 student First class student Strong first class The student will look to the supervisor for instruction; there is little sense of the student planning how best to progress the case. Case management skills are likely to be weak. The student will carry out tasks assigned to him or her but will rarely show initiative in planning how best to progress the case. [No applicable performance indicator provided] [No applicable performance indicator provided] The student requires little active supervision and can be trusted to identify tasks and take appropriate action subject to supervisor approval. Staff Opinions On Assessing Using Criteria And Grade Descriptors To gather staff opinion on the assessment regimes, SLO clinicians were invited to a focus session at which both the criteria and grade descriptors were discussed. Two staff focus meetings were held; one before and one after assessing using the new grade descriptor regime. Both meetings were attended by clinicians of different subject specialism, and in order to obtain a full spectrum of opinion, new SLO clinicians who had used neither regime also contributed. The first meeting At the initial meeting supervisors felt that by assessing using the criteria they were effectively free to grade a student as they wished because of the malleability of the criteria. There was a strong consensus that the most important criteria were the ones that related to a student’s proactivity on the file, namely the last two criteria on the list: • Begin to develop an ability to review case files and to plan the conduct of a case • Begin to develop an ability to manage and analyse factual information on case files There was a solid belief that these two criteria were critical in distinguishing between students and in providing a specific grade. Several supervisors confirmed that prior to marking they ranked their students before looking at the portfolios, then they would look at the portfolio to see if their ranking fit the criteria. They paid particular attention to the above two criteria to grade the students, although in most cases they already had the classification in mind. Overall supervisors felt the use of the criteria when marking did not particularly influence them in their assessment as most had a good idea of what the final grade was going to be for a particular student. However, they were concerned about the subjective nature of using criteria to assess students. It was agreed that under the criteria it was difficult for students to truly understand what performance was required to achieve a particular classification. This led one supervisor to state that all students “fear the subjectivity of supervisors”. However another supervisor argued that in a non clinical module, markers receive an answer guide but no indication of the level required for each classification, yet this is seen as objective marking.. At this first meeting the grade descriptors were also discussed to ascertain supervisors’ views on their use for the up-coming assessment. There was a general feeling that the descriptors would promote greater consistency of marking and that students would have more guidance as to what supervisors were looking for. This did pre-suppose that the students looked at the grade descriptors and worked with them throughout the course of the SLO.10 Supervisors generally agreed that having the grade descriptors made them feel more confident that their expectation of a 2.2 was the same as other supervisors 2.2; this was particularly the case for first time supervisors in the SLO who had not previously graded clinic. One concern with the descriptors was that they could be used as a ‘tick chart’ with supervisors simply ticking across the range of classifications with the ticks simply added up to establish what classification the student would achieve. It was, however, felt that this was unlikely to happen and the general consensus prior to marking was that a student would not be given a mark that the supervisor did not think they deserved. 10 One supervisor asked his students whether they had used the grade descriptors whilst in the SLO and the consensus was that they had looked at them at the start of the year however they had not then referred to them again until the mid-year appraisal. 53 Assessment– Are Grade Descriptors the Way Forward? An issue was raised both with the criteria and grade descriptors as to whether the students were graded on the day that the assessment was handed in or were they graded over the course of the year and therefore credit was to be given for improvement. There was a strong sense that the criteria did not address this and that arguably the grade descriptors did not either. One supervisor stated that, “I assess students all the time and what I am doing is developing an impression of them and varying the level of expectation. On the [hand in date] I will come to my final conclusion. That has got to be my assessment otherwise I will prejudice them ... and not give credit for improvement.” The second meeting After marking using the grade descriptors a second meeting was held with the same supervisors who had attended the initial meeting. They were asked what they felt about the grade descriptors having now utilised them to assess student performance. The initial expectation that the grade descriptors would inform the supervisor’s marking habits was confirmed previously using the criteria supervisors felt they graded using an element of gut instinct, due to the criteria being vague. Conversely, the much more informative nature of the grade descriptors promoted objective and consistent marking as everyone was singing off the same explicit hymn sheet. Clinicians felt more confident that they were marking to the same standard as their colleagues using the descriptors than when using the criteria, because there was no need to add flesh to the latter’s bones. It was stated above that the flexibility of written criteria afforded room for discretion when marking, which was, to some extent, a desirable feature. In the second focus group it was felt that as the descriptors were particularly descriptive, when grading there occasionally appeared to be a lack of room for manoeuvre. In particular, two supervisors felt that because of the prescriptive wording of the grade descriptors they felt compelled to award first class marks to students, who under the previous regime, would have received a 2.1 classification. Furthermore, the grade descriptors have not removed weighting issues. For example, one supervisor might attach more importance to one performance indicator over another supervisor and this might affect overall mark. It was also felt that the descriptors were used much more by some supervisors over the course of the year therefore they were not relied on heavily at the time of assessment since the supervisor already had an idea of the grade in mind. The descriptors still did not remove the normative element of assessing the students as it appeared that there was still a tendency for supervisors to rank students prior to finally assessing them and awarding an overall grade. The grade descriptors did not therefore remove all the subjectivity of the assessment but certainly tempered it and since the mark needed to be justified against the descriptor, objectivity was more pronounced. From the moderation process it was noted that a mix of liberal and strict markers still existed despite moving to a much more explicit marking regime. Overall, staff were satisfied with assessing via the grade descriptors and it has been agreed that they will be used for assessment purposes in the next academic year. Student Perspectives Although having no experience of grade descriptors elsewhere on the degree, students appeared equally positive towards them. Student views were obtained though an anonymous questionnaire, which was sent to all 130 full time final year students on the module, prior to being summatively assessed. A total of 45 questionnaires were completed, giving a return rate of 35%. 54 International Journal of Clinical Legal Education July 2009 The questionnaire selected 5 descriptors (commitment, key skills, insight, awareness of development and use of reflection) and reproduced the statement for that descriptor for each classification (fail, lower second class, upper second class, first class and strong first class student). For example, taking the descriptor for “commitment,” a failing student would demonstrate “little commitment or energy to achieving the best resolution for the client.” At the opposite end of the scale, the strong first class read “the student’s communications with the client instil a high level of confidence about their ability to empathise with, understand and serve the client’s interests.” The questionnaire then asked whether the student was confident that they understood the grade descriptor, whether they were confident they knew which level they were working at (fail, 2.2, 2.1, first class etc) and whether they were clear on what action was needed to improve to the next classification of descriptor. The responses available to students were: question not clear, strongly disagree, disagree, agree and strongly agree. The overwhelming majority of students indicated that they understood the descriptors.11 Interestingly, although not all students were confident they knew their current grade level, the results indicate that on the whole, they were clear about what action they needed to take to reach the next level of classification. The questionnaire also included two further statements which were intended to shed light on student motivation. The first statement was “the grade descriptors influence how I carry out my live client work;” the second “I try to perform well because I’m working for a real person, rather than because I am being assessed.” The same responses were available to students and they were also able to comment on the statements. There was an even split: 21 students disagreed or strongly disagreed that the grade descriptors influenced their clinical work and 22 students agreed or strongly agreed with the contrary proposition. The additional comments also reflected the divided opinion. One student commented, “You’re thinking about doing best [sic] for client rather than what grade band you’ll fit into.” Another student remarked that they tried, “to treat SLO like practice and strive to achieve my best, therefore [I] will meet...the grade descriptors.” Opinions about the second statement were much more uniform with 37 respondents agreeing or strongly agreeing that they were client rather than assessment driven. Some students further commented that they were motivated by a “combination of the two” and felt it was a matter of balancing the two competing interests as they were not mutually exclusive. One candid student noted “I’d be a liar if I said the SLO grade is not always at the back of my mind.” Similarly one of the cohort thought it was “unavoidable” that their performance was assessment driven due to the impact the SLO can have on degree classification. Whereas Rice12 views assessment driven motivation negatively, one student observed that, “in striving to get a good mark the client benefits from [a] high standard of work.” Assessment– Are Grade Descriptors the Way Forward? 55 11 For commitment students voted as follows: strongly agree 15, agree 26, disagree 3, strongly disagree 0 and question not clear 0. For key skills: strongly agree, 10, agree 32, disagree 2, strongly disagree 1 and question not clear 0. For insight: strongly agree 6, agree 35, disagree 4, strongly disagree 0 and question not clear 0. For awareness of development: strongly agree 9, agree 35, disagree 1, strongly disagree 0 and question not clear 0. Finally, use of reflection: strongly agree 8, agree 30, disagree 7, strongly disagree 0 and question not clear 0. 12 Rice, S. (2007) ‘Assessing – but not grading – clinical legal education’ Macquarie Law Working Paper No. 2007-16 available to download at http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1 061622 How The Regimes Compare – The Statistics In 2006/2007, using the list of criteria, the average mark awarded for the portfolio was 65.6%, compared to 67.3% when applying the grade descriptors the following year. This is by no means a dramatic rise, although a closer inspection of further statistics highlights some interesting results. In particular there was a significant increase in the highest overall mark – 76% in 2006/2007 rising to 85% in 2007/2008. Similarly, the number of first class marks jumped from 36 to 51. There are notable disparities in the year to year results, but what conclusions can be drawn from this data and how do the results compare to non clinical modules? It is possible that the higher results are a direct consequence of the explicitness of the grade descriptors. That is to say, students have a clear understanding of what is required to achieve the highest level and can therefore strive to reach those said levels. This cannot be said of the former criteria which lacked any meaningful guidance of what levels of proficiency were needed to achieve a particular classification. An alternative supposition is that the grade descriptors are too generously worded resulting in additional students scoring more highly than they ought. This is reinforced by the view, as stated above, of at least two clinicians who would have bestowed 2.1 classifications under the old regime, but for the wording of the grade descriptors compelling the award of a first class mark. This latter explanation for the increase in marks also draws support when one views the results of the same group of students in non clinical subjects. Take for example the performance in a taught, classroom based year long module which would typically be assessed by the student sitting an end of year exam and submitting a piece of coursework. Whilst the pedagogy will be vastly different, the two modules are of the same duration and both assessed, so the much touted notion of the “assessment driven student” is still omni-present. It is interesting to note therefore that in 2007/2008 the average mark for the clinical module was 68%, compared to 61% for non clinical subjects. What is perhaps more telling is the comparison of the marks awarded for the students’ dissertation, which, like clinic, is completed over an academic year with formative feedback. Also, not dissimilar to clinic, the student has relative autonomy over the subject area to be studied.13 One might therefore suspect that the results would be relatively similar. However, this is not entirely reflected in the results; the average mark in 2007/2008 for dissertations being 64%. Should Clinic Be Assessed At All? In the staff focus sessions, whilst looking at how the SLO approaches assessment of clinical work the question was raised whether we should move away from grading clinic and assess clinic on a pass/fail basis, or whether we should assess clinic at all. Given that the aims and format of clinic are incredibly diverse it is not uncommon for clinical modules to lack any form of summative assessment. The trend for not assessing clinic generally attaches to voluntary or optional clinical modules. Where law schools do formally assess clinic, again the practices vary. The module may be fully graded, marked on a pass fail basis or the student may be awarded a credit. If we take as a starting point Stuckey’s comment that “the current 56 International Journal of Clinical Legal Education July 2009 13 In the SLO students select a first and second choice area of law from a list of criminal appeals, employment, civil, business, housing, education and welfare benefits. The overwhelming majority if students are successfully placed according to their stated preference. assessment practices used by most law teachers are abominable,”14 we might well question whether no assessment is indeed good assessment. There are a variety of reasons why performance should be assessed, perhaps the most common being that it recognises the efforts displayed by students and it motivates them to achieve. This is supported by Brustin and Chavkin’s findings that numerical grading had a “significant positive impact” on clinical students’ motivation.15 Whereas Rice16 is wholeheartedly in favour of assessing clinic, that is where his support for assessment comes to a halt. He suggests that clinicians “take for granted”17 that clinic should be fully graded and advocates a pass/fail assessment regime as an alternative. In his working paper, Rice presents a robust attack on grading, arguing that it is “simple and simplistic mechanism. I suspect that it is attractive to teachers precisely because it is unspecific and impersonal.”18 Whilst it may be true to say that a number or letter in isolation can be perceived as impersonal or that it is not particularly helpful to the student in terms of highlighting where they have (under)achieved, Rice apparently disregards the vast amount of feedback students receive when undertaking clinical work. Unlike other classroom based modules, clinical students will invariably have each piece of work formatively assessed and often appraisals, together with regular supervisor contact, are a feature of the unit. Therefore throughout the clinical experience, students should have developed a clear understanding of their strengths, weaknesses and how they are performing generally. Their final grade is therefore unlikely to be a surprise given the extensive feedback with which they ought to have been furnished. Taking this into consideration Rice’s supposition that grading is impersonal and unspecific can be rebutted. It can further be argued that clinic is perhaps the most time intensive element of any law degree in terms of providing feedback and assessment. It should also be remembered that unlike traditional assessment methods, for example essays or problem based questions, with clinic there are no right answers. Consequently, it is arguably simpler to grade non clinical modules where you have the joy of an answer guide. It is extremely doubtful that students receive anywhere near the level of feedback outside of clinic and Rice’s implication that teachers grade because it is an ‘easy option’ is perhaps more than a little harsh. Regarding the assertion that a tangible grade will motivate students, Rice argues that the “clinical experience transcends students’ need for incentive.”19 He goes on to say that to “rely on grading as incentive for clinical students does either both of: patronising the students, as incapable of pursuing learning for its and their own sakes, and condemning the teachers, as incapable of inspiring students to do just that.”20 Whilst clinic can be the most invigorating and stimulating component of a degree, and may well be the sole motivator for some, it is difficult to suppose that this is true of all students. Perhaps where Rice goes awry is his submission that we rely on grading as a motivator. Is it not more accurate to say that it is a by-product of fully grading a module? When put into context and looking at the demands and constraints on today’s students, Rice’s Assessment– Are Grade Descriptors the Way Forward? 57 14 Stuckey, R, Best Practices, op. cit page 239 15 Brustin, S. L & Chavkin, D. F. (1997) ‘Testing the Grades: Evaluating Grading Models in Clinical Legal Education,’ 3 Clinical Law Review 299 – 336 at page 314. 16 Rice, S. op. cit. 17 Ibid. 18 Ibid. 19 Ibid. 20 Ibid. views may be criticised for being idealistic. Take for instance the notion that students might be incentivised and motivated by an actual grade, be it numerical or a letter grade, as opposed to a mere pass or fail credit. This argument seems feasible given the highly competitive and somewhat limited availability of training contracts or pupillages on graduation. By way of illustration, in order to qualify as a solicitor in the UK, the traditional route, having undertaken the academic stage of qualification, would be to complete a training contract of 2 years duration. In the UK in 2007 a total of 9,850 students had enrolled on the post graduate Legal Practice Course, the final stage academic requirement which renders a student eligible to undertake a training contract.21 However, in the year up to 31 July 2007 only 6,012 training contracts were registered with the Law Society.22 There is a clear shortfall in the number of training contracts available and the statistics are rendered yet more depressing when you take into account that graduates from previous years who have not been successful in securing a training contract will also be competing with the latest exiting cohort. If this were not competition enough, in 2007 of those who successfully completed the LPC, over one fifth of students were awarded a distinction and in excess of one quarter achieved a commendation, (the remaining students receiving a pass).23 For a student looking to enter a career in law in a climate where there are a disproportionately more candidates for training contracts than places, can we blame students for being assessment focussed? Arguably something has to act as a motivating factor for the student; if assessment promotes student engagement with the learning process is this so deplorable? We also have to consider that in clinic there is often a client involved and if students engage with that client’s problem but also work to the best of their ability, or beyond, to get the grade then that is preferable to a student who does not engage or achieve because it will not be recognised by a grade. Rice also claims that grading encourages surface learning as it places “greater value on learned skills and retained knowledge than on new thinking and awareness.”24 Whilst this may be true of traditionally taught subjects, clinical students do not have the opportunity to score highly from memorising and regurgitating lecture notes; the clinical pedagogy defies the surface learning approach. Furthermore, if the assessment incorporates a reflective element, then, on the contrary, this can be said to promote deep learning as the student will have considered their performance and the role of law from several perspectives. Another rationale for grading has its roots in the historical view that clinic is inferior to academic subjects and that to be elevated to the same stature, it must be graded.25 It has often been thought that clinic teaches skills rather than robust legal knowledge and consequently has not always been perceived as equal to non clinical subjects. Therefore assessing and fully grading students provides clinic with the same integrity as other degree subjects. Whilst this argument may at one time have 58 International Journal of Clinical Legal Education July 2009 21 Trends in the Solicitors’ Profession, Annual Statistical Report 2007, The Law Society at page 37. Available at http://www.lawsociety.org.uk/secure/file/174971/e:/te amsite-deployed/documents/templatedata/Publications /Research%20Publications/Documents/asr2007report. pdf. (accessed 12 August 2008). 22 Ibid, at page 37. 23 Ibid, at page 35. 24 Rice, op. cit at page 13. 25 Brustin, S. L & Chavkin, D. F. op. cit at page 301. been significant, the authors feel that given the increasing popularity of clinic, it is perhaps no longer a key concern as it may have been decades ago.26 Given the criticisms of assessment, discussed above, the SLO focus group was asked whether students should be assessed on a pass/fail basis. There was no support for this suggestion for several reasons. These included the notion that since students would graduate with both a law degree and postgraduate professional qualification, it was more befitting to award a mark as opposed to a pass/fail credit. It was also accepted that assessment can incentivise students to perform better, and since the supervisor’s practising certificate is potentially at stake, not to mention the client’s interests, it was thought that this was somewhat desirable. Conclusion It would seem overall that supervisors and students alike prefer the grade descriptors to the list of criteria. For supervisors it was felt that the grade descriptors afforded them some measure of the level that students should be achieving for a particular classification. For students the descriptors provided a solid base to work from and informed them of how they could achieve a better grade in the SLO. The grade descriptors also went someway to dispelling the fear held by some students that their grade was subjectively decided by a supervisor. It was agreed that the grade descriptors still required some further amendments and discussion but that they were a welcome move away from the criteria previously used, however the argument to retain an element of subjectivity in what is an individual assessment is still strong. The debate regarding the grading of clinic seems set to rumble on. For the SLO, due to the fact that it is a year long, compulsory subject that accounts for 40% of the students fourth year mark, it is our view that it has to be graded. This allows the student to demonstrate in tangible and meaningful terms their achievement in clinic. For clinics that are voluntary and/or not as intensive then there may be a more appropriate way to assess or recognise the student’s contribution. It would seem that there is still a long way to go in the debate over how to grade clinic and whether clinic should be graded. Overall our research indicated that, certainly for the SLO, grading using grade descriptors meant greater transparency and consistency and made grading less of an ambiguous art and more of a consistent science. Assessment– Are Grade Descriptors the Way Forward? 59 26 In 2006 of the 95 law schools surveyed in the UK, 53% were involved with pro bono activity, 12% intended to become involved in the following academic year and 8% were considering undertaking pro bono activity. This is a noticeable increase on the figures obtained in a similar survey in 2003. See Law Works Students Project Pro Bono – The Next Generation at page 3. Available at http://www.probonogroup.org.uk/lawworks/docs/Stu dent%20report%20Final.pdf