‘Pigs are not fattened by being weighed’ – so why assess clinic- and can we defend our methods? Carol Boothby1 Northumbria University, UK For those clinics that assess their students, there can be a panoply of issues to consider. The nature of clinic means that the experience of students is non- standardised, not least in terms of workload. Is it appropriate to assess such an experience? How can clinical teachers be sure that their assessment methods are valid and reliable? WHY ASSESS IN CLINIC? Perhaps because teaching and participating in a clinical experience can take such a wide variety of forms, the approaches to assessment have been similarly diverse. Many law schools have students involved in a range of pro bono activities, the majority of which will not be assessed. According to the LawWorks Law School Pro Bono and Clinic Report 2014,2 of those law schools that responded to the survey, 96% do pro bono work. This report suggests that (in the UK at least) clinics are increasingly becoming assessed as a credit bearing part of the curriculum and whereas previously only 10% of law schools in 2010 assessed student performance, 1 Carol Boothby is Director of the Student Law Office at Northumbria University 2 Carney, F. Dignan, R. Grimes, G. Kelly, G and R. Parker, “The LawWorks Law School Pro Bono and Clinic Report 2014”, LexisNexis. ( Special Issue Problematising Assessment in Clinical Legal Education ) ( 137 ) today this total is around 25% - relatively low, but apparently increasing, perhaps as clinics move from extra to intra curricular.3 Views diverge on the value of assessing clinic, as well as how to do so. In terms of the views of students, recent work by Combe indicates a minority (40%) responded negatively to the question “would you feel comfortable being assessed on law clinic work?”, suggesting that the majority were ‘either perfectly happy or indifferent to the prospect of assessment’.4 Brustin and Chavkin also found that the “overwhelming majority of the students believed that clinical courses should be available on a graded basis”, one student commenting that ‘grading permits rewarding those who make greater effort and excel…’.5 Other writers do not challenge assessment per se, but challenge the idea of grading. Rice argues that; ‘Grading undermines the collaborative role of the clinical teacher. This is not a journey where we arm students with a map and compass drop them in the wilderness, and give a prize to the first one home. This is a journey we travel with them, clearing the path ahead, holding back to 3 See D. Nicolson, “Legal Education or Community Service? The Extra-Curricular Student Law Clinic” [2006] 3 Web Journal of Current Legal Issues at http://www.bailii.org/uk/other/journals/WebJCLI/2006/issue3/nicolson3.html and subsequent exploration of the challenge of bringing clinic within the curriculum; D. Nicolson, “Calling, Character and Clinical Legal Education: Inculcating a Love for Justice from Cradle to Grave” (2013) 16(1) Legal Ethics 36. Also M. M. Combe (2014) “Selling intra-curricular clinical legal education” , The Law Teacher, 48:3, 281-295. 4 Combe, supra n. 2, at p. 292 5 Stacy L. Brustin and David F. Chavkin, “Testing the Grades: Evaluating Grading Models in Clinical Legal Education” (1997) 3 Clinical Law Review 299, p. 316. let them go ahead, offering them a steadying hand, coaxing them on narrow bridges over deep ravines, exhorting them to climb steep hillsides… grading distracts us from our teaching’.6 Hyams disagrees, seeing the reluctance to grade as ‘an evasion of our duty to our students’7and Levine sees pass /fail as not providing enough feedback to enable improvement.8 Nelson and Murray, reviewing the move to the use of grade descriptors at the clinic here at Northumbria Student Law Office , also challenge the case for pass/fail in clinic, arguing that grading recognises the efforts displayed by students and it motivates them to achieve. 9 Perhaps the idea that grading distracts from teaching is more likely where the assessment is summative in nature. Where supervisors are providing ongoing formative feedback, and where the method of assessment is fully aligned with the clinical work, assessment can drive learning. From the clinical supervisor’s point of view, one reason for assessing and grading could be that it isn’t enough to simply get students to a ‘pass ‘ level - we are wanting to help students to move along a continuum towards being ‘ practice ready ’- and perhaps for them also to have some awareness of how near or far they are from that. Stuckey’s definitive work, The Best 6 S. Rice, “Assessing - but not grading - clinical legal education”, Macquarie Law Working Paper No. 2007 – 16 available to download at http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1061622 (last accessed 29 October 2015) 7 R. Hyams, “ Assessing Insight: Grading Reflective Journals in Clinical Legal Education” (2010) 17 James Cook U. L. Rev. 25 p. 34. 8 M. Levine, “Toward Descriptive Grading” (1970-71) 44 California Law Review 696. 9 V. Murray and T. Nelson, “Assessment – Are Grade Descriptors the Way Forward? “ (2009) 14 International Journal of Clinical Legal Education 48, p. 57. Practices Report, suggests that assessment methods may have more influence on how and what students learn than any other factor.10 The benefit of assessment in providing information to students is touched on by Foxhoven; ‘Assessment is a powerful tool because law students uniformly desire to be prepared to become competent lawyers, but, being novices in the legal profession, they are unable to identify core competencies themselves.’11 Like a runner who checks their times and receives advice from their coach in order to improve their performance, students in clinic can (depending on the nature of the clinical experience) use formative assessment feedback to improve their performance. As Brown and Knight argue, ‘far from it being the case that you’ll not fatten a pig by weighing it… the science of weighing is necessary for the art of development’. 12 Assessment provides information about student learning – but a stronger claim (according to Brown and Knight) is that assessment shapes the curriculum; ‘Assessment defines what students regard as important ’.13 Coffield et al, in a comprehensive and critical examination of learning styles, refer to the work of Desmedt in finding that, ‘because of the curriculum, students are not 10 R. Stuckey et al. , “Best Practices for Legal Education: A Vision and a Roadmap “ (USA, Clinical Legal Education Association, 2007) p.266. 11 J. R. Foxhoven, “ Beyond Grading: Assessing Student Readiness to Practice Law” (2009) 16 Clinical Law Review 335, p. 344 12 S. Brown and P. Knight, “ Assessing Learners in Higher Education” (London, Kogan Page, 1994 ) p. 46 13 Ibid., at p. 12. interested in learning, but in assessment.’ 14 This may seem a depressing indictment of students, but surely it is not specific to students, but simply of human nature; if the way in which a race is run is judged on time, then no matter how much we may exhort a particular running style, unless this actually contributes to the goal of ‘best time’, it is likely to be discarded or ignored. For many clinicians in law clinics, particularly those driven by a social justice motive, (as Rice is) there is a risk that, unless the assessment and any grading leads to and measures progress in social justice terms, these aspects are merely a distraction. SO WHAT IS ‘GOOD’ ASSESSMENT? Arguably, a crucial factor underpinning all the support for assessing is how useful the assessment actually is in driving learning. What do we mean by ‘useful’? Taking forward the point about concerns over validity of assessment, this can be a perplexing area. One field that legal clinicians can (and have) drawn from is the medical profession. The use of problem based learning in the teaching of law has been derived in this way, as was the use of standardised clients and the training used in medicine continues to provide a rich seam of expertise. Those assessing medical students have puzzled over many of the same issues as legal clinicians. In particular, the work of Van der Vleuten, an academic in the field 14 F. Coffield et al. “Learning styles and pedagogy in post 16 learning: a systematic and critical review”. The Learning and Skills Research Centre, 2004. http://hdl.handle.net/1/273 (accessed 29th October 2015) not of medicine, but of education, led to him becoming the “accidental hero” 15 of medical education, who wanted to discover “promising ways to advance and to prevent repetition of the mistakes of the past in the future”, 16 moving away from high stakes assessment. In summary, Van der Vleuten uses a conceptual model for confirming the ‘utility’ 17- simply put, the ‘usefulness’ - of an assessment method by using a mathematical model incorporating key aspects such as validity, educational impact, and acceptability. This model can help us to examine what good assessment in clinic might look like, and this is a process which has been started at Northumbria University’s in- house law clinic, the Student Law Office. ASSESSMENT AT NORTHUMBRIA STUDENT LAW OFFICE Can the reflective and experiential elements of CLE be codified into assessment rubrics that provide guidance to students (and staff) without reducing their depth and complexity? At Northumbria, the law clinic moved in 2007-8 from criterion- referenced, outcome- focussed assessment to the use of 10 grade descriptors, including a range of skills and attributes from oral communication, written communication, to key skills such as a student’s ability to demonstrate autonomous 15 L. Pritchard “Cees van der Vleuten - Accidental hero” (2005) Medical Education Vol.39, Issue 8 p. 761. 16 L. W.T Schuwirth and C. van der Vleuten, “Changing education, changing assessment, changing research?” (2004) Medical Education Vol 38: p. 805. 17 C. P M Van der Vleuten and L W T Schuwirth, “Assessing professional competence: from methods to programmes” (2004) Medical Education Vol.39, Issue 3, p.309 learning. 18 There is no explicit ‘justice’ agenda within the clinic curriculum, although it is implicit in many of the activities and experiences. So how is the clinical training and assessment of students linked to the wider discourse of what a lawyer is, and can be? Through the development of reflection and reflective / reflexive practices, students have the opportunity to consider their own development, (see appendix A for assessment matrix). In addition to carrying out casework under supervision, and being assessed on these through the grade descriptors, students are currently required to produce two reflective pieces, 2000 words each, one on the development of their skills, and a second drawn from a list of topics such as ‘Law in Action’, ‘Clinic and my Career’, ‘Justice and Ethics’ and ‘Clinic and Legal Education’. These reflections are submitted with the portfolio evidencing their casework, at the end of the module. If challenged, how could we defend our use of our current form of assessment? Attacks can come from either end of the spectrum- those who see the social justice mission as too important for things like assessment 19 and grading 20 to get in the way, and others who worry that the experience of clinic is too non-standard and that this variety of experience needs to be narrowed into a check list of activities. The writer experienced such a challenge from an external examiner, who questioned the variability of the clinic experience, and the lack of control staff or students have over 18 Murray, supra n.7, pp. 48-60. 19 Nicolson, supra n.2. 20 Rice, supra, n.5. this, which provided much food for thought, reflection, and a useful opportunity to critique and justify our existing methods. It brought a realisation that it may not be enough to rely on the mantra of clinic being so good that challenging assessment validity is a heresy. On the other hand, it cannot be that simply because clinic is a non-standard experience, (arguably one of the reasons students engage with it) and because this makes assessment difficult, that we give up either on clinic within the curriculum or assessment of it. Being able to deconstruct and critique clinical methods, including assessment tools, should help to understand our clinical teaching more deeply- and perhaps also to see it from the students’ point of view, in terms of alignment and authenticity. So if non standardisation is one purported challenge to the validity of assessment in clinic, what are the other potential components of validity? Van der Vleuten’s work based on the ‘utility model’ gives a framework within which to carry out a methodical examination of our use of assessment. THE UTILITY MODEL Van der Vleuten uses the idea of ‘utility’ as a conceptual model, whereby criteria are multiplied together to produce a utility index. Those criteria can include; 1. Validity (does an assessment instrument measure what it purports to?) 2. Reliability (can scores for an assessment be reproduced ) 3. Educational impact – the impact of assessment on learning 4. Acceptability to stakeholders/Cost – in terms of resources He makes 2 further points; that · Selecting assessment methods involves context-dependent compromises · Assessment is not a measurement problem but an integrated design problem21, made up of educational, implementation and resource aspects. What was already known was that the usefulness of assessment depends on compromise between various quality parameters, but what Van der Vleuten highlights in his later work is that ; 1. ANY method of assessment may have ‘utility’, depending on use 2. We need more methods using qualitative information relying on professional judgement, the latter being highly valuable. 3. Assessment is an ‘educational design problem’22 that needs a holistic approach. In terms of reliability, here lies the importance of sampling, by which Van der Vleuten appears to mean that because competence is highly dependent on context or content, we need to use a large sample across the content of the subject to be tested, particularly if there are other potential effects on reliability, such as, in the case of clinic, clients, and single supervisor. This has relevance for assessment at Northumbria SLO, where it could be argued that, through the use of a wide range of grade descriptors, and ongoing assessment, we compensate to some extent for the ‘single supervisor ‘ aspect- but is that enough? The Northumbria SLO clinic assessment includes a thorough moderation process, where a sample of each supervisor’s marking is examined by a different supervisor. But we do not directly 21 Van der Vleuten, supra n.17, p.310. 22 Van der Vleuten, supra n.18, at p.314. involve more than one supervisor in the clinic assessment. Further, there is no real link between the practical work, the grade descriptors and the 2 reflective pieces, save that these pieces purport to be a reflection on the clinical experience. In reality, students treat these as an end point assessment, and for many it seems to take until the end of the clinic module for them to grasp what is required. Therefore the use of reflections as part of the clinical assessment is currently being re-examined, and Van der Vleuten’s framework has provided a useful structure. ASSESSING REFLECTIVE PRACTICE There is widespread use of reflections as a key part of clinic assessment; some clinics incorporate a presentation as well as written work, but no assessment of casework carried out by students.23 Others are all written but include formative pieces such as a ‘critical incident’ report.24 All appear to embrace the concept of reflection with gusto, although there have been critiques of the use of reflections.25 In looking to apply Van der Vleuten’s work on assessing competence, the area of student reflection is one which has been of concern to clinic staff. In 2013, the writer introduced the reflections matrix (Appendix B) to Northumbria SLO assessment, 23 J. McNamara, “Validity, Reliability and Educational Impact of Reflective Assessment in Clinical Legal Education”, presented at ALT seminar, 4 June 2015, https://www.northumbria.ac.uk/about- us/academic-departments/northumbria-law-school/law-research/legal-education-and-professional- skills/problematising-assessment-in-clinical-legal-education/ (last accessed 29th October 2015) 24 R. Spencer, “Holding Up the Mirror: A Theoretical and Practical Analysis of the Role of Reflection in Clinical Legal Education” (2012) International Journal of Clinical Legal Education, No. 18, pp. 181-216, available at SSRN: http://ssrn.com/abstract=2482873 (last accessed 29th October 2015) 25 J. Tummons, “‘It sort of feels uncomfortable’: problematising the assessment of reflective practice”, (2011) 36:4 Studies in Higher Education , pp. 471-483. providing rubrics to help clinic supervisors in assessing, and feedback from these assessors was generally positive, but other aspects of the way in which reflection was assessed remain the subject of concern, for example, the structure of the reflections as being essay-style pieces, submitted at the end of the module. Testing our assessment in clinic against the utility model using grid structured questions provides a structure for discussion. As an overview, the grid below can help to summarise Van der Vleuten’s approach, and enable a critique of current or proposed assessment practice. Utility model applied to SLO practical work using grade descriptors Element of trustworthiness Criteria To what extent is this achieved through current assessment strategy in the Northumbria clinic? high low Credibility (internal validity) Aligned with stage of competency (Miller’s triangle) Complex tasks/requiring mastery of skills, similar to legal practice Authentic integration of competencies at each stage Good integration of legal skills Lack of real integration of reflective practice Structural coherence within the programme Grade descriptors align to the skills required for clinical casework Some coherence and alignment of reflective work, but could be improved Prolonged engagement, triangulation and member checking Good training of assessors (clinic supervisors) Limited involvement of more than one supervisor(only at moderation) Transferability (generalizability) Time sampling Judgement based on broad sample of data points , repetition of tasks Thick description Assessors justify decisions in detail Dependability (reliability) Domain or content specificity Test across content and contexts, over time Awareness of complicating conditions Awareness of Impact of different clients/case complexity Actual Impact of different clients/case complexity Stepwise replication through sampling Uses assessors who have credibility Only one assessor used per students Confirmability (objectivity) Review and audit Detailed moderation process, with marks being sampled – peer review No real possibility for students to appeal or challenge the assessment decision For supervisors less familiar with the terminology used by Van der Vleuten, a more user friendly approach asks the question; ‘What would failure to meet/ meeting/ exemplary practice in relation to this criterion look like?’. A pilot group using this table plus a brief explanation of Van der Vleuten’s work were able to engage with a valuable critique of our current assessment of reflections. Points raised in relation to the current clinic assessment at the Student Law Office were: Competency- we would expect students to be able to reflect at a reasonably sophisticated level – but have we provided sufficient previous experience and support to raise their reflective skills to the level of study they were at, which is Masters level (level 7)? Integration of competencies - the use of end-point essay-style pieces for assessment of reflection separates reflective practice from the ongoing development and mastery of complex legal skills, so that learning and development of competency in reflection is not perceived by students in the same way as their development of those legal skills. This is reinforced by the contrast between the high level of formative feedback provided for practical casework, and the limited opportunities built in to the assessment for the purposes of reflection. Structural coherence within the programme- The reflections matrix sets out the way in which the written piece will be marked, but this does not link to or facilitate an ongoing reflective process- and perhaps fails to assess authentic growth in reflective skills. At a programme level, it could be argued that there is little prior preparation for the development of reflective skills. Prolonged engagement, triangulation and member checking – the current perception of the reflective pieces as ‘end point’ led them to be summative in nature. In reality, students can prepare them during the year, but the only point at which they have the opportunity to gain supervisor feedback is at the mid year appraisal, when students submit a one-page draft. There is little triangulation, in the sense that the reflections are freestanding pieces of writing. The use of a different format such as presentations might provide an opportunity to engage with students directly and assess the level of true understanding and genuine reflection. CONCLUSIONS The good news for clinicians is that, as Van der Vleuten says, there is ‘no need to banish from our toolbox assessment instruments that are rather more subjective and not perfectly standardised, on condition that we use them sensibly and expertly. We can move assessment back to the real world of the workplace as a result of the development of the less standardised but nevertheless reliable methods of practice based assessment’.26 Authenticity is valued, as is the role of professional judgment by those assessing. Tasks should be treated in a holistic rather than reductionist way. We need to ‘construct an overall judgment by triangulating information across these sources’ 27– perhaps something analogous to the way in which judicial judgements are reached. A thoughtful and informed approach to assessment in authentic learning environments such as law clinics should enable this assessment process to be both informative in terms of student development and reliable as a measure of achievement. 26 Van der Vleuten, supra n. 18, p. 312. 27 Van der Vleuten, supra n. 18, p.313. Appendix A: Grade descriptor for Student Law Office (The assessment criteria are equally weighted.) ( Special Issue Problematising Assessment in Clinical Legal Education ) ( 151 ) Assessment Criteria Grade descriptor F(below 50) 2:2 (50-59) 2:1(60-69) 1st (70-79) +1st (80+) Autonomy and efficiency Knowledge and understanding of the law / legal practice Strength of oral communication skills Poor initiative shown; routinely relies on supervisor / routinely requires instruction / routinely requires prompting / requires prompting significant correction of work Poor knowledge and understanding of law / legal practice issues; rarely draws on appropriate prior knowledge or legal principles Poor oral communication skills indicating enduring difficulties in articulating legal and factual material; regularly fails to plan, listen or adapt to the needs of the audience Fair/reasonable initiative shown, and often relies on supervisor / often requires instruction / often requires prompting / often needs significant correction of work Fair/reasonable knowledge and understanding of law / legal practice issues but little thinking across subject disciplines; sometimes draws on appropriate prior knowledge or legal principles Fair/reasonable oral communication skills; sometimes shows strong ability to articulate legal and factual material, plans, listens and adapts to the needs of the audience Good initiative shown but there is some evidence of the following; reliance on supervisor / requirement for instruction / prompting / significant correction of work Good knowledge and understanding of law / legal practice issues including thinking across subject disciplines; regularly draws on appropriate prior knowledge or legal principles Good oral communication skills; regularly shows strong ability to articulate legal and factual material, plans, listens and adapts to the needs of the audience Very good initiative shown and there is little evidence of the following; reliance on supervisor / requirement for instruction / prompting / significant correction of work Very good knowledge and understanding of law / legal practice issues including thinking across subject disciplines; routinely draws on appropriate prior knowledge or legal principles Very good oral communication skills; routinely shows strong ability to articulate legal and factual material, plans, listens and adapts to the needs of the audience Excellent/outstanding initiative shown, and the following are extremely rare; reliance on supervisor / requirement for instruction / prompting / significant correction of work; a very high level of trust and responsibility can be given Excellent/outstanding knowledge and understanding of law / legal practice issues including thinking across subject disciplines; almost always draws on appropriate prior knowledge or legal principles; stretches supervisor’s own understanding Excellent/outstanding oral communication skills; almost always shows strong ability to articulate legal and factual material, plans, listens and adapts to the needs of the audience; instils confidence in clients Strength of written communication skills Poor written communication skills; rarely shows clarity, precision and accessibility; drafts routinely require significant amendment Fair/reasonable written communication skills; sometimes shows clarity, precision and accessibility; drafts often require significant amendment Good written communication skills; regularly shows clarity, precision and accessibility; drafts sometimes require significant amendment Very good written communication skills; routinely shows clarity, precision and accessibility; drafts rarely require significant amendment Excellent/outstanding written communication skills; almost always shows clarity, precision and accessibility; drafts very rarely require significant amendment; excellent sentence and paragraph structure displays eloquence Strength of research skills Commitment to clients and the Student Law Office Poor research skills; rarely shows appropriate depth, detail and comprehensiveness; reports rarely display effective practical awareness and application Demonstrates little commitment or enthusiasm for achieving the best solution for clients; rarely puts more than the minimum required to perform tasks; completes insufficient work Fair/reasonable research skills; sometimes shows appropriate depth, detail and comprehensiveness; report sometimes display effective practical awareness and application Demonstrates some commitment or enthusiasm for achieving the best solution for clients; performs tasks with fair/reasonable diligence; completes sufficient work Good research skills; regularly shows appropriate depth, detail and comprehensiveness; reports regularly display effective practical awareness and application Demonstrates a good level of commitment or enthusiasm for achieving the best solution for clients; performs tasks with a high degree of diligence and shows pride in the work; completes sufficient work and shows willingness to help further Very good research skills; routinely shows appropriate depth, detail and comprehensiveness; reports routinely display effective practical awareness and application Demonstrates a very good level of commitment or enthusiasm for achieving the best solution for clients; performs tasks with a very high degree of diligence and shows pride and zeal for the work; completes sufficient work and actively seeks to help further Excellent/outstanding research skills; routinely shows appropriate depth, detail and comprehensiveness; reports almost always display effective practical awareness and application; research addresses problems holistically Demonstrates an excellent/outstanding level of commitment or enthusiasm for achieving the best solution for clients; performs tasks with an excellent degree of diligence and shows pride and zeal for the work; completes sufficient work and goes the extra mile for clients and the Student Law Office; Supervisor has to work hard to keep up Case management and strategizing Organisation: time and file management Cases are progressed poorly; very few ideas about cases are offered or are poorly formed and not thought through; there is little or no evidence of proactivity or thinking about the overall strategic direction of clients’ cases Displays poor organisational skills; makes little effective attempt to manage time; regularly fails to anticipate how long tasks will take or to plan use of time effectively; late on more than three occasions; files are often disorganised and not up to date; copes poorly under pressure and fails to achieve results when time is of the essence Cases are progressed reasonably; some ideas about cases are offered – these are sometimes poorly formed or not thought through; there is some evidence of proactivity or thinking about the overall strategic direction of clients’ cases but this tends to be limited and lacking imagination / insight Displays fair/reasonable organisational skills; makes a real attempt to manage time; sometimes fails to anticipate how long tasks will take or to plan use of time effectively; late up to three occasions; files are reasonably well organised but inconsistent and are sometimes not up to date; struggles under pressure but manages this Cases are progressed effectively; quite a few ideas about cases are offered – these are often well formed and thought through but with inconsistency; there is good evidence of proactivity or thinking about the overall strategic direction of clients’ cases with some imagination / insight Displays good organisational skills; makes a good attempt to manage time; generally anticipates how long tasks will take and plans use of time effectively but with some defects; late up to two occasions; files are well organised and up to date with few significant defects; copes well under pressure Cases are progressed highly effectively; lots of ideas about cases are offered – these are regularly well formed and thought through; there is very good evidence of proactivity and clear thinking about the overall strategic direction of clients’ cases with imagination / insight Displays very good organisational skills; makes a very good and sustained attempt to manage time; routinely anticipates how long tasks will take and plans use of time effectively with few defects; late up to one occasion; files are very well organised and up to date with very few significant defects; copes very well under pressure Cases are progressed excellently; lots of ideas about cases are offered – these are routinely well formed and thought through; there is excellent/outstanding evidence of proactivity or thinking about the overall strategic direction of clients’ cases with imagination / insight; routinely thinks “outside the box” which generates creative potential solutions to problems Displays excellent/outstanding organisational skills; almost always displays excellent time management; almost always anticipates how long tasks will take and plans use of time effectively with no significant defects; never late; files are excellently organised and up to date with no significant defects; organisational skills reveal a calm, unhurried attitude that can easily cope with significant pressure Teamwork skills and contribution to firm meetings Understanding of client care and professional conduct Poor working relationship with Supervisor / partner / peers; ineffective or negligible or disruptive contribution to firm meetings; may sometimes fail to attend firm or other meetings; relies heavily on other people to achieve client goals Displays a poor understanding of professional obligations; fails to take client care procedures seriously or fails to ascertain the appropriate office procedure; commits a significant breach of the Code of Conduct or error of professional judgment Fair/reasonable working relationship with Supervisor / partner / peers; some effort to contribute to firm meetings but mainly reactive / focused on own cases; contributes to achievement of client goals but provides limited support to others and little leadership Displays a fair/reasonable understanding of professional obligations; tries to comply with client care procedures but requires significant guidance; follows basic office procedure but is not always fully aware of the significance of this; struggles to articulate the rationale for ethical rules; treats clients well Good working relationship with Supervisor / partner / peers; good effort to contribute to firm meetings including discussions of other people’s cases and general discussions; contributes to achievement of client goals; provides ideas and support to others and some leadership Displays a good understanding of professional obligations; complies with client care procedures with limited guidance; follows office procedure and is aware of the significance of this; is capable of articulating the rationale for ethical rules; treats clients with care and respect Very good working relationship with Supervisor / partner / peers; very good, creative contribution to firm meetings including discussions of other people’s cases and general discussions; contributes fully to achievement of client goals; provides ideas and support to others and effective leadership but does not dominate others Displays a very good understanding of professional obligations; complies precisely with client care procedures with very little guidance; follows office procedure and is fully aware of the significance of this; clearly articulates the rationale for ethical rules and appreciates the context of SLO service; treats clients with a high degree of care and respect Excellent/outstanding working relationship with Supervisor / partner / peers; excellent, creative contribution to firm meetings including discussions of other people’s cases and general discussions; contributes fully to achievement of client goals; provides ideas and support to others and strong leadership but does not dominate others; embraces the notion of mutual assistance in clients’ best interests Displays an excellent/ outstanding understanding of professional obligations; complies precisely with client care procedures with no significant guidance; follows office procedure and is fully aware of the significance of this; clearly articulates the rationale for ethical rules and appreciates the context of SLO service; treats clients with a high degree of care and respect; makes clients feel the utmost confidence that their best interests are being served Appendix B Reflections matrix Student Law Office Third/fail (below 50) Lower Second (50-59) Upper Second(60-69) First/strong first (70+) Reflective No significant analysis or reflection on the Fair analysis and reflection on the Good analysis and reflection on the Excellent relevant in depth Analysis topic topic. Using some detailed topic. Specific and personal, using analysis and reflection on the examples but primarily descriptive some detailed examples, showing topic. Specific and (where with a lack of development or good ability to synthesise and appropriate ) personal, using analysis. evaluate information and ideas detailed examples showing excellent ability to synthesise and evaluate information and ideas (Self) Exhibits little or no self-awareness, Exhibits fair/reasonable levels of Exhibits good levels of self- Exhibits high /very high levels of Awareness and generalises experiences, fails to take into self-awareness, but some awareness, avoids generalisation of self-awareness, avoids insight (where account other perspectives or examine generalisation of experiences, experiences, often takes into account generalisation of experiences, appropriate, potential value sometimes takes into account other other perspectives and examines always takes into account other dependent on perspectives and examines potential value perspectives and examines the topic) * potential value potential value. Evidence of development/learning and future development/learning needs Context No evidence of relevant knowledge or Little evidence of relevant Some evidence of independent Good/ Excellent evidence of (Knowledge of independent reading. knowledge. Relies solely on reading such as books or journal independent reading such as relevant personal anecdote. articles. books or recent journal articles material and which supports the reflection and sources) or provides context Clarity of Not always clear what was intended. Very Some points may not be expressed Most points expressed clearly and All points expressed clearly and expression poor style. Extensive grammar or clearly. Poor style. A number of succinctly. Mainly engaging and succinctly. Engaging and vocabulary errors grammar or vocabulary errors. comprehensible style. Mainly correct comprehensible style. Correct grammar and vocabulary grammar and vocabulary Organisation Little or no organisation of the material Clear organisation of material but Very clear organisation of material. Excellent organisation of the at times the transitions are unclear. material, forming a coherent whole. *this may be slightly less relevant in some of the optional titles, such as Clinic and Legal Education